When you work on a piece of software, you usually want to be able to build and test it manually on your local system, but without compromising your system or destabilizing the distribution provided software. How do you install the dependencies you need that are not found in your current distribution, and how do you test your project if you need to install it first? This article explains what I do with Weston, and notes a related but important pitfall with my particular approach.

All the dependencies that are provided by the distribution I naturally install from the distribution. Just install the relevant dev or devel packages, that is clean and easy. For those that are not found in the distribution I install manually into a special prefix under my home directory. Such prefix needs a little environment setup to work.

Installing to /usr/local, which is the default installation path in the most popular build systems, is risky and I practically never do that. There are several reasons:

• It requires root privileges to install to.
• Most distributions set up their normal environment such that software from /usr/local is automatically used. Therefore installing a common library like Mesa in there will get the library used by your whole system, instead of only by the piece of software you intended to run. This can easily break your normal working environment.
• If the system uses software from /usr/local, it means the same libraries from the distribution packages get ignored. You can install all the updates from your distribution you want, but in reality you are stuck with what you happened to install to /usr/local. In other words, you stop getting security updates and bug fixes.

Installing into a prefix under your home directory is really convenient and keeps things contained, but there are cases where it does not really work (or cases I have not bothered to make work). If a project needs to install some user or system-wide settings, like systemd units or some D-Bus things, those are obviously not going to work from an arbitrary user directory.

I tend to install everything related into the same custom prefix. It would also be possible to install every dependent project into a prefix of its own, but then the environment setup for your main project gets more tedious. Doing that could be worth it though, so you can find out if you miss some header or library paths from your project build files.

Environment setup

To help with setting up the environment for a prefix in my home directory consistently, I use the following Bash script.

projectshell.bash:

function concatenate_colon {
	local IFS=':'
	echo "$*"
}

function add_export_env {
	local VAR="$1"
	shift
	local VAL=$(eval echo "\$$VAR")
	if [ "$VAL" ]; then
		VAL=$(concatenate_colon "$@" "$VAL")
	else
		VAL=$(concatenate_colon "$@")
	fi
	eval "export $VAR=\"$VAL\""
}

function prefix_setup {
	local PFX="$1"

	add_export_env PATH "$PFX/bin"
	add_export_env LD_LIBRARY_PATH "$PFX/lib" "$PFX/lib64" "$PFX/lib/x86_64-linux-gnu"
	add_export_env PKG_CONFIG_PATH "$PFX/lib/pkgconfig/" "$PFX/lib/x86_64-linux-gnu/pkgconfig" "$PFX/share/pkgconfig/"
	add_export_env MANPATH "$PFX/share/man"
	export ACLOCAL_PATH="$PFX/share/aclocal"
	export ACLOCAL="aclocal -I $ACLOCAL_PATH"
}

function projectshell {
	local PROJECTSHELL="$1"

	case "$PROJECTSHELL" in
	wayland)
		export WLD="$HOME/local"
		prefix_setup "$WLD"
		add_export_env LD_LIBRARY_PATH "$HOME/install/xkbcommon/lib/x86_64-linux-gnu"
		add_export_env PKG_CONFIG_PATH "$HOME/install/xkbcommon/lib/x86_64-linux-gnu/pkgconfig"

		export MESA_DEBUG=1
		#export EGL_LOG_LEVEL=debug
		export LIBGL_DEBUG=verbose
		#export WAYLAND_DEBUG=1

		export XDG_CONFIG_HOME="$WLD/etc"
		export TOYTOOLKIT_CURSOR_THEME=Vanilla-DMZ
		#cd ~/git/weston
		;;
	home.git)
		export GIT_DIR="$HOME/git/home.git"
		export GIT_WORK_TREE="$HOME"
		;;
	weston-opt)
		export WLD="/opt/weston"
		prefix_setup "$WLD"
		;;
	*)
		echo "Warning: unknown projectshell '$PROJECTSHELL'."
		;;
	esac
}

Sourcing that script in your shell will just define a few functions. Function projectshell has my project specific environment definitions. "wayland" is for Weston development, "home.git" is for managing parts of my home directory with git, and "weston-opt" is yet another Weston prefix for real use rather than development.

The script sets up a bunch of environment variables, most importantly PATH, LD_LIBRARY_PATH and pkg-config path. This way when I build or run something, the prefix is searched first. Some of the paths are a bit of a jumble, since I created this script on Gentoo and later migrated to Debian, and e.g. library paths are slightly different. It more or less follows Installing in a custom location.

To make use of that script, my .bashrc has the following lines:

source ~/scripts/prompt.bash
source ~/scripts/projectshell.bash

add_export_env PATH "$HOME/bin" "$HOME/.local/bin"

if [ "$PROJECTSHELL" ]; then
	projectshell "$PROJECTSHELL"
	prompt_pq "($PROJECTSHELL) "
else
	prompt_pq ""
fi

There, projectshell.bash is given above (but cleaned up, I actually have a lot more different environment definitions). prompt.bash is not given here, but it only defines the function prompt_pq that gives a nice shell prompt containing the given argument, so that I know which environment I am in.

When I want to enter a specific environment, I can do:

$ PROJECTSHELL=wayland bash

which starts a nested shell with my environment set up, and the prompt to indicate it.

I only use that from the local text console (virtual terminal) though. When I work in a graphical environment, I have different launchers for different environments in a terminal emulator. The command in one of them is:

PROJECTSHELL=wayland urxvt -tint Green

Not only the shell prompt tells me my environment, the terminal window background is also tinted for it.

Working inside the environment

Using the environment is easy, there is only one thing to do specially. When you configure a build of a project, set the prefix, e.g.

$ meson mybuilddir --prefix=$WLD

This makes the project install into the prefix in my home directory. All dependencies are automatically found from the prefix if they exist there, because of the environment variables. Of course, Meson has configuration options to set up pkg-config search paths, and you really should consider using them. My setup here is from autotools era.

The pitfall

This setup has one grave pitfall that is going to bite you if you don't know about it, and it will baffle you when it hits you the first time.

Because the script sets up LD_LIBRARY_PATH, the installed libraries from the prefix may be used before the libraries from the build directory (for more details, see the man page of ld.so: DT_RPATH vs. DT_RUNPATH). This means that if you run a program like a weston test from the build directory without running ninja install first, it will use an outdated libweston from the prefix and not the current libweston from the build directory.

I work around that with a script that cleans up every trace of weston from my prefix. I have a habit of doing things like

$ git rebase -i --exec 'cd build && ninja && meson test' origin/master

to run the test suite against every commit in a branch, and that will not do the right thing if weston is installed in the prefix. Alternatively I could just do ninja install, but then I will not be sure that running the tests from the build directory works without installing.

Final words

The approach here builds heavily on environment variables, which is no wonder since I started this when all projects I needed were using autotools. Therefore it is important that whenever you touch any project at all, always make sure to set up the environment first! That includes simply running things from your prefix.

There is no commmon convention about which environment variables each build system will remember or forget and when. Keeping things consistent is up to you. So make a habit of always using the environment setup, and to support that habit make the environment easy to enter.