I'm a very recent (3 days) user of scratchbox2 for cross-compiling. My initial response was "this is crazy magic" awe, followed shortly by "this is a crazy mountain of hacks" skepticism. However, after some use & success now, it seems to be a really neat tool.

The best introductions to scratchbox2 that I've found are this wiki page from the plugcomputing people, and this nice succinct set of presentation slides from Riku Voipi.

One of the big annoyances has been seeking information, Google likes to replace sb2 (the main scratchbox2 command) with db2 and scratchbox2 with scratchbox.

Multiarch

I have found that scratchbox2 (as of 2.3.3) is a bit fiddly when targeting Debian wheezy or sid (testing or unstable.) This is due to the new Debian Multiarch support.

In a nutshell, multiarch means libraries and tools are moving from locations like /usr/lib to arch-specific locations like /usr/lib/[triplet] where triplet identifies the architecture, ie arm-linux-gnueabi for ARM systems.

The symptoms I saw were linking errors (or configuration failures) due to missing libraries, when you know they are installed on the target. The crucial test is whether "sb2 [some-simple-command]" fails while "sb2 -e [same-simple-command]" passes - this implies that the problem is in the host translation, not in the target (otherwise it would fail in the -e qemu'd environment, as well.)

Here's what worked for me. In the file \~/.scratchbox2/[TARGETNAME]/gcc.config.lua, I sought out and changed the following two config items as follows, to add linker search paths:

extra_cross_compiler_args="   -L@SBOX_TARGET_ROOT@/usr/lib -L@SBOX_TARGET_ROOT@/usr/lib/arm-linux-gnueabi -L@SBOX_TARGET_ROOT@/lib  -L@SBOX_TARGET_ROOT@/lib/arm-linux-gnueabi",

extra_cross_ld_args="-rpath-link @SBOX_TARGET_ROOT@/usr/lib/arm-linux-gnueabi:@SBOX_TARGET_ROOT@/usr/lib:@SBOX_TARGET_ROOT@/lib:@SBOX_TARGET_ROOT@/lib/arm-linux-gnueabi",

I also ran the command

sb2-config setenv PKG_CONFIG_PATH /usr/lib/pkgconfig/:/usr/lib/arm-linux-gnueabi/pkgconfig/

(In all the above, replace arm-linux-gnueabi with your target architecture's "triplet" of choice.)

As always, the solutions are simple and easy to follow once you know what they are. :) Please bear in mind also that this may not be the best solution, I'm still a scratchbox2 newbie.

Distro Versions

The other major gotcha, hinted at in the presentation I linked to, is that the host and the target distros must match exactly. If you have different library versions on each then you are headed for problems.

This can be tricky even when the distros match. At time of writing debian wheezy uses libc6-dev 2.13 on amd64 but emdebian wheezy uses libc6-dev-armel-cross 2.11 for the cross-compiler. The header files are different enough to cause builds to fail. To get matching headers I had to change to emdebian sid, which currently has libc6-dev-armel-cross 2.13.

An alternative, also hinted at in the presentation, is to set up a "tools" distro. This is a directory with a complete host-arch distribution, matching perfectly the target's distro except for the architecture. It seems the idea there is to create a controlled single-purpose environment, instead of just sharing the host's tools. Maybe that's a cleaner path, especially if it lets you sandbox a stable distro release with predictable versions.

(Extended Rant, Feel Free to Ignore)

The header-mismatch problem seems to be caused because programs compiled under sb2 can search for headers in both the host directory /usr/include/, and also in the cross-compiler directory (ie /usr/arm-linux-gnueabi/include/), -and- in the target rootfs directory.

Exactly which header file gets chosen for which purpose seems to be somewhat haphazard - in the example I saw, a #include <fcntl.h> was pulling in /usr/include/fcntl.h which was then pulling /usr/arm-linux-gnueabi/include/bits/stat.h.

I haven't dug deep enough into sb2 to see what's really going on here with gcc include paths, but it seems like it should be possible to workaround this and ignore the host's headers entirely - in favour of the cross-compiler headers & those installed in the target rootfs.

However, passing these paths explicitly is exactly the kind of drudgery that sb2 tries to avoid with its "magic", so at this stage I'm leaving it - my program compiles & links, so that's good enough for now. If I end up having to explicitly set everything then I may as well create a conventional cross-compiler environment.

Thoughts on “Notes using scratchbox2 with debian multiarch