Did you know you could run a permissively-licensed MTP (Media Transfer Protocol) implementation with minimal dependencies on an embedded device? Here's a step-by-step guide on how to easily run cmtp-responder on a Rock Pi 4 or any other board equipped with a UDC.

To recap: In part1 of this series I introduced you to the concept of USB gadgets, their configfs composition interface, available opensource tools and basic systemd integration. In part2 I wrote about one particular USB gadget function - FunctionFS - and its integration with systemd. Then I presented cmtp-responder, a permisively-licensed MTP responder implementation and showed how to play with it on your PC with dummy_hcd driver. It is in this latter post that I promised you running cmtp-responder on real hardware. You can also watch me talking about USB gadgets at ELC 2019 and Linux Piter 2019.

The scope

In this post I won't go into details of how to compose a gadget or how to integrate that process with systemd. The first paragraph contains enough links for you to follow, and I encourage you to do it if you haven't done so yet. What I will talk about today is how to build the needed components for running on real hardware.

As far as the hardware is concerned, you can run your gadgets on any Arm board which provides a UDC (OTG) chip which is wired to an actual USB connector on the board (so RPi Zero will work with this, but RPi will not). It is definitely beyond the scope of this post to talk about all possible pieces of hardware. The hardware I am using is a RockPi 4 model B, but you can easily adjust the instructions to your particular board and I will mark all the places which are hardware-specific.

The software pieces which you need to build specifically for your target board are:

• libusbgx
• gt
• cmtp-responder
• linux kernel.

Systemd units and the gadget scheme are obviously platform-independent. So let's go step by step through building the platform-dependent pieces.

Options to build natively

First of all, you can build natively on the target board. It will definitely take more time than cross-compilation, but is a viable option. For building natively you have yet another option, you can compile on your PC in an emulated environment (which does not need to be full system emulation, just a target rootfs and a static qemu inside it is enough to chroot into and build as if you were building natively). In either case the process should look like building for an x86 on an x86, and you need the same kind of packages installed on the board (or in your rootfs). If, however, you expect you will be rebuilding cmtp-responder often, cross-compiling it is definitely worth the initial effort and the rest of this post is dedicated to this latter approach.

Cross-compiling in general

What we want to achieve is to build a piece of software so that it runs on a different kind of machine than the one used for building. This process is known as cross-compiling. No matter if we need to use autotools style (libusbgx), cmake style (gt, cmtp-responder) or, say, meson style, the general concepts are the same.

You need a cross-compiler in the first place. It is a compiler which itself is a program built for running on your build host and generating binaries for the target host.

Unless you are compiling e.g. a kernel, you cannot get away with just a bare compiler. While it is doing its job it needs to consult several files, e.g. library header files, which should match those in the target host rather than those in your build host. That is why you need sysroot - more on sysroot below. Some libraries have their files installed in a known place, but Linux distributions can choose their custom locations. That is why a program called pkg-config exists. Instead of hardcoding library files locations into your build files, you put pkg-config invocations there. Please note that pkg-config itself has nothing to do with your distribution's packaging system (deb, rpm etc.). Of course, your development libraries (and pkg-config itself!) can be provided by your distribution packages, but once installed, the way to find libraries while compiling your programs is to use pkg-config - and you do it the same way regardless of which distribution you are using (or if you are not using any distribution at all, but compile the whole system from sources).

When invoking your cross-compilation toolchain you usually need to point it to several places:

• sysroot
• path for pkg-config

You use sysroot so that your cross compiler knows where to look for certain files specific to your target. The sysroot needs to mirror your target rootfs's libraries and headers. You can just as well point at your actual target rootfs if you e.g. mount it over NFS on your target board.

You use PKG_CONFIG_PATH so that pkg-config knows where to look for its files specific to your target.

With these prerequisites in place let's finally get to actually building our stuff. I assume Debian-based systems on host and target. If you use different distribution or no distribution at all adjust the commands used to install needed packages.

Cross-compile libusbgx

# adjust your SYSROOT accordingly
$ export SYSROOT=/home/ap/Collabora/rootfs/arm64/debian-testing

$ sudo apt-get install pkg-config (in the sysroot/target rootfs)
$ sudo apt-get install libconfig-dev (in the sysroot/target rootfs)

$ git clone https://github.com/libusbgx/libusbgx.git
$ cd libusbgx
$ autoreconf -i

# subdirectory under usr/lib is platform-specific, so is --host=
$ PKG_CONFIG_PATH=$SYSROOT/usr/lib/aarch64-linux-gnu/pkgconfig ./configure --host=aarch64-linux-gnu --prefix=/usr --with-sysroot=$SYSROOT

$ make CFLAGS="--sysroot=$SYSROOT"
$ sudo make DESTDIR=$SYSROOT install

Cross-compile gt

$ sudo apt-get install asciidoc-base
$ sudo apt-get install libglib2.0-dev
$ sudo apt-get install libsystemd-dev

$ git clone https://github.com/kopasiak/gt.git
$ cd gt/source

# create a toolchain file with the below contents
# CMAKE_SYSTEM_PROCESSOR, CMAKE_C_COMPILER and PKG_CONFIG_LIBDIR are platform-specific
$ cat aarch64-toolchain.txt 
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

# adjust accordingly
set(CMAKE_SYSROOT /home/ap/Collabora/rootfs/arm64/debian-testing)

set(CMAKE_C_COMPILER aarch64-linux-gnu-gcc)

set(ENV{PKG_CONFIG_LIBDIR} "${CMAKE_SYSROOT}/usr/lib/pkgconfig:${CMAKE_SYSROOT}/usr/lib/aarch64-linux-gnu/pkgconfig")`
set(ENV{PKG_CONFIG_SYSROOT_DIR} ${CMAKE_SYSROOT})

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

$ cmake -DCMAKE_INSTALL_PREFIX=$SYSROOT -DCMAKE_RUNTIME_PREFIX=/ -DCMAKE_TOOLCHAIN_FILE=aarch64-toolchain.txt
$ make
$ sudo make install

Cross-compile cmtp-responder

Before we delve into cross-compiling cmtp-responder, you must know about the descriptor and strings blobs, which it needs. They can be hand-crafted in a way similar to what is shown in one of the previous blog posts in this series. The good news is that you don't need to do it by hand, because the build system of cmtp-responder can do it for you, too. However, at the moment this feature works only when building natively, so for the sole purpose of building the descriptor and string blobs you are better off compiling natively on the target board. When cross compiling this is still useful, you need to invoke both the native and cross builds. The "side effect" of the native build will be our descriptor blobs which don't depend on the target platform and don't need to be re-built each time cmtp-responder is compiled, then we remove everything else and run the cross build.

$ git clone https://github.com/cmtp-responder/cmtp-responder.git
$ cd cmtp-responder

# build natively to get the descriptor and string blobs
$ cmake -DBUILD_DESCRIPTORS=ON .

$ ls -l descs strs
-rw-r--r-- 1 ap ap 80 sty  5 13:08 descs
-rw-r--r-- 1 ap ap 32 sty  5 13:08 strs

# move descriptor and string blobs to a safe place
$ mv descs ..
$ mv strs ..

# revert the working dir to a fresh state
$ rm -rf *
$ git reset --hard HEAD

# restore descriptor and string blobs
$ mv ../descs .
$ mv ../strs .

# now do the cross build
$ cp ../gt/aarch64-toolchain.txt .

$ cmake -DCMAKE_TOOLCHAIN_FILE=aarch64-toolchain.txt
$ make
$ sudo make DESTDIR=$SYSROOT install

Cross-compile linux kernel

There are many online sources describing this topic, please use your favorite search engine to find instructions. For the .config:

CONFIG_CONFIGFS_FS=y               # ConfigFS support
CONFIG_USB=y                                # USB support
CONFIG_USB_GADGET=y               # USB gadget framework
CONFIG_USB_CONFIGFS=y             # composing USB gadgets with ConfigFS
CONFIG_USB_CONFIGFS_F_FS=y  # make FunctionFS a component for creating USB gadgets with ConfigFS

On top of that you need to enable your UDC, which is platform-specific. For RockPi 4 I had to say:

USB_DWC3=y
USB_DWC3_GADGET=y
USB_DWC3_OF_SIMPLE=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y

Then proceed as usually with cross-compiling the kernel, e.g. for RockPi 4:

$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8

Configure cmtp-responder

# On the build host, while in cmtp-responder directory
$ sudo mkdir $SYSROOT/etc/gt/templates
$ sudo cp systemd/mtp-ffs.scheme $SYSROOT/etc/gt/templates
$ sudo cp systemd/*.socket $SYSROOT/etc/systemd/system
$ sudo cp systemd/*.service $SYSROOT/etc/systemd/system
$ sudo cp systemd/*.mount $SYSROOT/etc/systemd/system

# On the target

$ sudo systemctl enable usb-gadget.service
$ sudo systemctl enable run-ffs_mtp.mount
$ sudo systemctl enable ffs.socket

After rebooting the board it should appear as an MTP device seen by your USB host. Enjoy!