Ana Guerrero López
June 27, 2018
Reading time:
In Debian and derivative systems, there are many ways to build images. The simplest tool of choice is often debootstrap
. It works by downloading the .deb files from a mirror and unpacking them into a directory which can eventually be chrooted into.
More often than not, we want to make some customization on this image, install some extra packages, run a script, add some files, etc
debos is a tool to make thiese kinds of trivial tasks easier. debos works using recipe files in YAML listing the actions you want to perform in your image sequentially and finally, choosing the output formats.
As opposite to debootstrap and other tools, debos doesn't need to be run as root for making actions that require root privileges in the images. debos uses fakemachine
a library that setups qemu-system allowing you to work in the image with root privileges and to create images for all the architectures supported by qemu user. However, for this to work, make sure your user has permission to use /dev/kvm
.
Let's see how debos works with a simple example. If we wanted to create an arm64 image for Debian Stretch customized, we would follow these steps:
This would translate into a debos recipe like this one:
{{- $architecture := or .architecture "arm64" -}} {{- $suite := or .suite "stretch" -}} {{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }} architecture: {{ $architecture }} actions: - action: debootstrap suite: {{ $suite }} components: - main mirror: http://deb.debian.org/debian variant: minbase - action: apt recommends: false packages: - adduser - sudo - action: run description: Set hostname chroot: true command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname - action: run chroot: true script: scripts/setup-user.sh - action: overlay description: Add sudo configuration source: overlays/sudo - action: pack file: {{ $image }} compression: gz
(The files used in this example are available from this git repository)
We run debos on the recipe file:
$ debos simple.yaml
The result will be a tarball named debian-stretch-arm64.tar.gz
. If you check the top two lines of the recipe, you can see that the recipe defaults to architecture arm64 and Debian stretch. We can override these defaults when running debos:
$ debos -t suite:"buster" -t architecture:"amd64" simple.yaml
This time the result will be a tarball debian-buster-amd64.tar.gz
.
The recipe allows some customization depending on the parameters. We could install packages depending on the target architecture, for example, installing python-libsoc
in armhf
and arm64
:
- action: apt recommends: false packages: - adduser - sudo {{- if eq $architecture "armhf" "arm64" }} - python-libsoc {{- end }}
What happens if in addition to a tarball we would like to create a filesystem image? This could be done adding two more actions to our example, a first action creating the image partition with the selected filesystem and a second one deploying the image in the filesystem:
- action: image-partition imagename: {{ $ext4 }} imagesize: 1GB partitiontype: msdos mountpoints: - mountpoint: / partition: root partitions: - name: root fs: ext4 start: 0% end: 100% flags: [ boot ] - action: filesystem-deploy description: Deploying filesystem onto image
{{ $ext4 }}
should be defined in the top of the file as follows:
{{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }}
We could even make this step optional and make the recipe by default to only create the tarball and add the filesystem image only adding an option to debos:
$ debos -t type:"full" full.yaml
The final debos recipe will look like this:
{{- $architecture := or .architecture "arm64" -}} {{- $suite := or .suite "stretch" -}} {{ $type := or .type "min" }} {{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }} {{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }} architecture: {{ $architecture }} actions: - action: debootstrap suite: {{ $suite }} components: - main mirror: http://deb.debian.org/debian variant: minbase - action: apt recommends: false packages: - adduser - sudo {{- if eq $architecture "armhf" "arm64" }} - python-libsoc {{- end }} - action: run description: Set hostname chroot: true command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname - action: run chroot: true script: scripts/setup-user.sh - action: overlay description: Add sudo configuration source: overlays/sudo - action: pack file: {{ $image }} compression: gz {{ if eq $type "full" }} - action: image-partition imagename: {{ $ext4 }} imagesize: 1GB partitiontype: msdos mountpoints: - mountpoint: / partition: root partitions: - name: root fs: ext4 start: 0% end: 100% flags: [ boot ] - action: filesystem-deploy description: Deploying filesystem onto image {{end}}
debos also provides some other actions that haven't been covered in the example above:
The example in this blog post is simple and short on purpose. Combining the actions presented above, you could also include a kernel and install a bootloader to make a bootable image. Upstream is planning to add more examples soon to the debos recipes repository.
debos is a project from Sjoerd Simons at Collabora, it's still missing some features but it's actively being developed and there are big plans for the future!
Visit Ana's blog.
08/10/2024
Having multiple developers work on pre-merge testing distributes the process and ensures that every contribution is rigorously tested before…
15/08/2024
After rigorous debugging, a new unit testing framework was added to the backend compiler for NVK. This is a walkthrough of the steps taken…
01/08/2024
We're reflecting on the steps taken as we continually seek to improve Linux kernel integration. This will include more detail about the…
27/06/2024
With each board running a mainline-first Linux software stack and tested in a CI loop with the LAVA test framework, the Farm showcased Collabora's…
26/06/2024
WirePlumber 0.5 arrived recently with many new and essential features including the Smart Filter Policy, enabling audio filters to automatically…
12/06/2024
Part 3 of the cmtp-responder series with a focus on USB gadgets explores several new elements including a unified build environment with…
Comments (10)
Osqui:
Jun 28, 2018 at 10:29 AM
Excellent.
Do you plan extend this to create not Debian-like images?
Thanks!
Reply to this comment
Reply to this comment
Ana:
Jul 06, 2018 at 01:52 PM
Hi Osqui!
As the name hints, we're not planning to work on creating images for other distributions. However we'll be happy to add this if somebody send a pull request :)
Cheers,
Ana
Reply to this comment
Reply to this comment
Mrfai:
Jul 08, 2018 at 11:58 PM
The Debian cloud team is using the FAI software (https://fai-project.org) for different cloud images and has it's configuration at https://salsa.debian.org/cloud-team/fai-cloud-images. FAI can create installation images and bootable disk images for other distributions like Ubuntu, CentOS, Scientific Linux.
Reply to this comment
Reply to this comment
Rich:
Jun 28, 2018 at 11:57 AM
Why didn't you use virt-builder? It does all of this already.
Reply to this comment
Reply to this comment
Osqui:
Jun 28, 2018 at 03:40 PM
Or mkosi...
Reply to this comment
Reply to this comment
Paul Menzel:
Jun 29, 2018 at 07:45 AM
To my knowledge, both of them require root privileges, don’t they?
Reply to this comment
Reply to this comment
Rich:
Jun 29, 2018 at 12:04 PM
No, virt-builder does NOT require root privileges.
Reply to this comment
Reply to this comment
Ana:
Jul 06, 2018 at 01:57 PM
Hi everybody,
mkosi has focus on VM and container images rather than images for real hardware and doesn't support all the architectures debos suppots (via qemu-user)
virt-builder itself doesn't create images, it's a frontend for the images you have already created.
Cheers,
Ana
Reply to this comment
Reply to this comment
vitor:
Jan 31, 2023 at 10:09 PM
Is it possible to install a package that is not in a standard ubuntu repository? I've been trying to install docker but I'm not succeeding, do you have any examples?
Reply to this comment
Reply to this comment
Christopher Obbard:
Feb 01, 2023 at 03:25 PM
Hey! You can for sure install docker (or simply do anything you normally would in a terminal) with debos. Something like:
- action: run
chroot: true
script: scripts/install-docker
and in the scripts/install-docker script, you could put Docker installation commands from https://docs.docker.com/engine/install/debian/.
Reply to this comment
Reply to this comment
Add a Comment