KernelCI: more than just boot testing

Enabling automatic bisection and expanding functional testing for Google's Chromebooks

The KernelCI project aims at continuously testing the upstream Linux kernel, from stable branches to maintainers' trees and linux-next on a variety of platforms. The main goals are to increase test coverage while reducing the work required by developers and to further the stability of long-term support (LTS) kernels.

The kernel images are built automatically for a number of trees and sent to be tested on the devices in distributed labs featuring a wide range of hardware. Results are stored in a centralized database. When a revision fails to build, boot or function against a test, kernel developers get informed via email and the kernelci.org dashboard. Until recently however, only boot tests were run.

Working alongside Google's engineering team to add support in mainline Linux kernel for Chromebooks, Collabora has enabled automatic bisection and expanded functional testing to KernelCI, allowing notably for Chromebooks to now be continuously tested.

Automated bisection of boot failures in production

It’s not always obvious what causes a failure to boot. Development branches get typically merged with many commits on them, and only one boot test of the main branch is run after the merge. Luckily, between the last known working revision and the failing one, it is now possible to automatically run more boot tests using Git’s bisection feature until the probable culprit is found.

With this new feature now deployed in kernelci.org and used with mainline and stable trees, the next steps will be to cover additional trees and send the results to a wider audience.

Functional testing is growing fast

Another important step for the project is functional testing to guarantee that the kernel not only continues to boot but also that its features do not regress. Efforts are being put into developing functional testing as it includes many moving parts, such as adding extra support to download and run the test suites, developing a new UI to show the results and running an automated bisection of test failures.