Earlier this year, I joined Collabora as an intern to work on improving testing in libcamera and automating it through KernelCI. Having recently completed the internship, here's a look back at this experience and what was accomplished.

Background

Put simply, libcamera is a library that handles acquiring, configuring and capturing frames from a camera. Camera pipelines have become increasingly complex, and traditionally this complexity has been exposed by the kernel through the V4L2 APIs, for applications to deal with directly. libcamera is the layer in-between V4L2 and the application so that camera handling can become simple.

Inside libcamera, the pipeline handler is the one that coordinates the capture pipeline, and there's generally one for each camera driver. To capture a frame, the application allocates a buffer and wraps it in a request, which is submitted to the pipeline handler. The pipeline handler then submits the buffer to the video capture device, and when the buffer is filled with the image it signals the application that the request is ready.

libcamera's compliance tests refactoring

When I began this project, libcamera already had a compliance test suite called lc-compliance. Its purpose is to ensure that libcamera's API works as expected, that is, it makes sure that setting up the capture pipeline using libcamera and capturing a few frames works well.

While this compliance tool worked fine for testing purposes, it was still new, and had some shortcomings: the structure of the tests wasn't clear and the returning of test results was cumbersome.

I worked on improving the situation by adapting the lc-compliance code to use the GoogleTest testing framework. This involved thinking ahead to how the test result output would be integrated into KernelCI, as well as deciding how to pass the camera instance to be tested through the framework.

Now the architecture is clearer: test cases are always part of a test suite. Also, expectations in the test are checked with GoogleTest's ASSERT and EXPECT macros, which automatically raise an exception on failure, removing the need for the cumbersome test result returning. On top of those improvements, this change also brought GoogleTest's ability to list available test cases and execute only a subset of them through a given filter.

The series can be seen on the mailing list and the individual commits here: 1, 2, 3, 4, 5.

Automated testing with KernelCI

In addition to improving the testing infrastructure in libcamera, another goal was to make these tests run automatically on kernel releases. This meant adding lc-compliance to KernelCI.

KernelCI is a continuous integration platform that basically executes test cases on trees of the Linux Kernel with the aim of detecting regressions in the kernel. The tests are run on real hardware hosted on LAVA labs and the results are stored and available at its dashboard.

The wiring of tests, devices and labs in KernelCI is done through a few different YAML configuration files, but I also needed to write a shell script to collect build-time dependencies and compile libcamera and its lc-compliance tool for the rootfs, as well as a parser script to notify LAVA about the test results output by lc-compliance.

I won't go over the details of these changes since I've already written a post about that work. The result is that libcamera is now being tested on an Acer Chromebook R13, which has an USB camera, with Rock Pi 4 boards joining in the near future. This ensures that changes on the kernel that cause regressions on real life use cases of cameras will now be detected. And as more devices with cameras are added to the lab and have the test enabled, the coverage will increase further.

Testing overflow of requests with lc-compliance

One open bug in libcamera was that when too many requests were queued at once to the pipeline handler, it would fail to queue them all to the capture device and either cancel the requests or drop them entirely.

That issue was fixed on the IPU3 pipeline handler, but not on the others. The fix consists of creating an internal queue in the pipeline handler where it can queue the requests internally while there aren't buffer slots available to satisfy them.

In order to really fix the issue, that same pattern of creating an internal queue should be implemented in all the existing pipeline handlers, and a new test should be written for lc-compliance to test that scenario and confirm that it's being handled correctly.

I worked on implementing the internal queue in the other pipeline handlers (rkisp1, simple, vimc and uvcvideo) and submitted the series to the mailing list as can be seen here, though it hasn't been merged yet.

As for the new lc-compliance test, it would need to allocate a bigger number of buffers than usual to use them in the requests sent to the pipeline. The way buffers are usually allocated by an application using libcamera is through the FrameBufferAllocator.

The current implementation of the FrameBufferAllocator only allowed a fixed number of buffers to be allocated, however, and my test needed a custom number of buffers. While I could allocate the buffers elsewhere it would be nicer to use libcamera's own allocator, so I decided to extend the FrameBufferAllocator::allocate() function to allow the number of buffers to be specified.

As I wrote the test, I noticed that one issue the explicit buffer count in allocate() exposed is that depending on the pipeline handler there's a minimum number of requests required for capture to even be possible. That means that a way is needed to report this number for each pipeline. This was done by adding another patch introducing the MinimumRequests property which is set accordingly by the pipeline, and that required a lot of discussion to decide on reasonable values for.

With allocate() now accepting a count parameter, the bufferCount field from the StreamConfiguration class had one less usage. Talking on the IRC with Laurent Pinchart, one of the maintainers of libcamera, revealed that that bufferCount field was slated for removal at some point.

Since I was here to learn, and I was already touching part of that code, I ended up also adding patches to rework the pipeline handlers to no longer depend on bufferCount and remove it.

The benefit of this change is that now there's a clear divide between the number of buffer slots and internal buffers allocated by the pipeline handlers, which is good since those shouldn't be related.

So one thing lead to the other and that's the reason this series ended rather big. You can check it out here, however it is not yet merged.

Flash LED driver side-project

Development with a community takes time, not only for discussing the best approach for solving a problem, but also to review the code submitted. While I was waiting for some reviews, I worked on the port of a downstream driver for the Nexus 5's flash LED which I had started as a personal project but didn't have time to carry on since. During this time I was able to address the feedback I had received for the previous version and send a new one for review. You can check out the latest version of the driver series here.

Wrapping up

I really enjoyed the experience of this internship: I had the opportunity to learn about areas like V4L2, libcamera and KernelCI, to interact closely with open-source communities like libcamera and KernelCI, to see my work causing a positive impact in these projects and overall I feel that I've matured both as a developer and as an open-source citizen.

If this seems interesting to you, look out for internship opportunities in our Careers page!