Emil Velikov
August 31, 2020
Share this post:
Reading time:
Having a comprehensive and accelerated graphics stack is essential in today's world. But where would we be without one, or how do we as developers handle the lack or instability of drivers during very early hardware bring up?
Simple - we use drivers which do all the rendering via the CPU. An interesting fact is that during the dawn of 3D graphics CPU rendering was not uncommon.
With time, GPU devices became more popular, powerful and cheaper. Thus CPU (software) rendering, is not so wide spread these days and mainly used as a fallback.
Within this article, we'll provide a high-level introduction of the Linux graphics stack, how it is used within ChromeOS and the work done to improve software rendering (while simultaneously improving GPU rendering, by reducing the boilerplate needed in applications).
 |
| Original photo by Anete Lusina on Unsplash. |
Mesa is de-facto the open-source driver implementation for accelerated graphics in the Linux world. Although as we can see there is ongoing work to make it a significant component on Microsoft Windows.
The drivers themselves, provide our rendering APIs - OpenGL and GLES To bridge those with the OS windowing system (like Wayland, X11 or KMS) we need additional API - EGL or GLX, which is part of the drivers provided by Mesa.
One of our goals is to be as flexible as possible, while minimising the amount of legacy code required - so in our case we're using OpenGL/GLES and EGL. In particular we are making use of the EGL_MESA_platform_surfaceless extension. It allows us to use OpenGL or GLES and render into a memory area, not requiring integration with the display subsystem.
Effectively allowing us to have a single code path, devoid of deep and convoluted logic about Wayland, X11 or KMS. So, job well done you say?
Not quite, there is this thing called authentication which is stopping us.
The Linux kernel graphics subsystem is called Direct Rendering Manager or DRM for short. Not to be confused with Digital Rights Management which is a completely separate thing.
The DRM subsystem has a wide variety of jobs. At the high-level they fall into one of several groups - vendor agnostic API for managing the display controller (KMS), GPU rendering, buffer allocation and cross process/context sharing.
Each application using DRM is called a client. There can be multiple clients using a DRM driver at any given time.
In the early days, both hardware and drivers, lacked the relevant separation between clients. It meant that, we needed a way to 'allow' only a given number or set of clients. This is something orchestrated by the first and most versatile client - the master. The master has the powers to grant access via an authentication session triggered by each client.
To accommodate for such requests, both Wayland and Xorg, had to wrap around the native DRM/KMS authentication method and define their own respective protocols. Thus programs, or their respective toolkits like GTK or Qt, were responsible for detecting the session type (Wayland, Xorg or KMS) and triggering the corresponding authentication code path.
Fast forward a couple of decades - both hardware and drivers have become more capable and secure, providing robust isolation across clients. Yet the authentication remained :-\
So despite our use of EGL_MESA_platform_surfaceless, we still needed some Wayland/X11 specific code. But why? As said a few moments ago, modern drivers are robust enough. Can we can simply remove the authentication requirement? Turns out we can.
As a nice bonus point, this also helps programs which were missing any of the three authentication paths - Wayland, X11 or KMS. If the said program was not open-source, we had no way of adding the required code. Therefore there was only one way to make it work - set the CAP_SYS_ADMIN capability, effectively running the program with root privileges. Something which many of us feel uncomfortable with as it grants complete access to the system.
Indeed, I did.
Today there are two types of software GL drivers, or rasterizers as they're being called. Ones that talk through X and ones which use an unaccelerated "dumb" framebuffers, exposed by the DRM driver.
The dumb buffers themselves, are a special subclass of objects exposed by DRM - they are exposed in a consistent vendor-agnostic manner, yet they lack any acceleration by the rendering device. Hence applications are allowed to draw onto those buffers only via the CPU - our usecase.
By using them, we can omit the final piece of legacy X11/Xorg code for ChromeOS, in its platform abstraction layer Ozone.
Although, using dumb buffers also requires authentication. So removing the requirement, as per our last section, effectively allows us to use software GL drivers within ChromeOS.
Among the first parts of my investigation, was to examine the inner workings of the authentication mechanism both within the kernel DRM subsystem, and the respective wrappers for Xorg and Wayland.
The results where promising - the APIs were designed only to grant permission, with no mention of revocation. This indicated that there was no explicit tracking within Xorg or Wayland but only within DRM itself.
For the next step, I embarked on auditing of all GPU DRM drivers in the kernel. While many drivers required a trivial patch to remove the DRM_AUTH notation, others like QXL where found to lack any client separation. In one particular case, additional work was required by the driver maintainers to make the existing code more robust.
With the individual drivers complete, it was time for the final kernel piece - the dumb buffers handling, as part of core DRM. Those proved safe and my observations were confirmed by the DRM maintainers.
As seemingly all the work was done, it was time to verify my analysis and observations.
I've started small: To verify that the overall authentication removal works as expected, I've hacked a few standalone utilities like vainfo. Additionally experimented with removing the authentication requests from Mesa and running GL applications against it. Without much surprise everything was working like a charm - regardless whether the application was using GLX or EGL, or was ran within a Xorg or a weston (a Wayland compositor) session.
The final step was to build ChromiumOS and run it. The initial run was within QEMU backed by the virgl GPU driver. For this use-case, I've deployed my earlier Mesa hacks, to verify the authentication, or lack thereof, did not cause any issues. With that working as expected, I also confirmed that software rendering was working fine on an actual Chromebook device. Despite that software GL drivers were used, the overall experience of booting the system and light web browsing was surprisingly smooth and responsive.
In a true, open-first manner, all the patches have been upstreamed. In particular all the patches are in Linux 5.6. If you're using an older kernel you can apply the patches manually:
19/12/2024
In the world of deep learning optimization, two powerful tools stand out: torch.compile, PyTorch’s just-in-time (JIT) compiler, and NVIDIA’s…
08/10/2024
Having multiple developers work on pre-merge testing distributes the process and ensures that every contribution is rigorously tested before…
15/08/2024
After rigorous debugging, a new unit testing framework was added to the backend compiler for NVK. This is a walkthrough of the steps taken…
01/08/2024
We're reflecting on the steps taken as we continually seek to improve Linux kernel integration. This will include more detail about the…
27/06/2024
With each board running a mainline-first Linux software stack and tested in a CI loop with the LAVA test framework, the Farm showcased Collabora's…
26/06/2024
WirePlumber 0.5 arrived recently with many new and essential features including the Smart Filter Policy, enabling audio filters to automatically…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (3)
Krishna:
Sep 01, 2020 at 07:35 PM
This is fantastic work! Abstracting away the wayland vs xorg distinction is wonderful news for developers. Keep up the great work.
Reply to this comment
Reply to this comment
Dmitry:
Sep 02, 2020 at 01:06 PM
Is it possible to run ChromeOS with std or cirrus qemu video cards?
Reply to this comment
Reply to this comment
Emil Velikov:
Sep 03, 2020 at 09:59 AM
Hi Dmitry,
Last time I've looked the cirrus qemu DRM driver lacked atomic modeset. A feature that, I believe, is a hard requirement for modern CrOS.
In general I would suggest using the virtio GPU as it is more actively developed, plus it allows for hardware acceleration via virglrenderer and the host GPU.
Reply to this comment
Reply to this comment
Add a Comment