Erik Faye-Lund
March 24, 2020
Share this post:
Reading time:
For the last few months, we have been working on two exciting new projects at Collabora, and it's finally time to share some information about them with the world:
We are partnering with Microsoft DirectX engineers to build OpenCL and OpenGL mapping layers, in order to bring OpenCL 1.2 and OpenGL 3.3 support to all Windows and DirectX 12 enabled devices!
This work builds on a lot of previous work. First and foremost, we are building this by using Mesa 3D, with the Gallium interface as the base for the OpenGL layer, and NIR as the base for the OpenCL compiler. We are also using LLVM and the SPIRV-LLVM-Translator from Khronos as the compiler front-end.
In addition, we are taking advantage of Microsoft's experience in creating their D3D12 Translation Layer, as well as our own experience from developing Zink.
Mesa 3D is an open source implementation of several graphics technologies, including OpenCL and OpenGL. The OpenGL implementation in Mesa is robust and is used as the base for several industry-strength OpenGL drivers from multiple GPU vendors.
Among other things, Mesa consists of several API implementations (called state-trackers) as well as the Gallium low-level driver interface. The Gallium interface hides a lot of the legacy OpenGL details and translates OpenGL calls into something that looks more like modern GPU primitives.
Not all Windows-powered devices have consistent support for hardware-accelerated OpenCL and OpenGL. So in order to improve application compatibility, we are building a generic solution to the problem. This means that a GPU vendor only has to implement a D3D12 driver for their hardware in order to support all three APIs.
This mapping layer is also expected to serve as a starting point in porting older OpenCL and OpenGL applications over to D3D12.
In addition, we believe this is good for the wider open source community. A lot of the problems we are solving here are shared with other drivers and translation layers, and we hope that the code will be useful beyond the use cases listed above.
The work is largely split into three parts: an OpenCL compiler, an OpenCL runtime, and a Gallium driver that builds and executes command-buffers on the GPU using the D3D12 API.
In addition, there is a shared NIR-to-DXIL shader compiler that both components use. For those not familiar with NIR, it is Mesa's internal representation for GPU shaders. Similarly, DXIL is Microsoft's internal representation, which D3D12 drivers will consume and translate into hardware-specific shaders.
The OpenCL compiler uses LLVM and the SPIRV-LLVM-Translator to generate SPIR-V representations of OpenCL kernels. These, in turn, are passed to Mesa's SPIR-V to NIR translator, where some optimizations and semantical translations are done. Then the NIR representation is finally passed to NIR-to-DXIL, which produces a DXIL compute shader and the needed metadata so it can be executed on the GPU by the runtime using D3D12.
Here's a diagram of the complete process, including NIR-to-DXIL, which will be described below:
While Mesa provides an OpenCL implementation called Clover, we are not using it for this project. Instead, we have a new OpenCL runtime that does a more direct translation to the DirectX 12 API.
DXIL is essentially LLVM 3.7 bitcode with some extra metadata and validation. This was a technical choice that made sense for Microsoft because all the major driver vendors already used LLVM in their compiler toolchain. Using an older version of the LLVM bitcode format gives good compatibility with drivers because the LLVM bitcode format is backwards compatible.
Because we depend on a much more recent version of LLVM for the compiler front-end, we sadly cannot easily use the DirectX Shader Compiler as a compiler back-end. The DirectX Shader Compiler is effectively a fork of LLVM 3.7, and we are currently using LLVM 10.0 for the compiler front-end. Using DirectX Shader Compiler as that would require us to link two different versions of LLVM into the same binary, which would have led to problems.
We also cannot easily use LLVM itself to generate the bitcode. While the LLVM bitcode format is backwards compatible, LLVM itself is not *forward compatible*. This means that newer versions of LLVM cannot produce a bitcode format that is understood by older versions. This makes sense from LLVM's point of view because it was never meant as a general interchange format.
So instead, we have decided to implement our own DXIL emitter. This is quite a bit harder than it looks because LLVM bitcode goes to great lengths to try to make the format as dense as possible. For instance, LLVM does not store its bitcode as a sequence of bytes and words, but rather as variable-width bitfields in a long sequence of bits.
There are a lot of tricky details to get right, but in the end we have a compiler that works.
The D3D12 Gallium driver is the last piece of the puzzle. Essentially, it takes OpenGL commands and, with the help of the NIR to DXIL translator, turns them into D3D12 command-buffers, which it executes on the GPU using the D3D12 driver.
There are a lot of interesting details that makes this tricky as well, but I will save those details for later.
But to not leave you empty-handed, here's a screenshot of the Windows version of the famous glxgears, wglgears:
In the short term, the source code can be found here. We intend on upstreaming this work into the main Mesa repository shortly, so it is not a permanent home.
This is just the announcement, and a whole lot of work is left to be done. We have something that works in some cases right now, but we are just starting to scratch the surface.
First of all, we need to get up to the feature-level that we target. Our goals at the moment is to pass conformance tests for OpenCL 1.2 and OpenGL 3.3. We have a long way to go, but with some hard work and sweat, I am sure we will get there.
Secondly, we need to work on application compatibility. For now we will be focusing on productivity applications.
We also want to upstream this in Mesa. This way we can keep up with fixes and new features in Mesa, and other drivers can benefit from what we are doing as well.
It is also important to point out that I am not the only one working on this. Our team consists of five additional Collabora engineers (Boris Brezillon, Daniel Stone, Elie Tournier, Gert Wollny, Louis-Francis Ratté-Boulianne) and two Microsoft DirectX engineers (Bill Kristiansen, Jesse Natalie).
20/12/2024
The Rockchip RK3588 upstream support has progressed a lot over the last few years. As 2024 comes to a close, it is a great time to have…
09/12/2024
Collabora will be at NeurIPs this week to dive into the latest academic findings in machine learning and research advancements that are…
05/12/2024
Now based on Debian Bookworm, Apertis is a collaborative OS platform that includes an operating system, but also tools and cloud services…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (10)
Nathaniel MacArthur-Warner:
Jun 18, 2020 at 09:01 PM
>Not all Windows-powered devices have consistent support for hardware-accelerated OpenCL and OpenGL.
And how many Windows-powered devices have support for DirectX 12 and WDDM 2.9, both of which are required for this OpenGL-on-DirectX-on-GPU-PV Frankenstein of a setup to work? My computer has full support for OpenGL 4.5 and Vulkan, but only WDDM 2.1.
Why not give people the option of using OpenGL/OpenCL/Vulkan directly? It seems like this would be a lot less engineering work on Microsoft's part.
Reply to this comment
Reply to this comment
Daniel Stone:
Jun 19, 2020 at 03:34 PM
Deprecating DirectX and moving the entire world to Vulkan/GL/CL would not be less effort on Microsoft's part. I don't think developers who currently target DirectX would be very happy about it, and they'd have to keep on supporting it forever.
Vendors can still build native GL/CL drivers if they want - as you noted, many of them do. For OpenGL ES, that's even somewhat tractable. But for full OpenGL, supporting everything from the modern core contexts with a comparable rendering pipeline to DirectX and Vulkan, all the way back to the old fixed-function pipeline, is incredibly difficult. Writing a new OpenGL implementation from scratch is pretty much impractical. This is why we have this project to reuse (and contribute improvements back to) Mesa, because it really is less effort than having every vendor come up with a complete, conformant, and performant, GL/CL implementation.
Reply to this comment
Reply to this comment
Nathaniel MacArthur-Warner:
Jun 20, 2020 at 01:56 AM
I didn't suggest that Microsoft "move the entire world to Vulkan/GL/CL", or that they create a new implementation. I said it would make sense for Microsoft to allow driver support for Khronos APIs to be passed through to the guest (WSL in this case). Android Studio's built-in emulator does that, so why can't Microsoft?
The only technical advantage I can see to the current approach is having DirectX fully available in Linux, which makes no sense for Linux software development. Is Microsoft trying to get people to write software for Linux that only works in the context of WSL?
Reply to this comment
Reply to this comment
Daniel Stone:
Jun 20, 2020 at 11:54 AM
> I said it would make sense for Microsoft to allow driver support for Khronos APIs to be passed through to the guest (WSL in this case). Android Studio's built-in emulator does that, so why can't Microsoft?
Sure, the vendors can do that if they want to. I assume they probably don't want to because it means creating Linux EGL implementations atop their existing Windows driver, which is not all that straightforward.
The new /dev/dxg API and the Android/ChromeOS approach are very different. /dev/dxg does _not_ pass through 'the DirectX API' in the sense of the D3D calls which you use to render to. It's a very low-level API, much like DRM on Linux, which allows userspace to make buffer allocations, submit shader programs pre-compiled into hardware bytecode for execution, and synchronise against that execution - that's about it. This is what both DirectX and OpenGL drivers run atop of on Windows. In order to run DirectX drivers inside WSL, you thus need to port both the DIrectX core and each hardware driver over to running in Linux, or the rough equivalent of porting all of Mesa and its drivers. It would be possible for IHVs to port their OpenCL/GL drivers over to using this interface as well, but maintaining that takes substantial work.
ChromeOS and Android emulation use VirGL, which instead of exposing a low-level hardware interface, effectively stream OpenGL calls over the wire after some guest-side optimisation. VirGL isn't as performant as the /dev/dxg approach, and never will be. As you note though, it is maximally compatible: all you need is a KVM virtualisation platform and a host with OpenGL (ES) drivers.
> The only technical advantage I can see to the current approach is having DirectX fully available in Linux, which makes no sense for Linux software development. Is Microsoft trying to get people to write software for Linux that only works in the context of WSL?
Neither of the solutions (exposing hardware interface vs. exposing high-level API) is really any better or worse than the other; they're just different trade-offs.
Reply to this comment
Reply to this comment
Nathaniel MacArthur-Warner:
Jun 20, 2020 at 07:37 PM
Thanks for the clarification. But if /dev/dxg is just exposing a hardware interface, why not expose that hardware interface using the existing mechanisms of the DRM?
Reply to this comment
Reply to this comment
Daniel Stone:
Jun 22, 2020 at 11:44 AM
Because DRM isn't a perfectly hardware-independent interface (you can't actually execute any GPU commands without driver-specific ioctls, mostly called 'execbuf'), and it also isn't a native virtualisation interface (being as it depends quite a bit on file descriptors). It also isn't ported to Windows, so you'd need to port every driver's DRM infrastructure to Windows (with shims allowing for behavioural differences between different OS implementations of the vendor drivers) and figure out a way to virtualise DRM?
Reply to this comment
Reply to this comment
Salvador:
Dec 14, 2020 at 08:30 PM
great work!!
Reply to this comment
Reply to this comment
Victor Milovanov:
Apr 18, 2021 at 11:25 PM
Can this be used in UWP apps? Can I use it on Xbox?
Reply to this comment
Reply to this comment
Erik Faye-Lund:
Apr 19, 2021 at 02:19 PM
I'm not sure I can answer those questions with any sort of authority, but I'll do my best. I've never developed with either technology, so take my response with a grain of salt.
I would be surprised if it this couldn't be used for UWP, in the case of an application developer wanting to run OpenGL code on top of DirectX. The reason I expect that use-case to work, is that there shouldn't be anything as far as I can tell that would prevent the developer from linking the code into their own application.
XBox, I can't really answer at all. You might be in the same situation as above, and you might not be. I don't think I can find this out without becoming a licensed XBox developer, which I'm not planning on. And even in that case, revealing this might breach some NDA ;)
In both cases, you might have to do some changes to Mesa, particularly in the window-system integration parts.
Reply to this comment
Reply to this comment
Earle:
Dec 22, 2021 at 06:17 PM
Good day!
Really appreciate all the efforts and hard work on this project!
Is there a timeline on when OpenGL 4.4+ will be supported?
Thanks!
Reply to this comment
Reply to this comment
Add a Comment