One of the most difficult challenges in creating any polished and smooth system—from an embedded system running your car stereo to a full-fledged desktop—is getting the graphics software right.
The graphics stack does not simply stop at having OpenGL ES-capable hardware. As user requirements have grown and evolved beyond having a single full-screen game, so have the demands on the graphics software, to the point where it is one of the most complex — yet least well-understood — parts of a modern system. The graphics pipeline can no more be reduced to "display some content" than media can be reduced to "play a video": it requires a careful balancing act between visible performance on one hand, and power and memory consumption on the other, whilst ensuring proper synchronisation, fluid animations, high throughput, accurate colour reproduction, and more.
Working across a myriad of form factors and device types also presents a strong challenge. A change which improves performance for a particular usecase on some hardware may also, counterintuitively, cause a severe performance decrease on another device target.
X11 has been the industry standard for almost all UNIX systems since its inception in 1986. Collabora engineers have been heavily involved in advancing the development of X11 for over a decade between them, but its fundamental limitations have led the open source community to develop a replacement more suited to modern and fluid UIs.
Wayland is an open source initiative backed by Intel, Red Hat, Collabora and more, which aims to produce a base on which applications can deliver fluid, modern user interfaces. Wayland acts as a replacement for X11, integrating with OpenGL ES and EGL for 3D-rendered display, media frameworks such as GStreamer, OpenMAX and VA-API for video playback, and more. With further support from the X.Org development community, semiconductor vendors, and OEMs, the Wayland project is paving the way for the future of open-source graphics.
Wayland's core underlying principle is simple: every frame must be perfect. By that, Collabora means that the user must never see incomplete or transitional results: seeing a window appear first as a gray square, followed by half of the UI elements, with the rest filled in at a later date, is simply not good enough by modern standards. Similarly, users are now unwilling to accept jerky animations, seemingly random freezes, or anything less than perfect graphical interfaces. While X11 must be fought to achieve these results, often with band-aid solutions which are not portable beyond a single product, Wayland has these principles baked in from scratch.
Another core goal of Wayland is to vastly improve the driver support story compared to X11. The X11 driver model is extremely complex and fragile, and requires extensive integration from the hardware vendor in order to deliver basic enablement, as well as ongoing rolling updates whenever the overall software stack is updated. Wayland offers a much smaller surface area, requiring only support for standard Linux kernel display APIs, as well as one EGL extension for accelerated 3D. This not only decreases the upfront development cost to the hardware vendor, but drastically reduces the ongoing support cost.
The Wayland project is comprised of the following:
Collabora works closely on the entire Wayland ecosystem, from core protocol changes affecting how all content is displayed on screen, to new features inside Weston's UI, to hardware enablement on a number of platforms, to adding and enhancing Wayland support for higher-level toolkits such as Clutter and GTK+. Collabora's deep involvement, across a number of devices, applications and verticals, gives Collabora a unique insight into Wayland's design and direction.
Weston is a compositor implementation targeted as the base for embedded, mobile and desktop environments. Weston provides three major components: the core Wayland protocol handling, an abstracted hardware layer allowing it to run on a number of platforms with varied hardware and display control interfaces, and a UI framework allowing rapid development of new interfaces. Weston supports the most common input and output frameworks under the Linux kernel, and is able to use any of OpenGL ES, Pixman, or dedicated display hardware (see the Raspberry Pi case studyfor more information). Its first-class support for running inside an existing X11 or Wayland session allows rapid development and prototyping without the need for a separate virtual machine.
Perhaps Weston's most compelling feature for embedded, mobile and media platforms is its extensive support for dedicated display hardware. Most X11-based systems either do not support use of dedicated 2D compositing blocks, requiring either power-hungry OpenGL or slow software-based rendering, or suffer from a lack of integration between the hardware acceleration and user interface. Weston supports dedicated compositing hardware as a first-class citizen, enabling fluid animations and transitions and high-quality video decoding, all whilst keeping power usage and CPU utilisation to an absolute minimum. With this, Collabora is able to deliver slick user interfaces to platforms whose true hardware capabilities have often gone to waste.
Collabora has successfully delivered Wayland hardware enablement for a number of platforms, from powerful media-led hardware to mobile devices and everything in between. For instance, Collabora contributed to Wayland-based solutions for the Raspberry Pi, Android, PowerVR RGX-based SOCs, etc.
Wayland delivers a much lower initial and ongoing cost with regards to hardware enablement than previous heavyweight solutions. Collabora is able to rapidly deliver a flexible solution which meets any needs.
One key feature of Collabora's hardware enablement has been a fully zero-copy pipeline, allowing our customers to achieve the highest possible throughput, which is particularly crucial for media and video applications, as well as high-resolution and high-DPI displays. Wayland's unique architecture has allowed Collabora to include platform-specific optimisations and tweaks crucial for full performance without invasive patches to Wayland itself, allowing full portability between Wayland implementations whilst still using the hardware to its full potential.
Most applications are written using a toolkit such as GTK+, Qt or Clutter, which provides the basic UI elements and look and feel for applications, as well as papering over differences in the underlying graphics stacks.
With many years of experience on the desktop and mobile stacks, be it for Linux or other platforms, Collabora is uniquely positioned to design and implement beautiful and functional graphical user interfaces. Collabora uses technologies such as GTK+, Qt, Clutter, Mx, and WebKit on a regular basis. Using those toolkits is not all, of course: Collabora has also been helping adapt those toolkits, along with various applications and frameworks, so that they work with Wayland.
Should your needs go beyond what these toolkits already offer, Collabora can help you improve them and contribute the improvements back to the community.
Collabora also offers ongoing support, training around these technologies or user interaction design services.
...and more! Please get in touch with us so we can start helping you today.
Click below to learn more about the various Open Source projects we're actively involved in.
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português