This is the third blog post in a series about the upcoming release of WirePlumber 0.5. The first two posts discussed the configuration system refactoring and presented the new Event Dispatcher. This latest entry explores how WirePlumber's Lua scripts are transformed with Event Dispatcher.

WirePlumber lua scripts are transformed with Event Dispatcher, they look and feel completely different. They are much more modular and extensible with very little redundant processing and almost no hacks. Traditionally they used an object manager API and for the reasons explained in the previous blog post, we have moved them to Event Dispatcher. Now they are a bunch of hooks responding to appropriate events.

For all the WirePlumber's system scripts (policy-node.lua, default-nodes, restore-stream, monitor/alsa.lua, etc) you can find their .c counterparts in the PipeWire Media Session project. During the initial days of WirePlumber we translated this C logic into Lua; of course with WirePlumber elements like Object Manager. But with the advent of Event Dispatcher, they are not what they were. Further in this post, I will break down what has happened to them.

Linking scripts are by far the most complex of the WirePlumber system scripts and so I will focus on how they changed over. I will also touch upon the rest of the scripts.

WirePlumber linking scripts

Complexity

policy-node.lua is the main linking script. It is like the conductor of the whole pipewire symphony and so living up to the complexity of the task, it achieves so much out of one single lua source file. Let me try to sum up its functionality as briefly as I can.

It looks for new streams started, new devices added, and changes in user preferences and rescans the graph, i.e. it scans through all the linkable nodes and links them so that the media starts flowing through the pipeline(rather PipeWire :)).

Design

The whole of this complexity is addressed out of one single file i.e. policy-node.lua and in that file mainly around one single function i.e. handleLinkable. Let me try to outline what this function does.

Not Modular

handleLinkable tries to find a target node for a given stream node(for example pw-play/pw-cat client nodes); for that it first looks for user-preferred targets followed by default targets. If none of them are available, it then looks for the best possible target out of the available targets. Lastly it prepares and links them, so the code is simply not modular.

Redundant Processing

To top it off, there are many hacks and quite a lot of redundant processing. All the WirePlumber Modules and Scripts register for Object Manager Callbacks. Sometimes many of them register for the same callback(for example node-added, default-nodes-changed). Since there is no way to control the order of execution of these callbacks, Modules and Scripts are forced to add redundant checks and processing. As the linking script deals with many generic Object Managers of this kind, it registers so many callbacks and so there are many of these redundant checks and processing, for example the scheduleRescan() is called many times and from many different parts of the code.

This is the not-so-nice part of WirePlumber, coming as I did from my previous company the WirePlumber/PipeWire code & design looked pretty clean, but the way rescan() is handled was difficult to swallow for me. It gets called for way too many events and most of the time the function would return empty-handed. By that, I mean without processing or worse, halfway through the processing, as some condition is not met or some new callback has come that requires abandoning the current pass. Often, it would queue a rescan by itself and exit, which is a classic sign of mediocre design, and so it felt all the more fulfilling and redeeming when we got a chance to clean up the mess with ourselves.

Hacky

Sometimes the linking script changes the properties/state of the Pipewire object in the Object Manager callbacks and some other scripts might be dependent on the state of the same object, in this way the code is prone to race conditions like this. The way out of these situations is hacks.

Design with Event Dispatcher

Rescan runs last

As I poured out in redundant processing, the rescan event kicking in *at the drop of a hat* is the crux of the problem. With the help of Event Dispatcher, rescan is now converted into an event and is accorded the lowest priority and so it gets a chance only after all the dust settles, meaning after all the higher priority events are processed. I would welcome you to take a look at this video from the previous blog, to get a clear idea of how this is achieved.

So the fact that linking happens towards the end, obviated most of the redundant processing and almost all the hacks simply vanished(Abracadabra :)).

Modularity

Like small children, we took pleasure in breaking up the policy-node.lua monolith into multiple pieces. Let me briefly outline how we achieved this.

rescan.lua registers a hook for a rescan event, which runs at the lowest priority and raises a select-target event for that particular source stream node(for example the pw-play/pw-cat client node). Please note that the rescan event is only picked up at the end of processing of higher priority events like node-added/removed, etc. If this is confusing you, I would again point you to the video to connect the dots.

select-event traverses through find* hooks, which help pick the target, the picked target is prepared(prepare-link.lua) and eventually linked(link-target.lua). There are three find hooks, the first one(find-defined-target.lua) looks for defined targets, if there are none defined, the second hook comes into the picture and checks for defined targets(find-default-target.lua). If the target is still not found, it picks the best target with the help of the third hook(find-best-target.lua)

All the hooks run in the aforementioned order, this is made possible with the prioritization mechanism built into them. Hooks are defined with before and after tags(inspired by systemd). For a more detailed dissection of a hook, please take a look at our previous blog post(look for "An example hook" section)

Extensibility

Users can override the default handling of WirePlumber by selecting the right event and registering a user-defined hook with the right priority against this event. Just add this new hook in a separate Lua source file, then add an entry in the wireplumber.conf. And boom! Your hook gets executed, without changing even a single line of upstream code.

For example, find-user-target.lua.example is an example hook that demonstrates how to add a custom way to pick a target for linking. This is a hook registered against the select-target event and it will be the first hook to run for this event.

Remaining system scripts

Applying similar reasoning, the remaining system scripts, like the default nodes module, device profile selecting script, and device routes selecting script, are also logically broken down into rescanning, finding, and applying.

WirePlumber user scripts

WirePlumber user scripts are small to fairly large snippets of Lua code written mostly with object manager, here are a bunch of examples. If you haven't tried one, you should give it a shot, it's a very simple way of harnessing the power of PipeWire through the WirePlumber Lua API.

These scripts are simply run with wpexec. Another method is to copy them to the src/scripts folder and add an entry in the wireplumber.conf. With the wpexec it runs as a separate process and in the later method it runs as a part of the WirePlumber daemon.

Now, for this type of use case, we recommend sticking with object manager. However, if the user is interested in influencing the WirePlumber daemon logic, for example linking, default nodes, profile, routes, etc., we invite him/her to do it via events and hooks. Thanks to Event Dispatcher, WirePlumber can be overridden or extended with ease.