We're hiring!
*

Smart audio filters with WirePlumber 0.5

Julian Bouzas avatar

Julian Bouzas
June 26, 2024

Share this post:

Reading time:

After more than 2 years of development since WirePlumber 0.4 was released, WirePlumber 0.5 is finally here, and with it, a lot of new features and improvements are now available. One of the most important features of this latest major release is the Event Dispatcher, but since there is already a blog post dedicated to it by my colleague George, I want to shift the focus to another interesting feature, the Smart Filter Policy.

What are audio filters?

As you are probably already imagining, an audio filter is just a processing unit that transforms the frequencies of an audio signal. Those can be applied to a wide range of user cases and, similar to PulseAudio, PipeWire already provides modules that include the most common audio filters. Some examples of those PipeWire modules are the Filter-Chain module, which constructs a “pipeline” filter from a chain of other filters: Both third party filters from other projects such as LADSPA or LV2; built-in PipeWire filters (equalizers, spatializers, mixers, etc...); the Echo-Cancel module which cancels echo from capturing devices by monitoring their respective playback device; and even the Loopback module which does nothing but forward the audio signal as it is, without changing any of its frequencies... this one is mostly useful when the user wants to hide a specific device.

How do audio filters work in PipeWire?

In PipeWire, audio filters are always represented as a pair of 2 nodes: one virtual client node and one virtual device node. This is done so that users can control (and change) the real device linked to the output and the real client sending audio to the input of the actual filter. These use existing GUI tools, such as PulseAudio Volume Control or the native WirePlumber CLI tool, wpctl, without exposing those details to regular applications.

If we want to see the PipeWire graph when one of those filters is being used, it would look like this for sink filters:

Collabora - WirePlumber Sink Filter

And like this for source filters:

Collabora - WirePlumber Source Filter

What is the smart filter policy in WirePlumber 0.5?

Up until now, audio filter nodes were always treated like regular nodes in WirePlumber. This meant that if you only wanted to use a particular audio filter with a specific device, you would have to move it manually every time a new audio device was connected (if this device was not the default). Apart from this, you would also have needed to move the client stream to target the actual filter and not the real device. This is usually the case for the Echo-Cancel filter, because we only want to use it when the speakers and the microphone devices are being used, but not if the user has a headset connected; headsets don't play loud audio so there is really no need cancel echo when using the attached microphone, it would be redundant and would waste CPU cycles.

This manual user intervention is usually not a big deal if we only have 1 filter, but for more complex systems, if we have more and we want to chain them together, it starts to become annoying and cumbersome. For every new filter we need to manually select what device is going to be linked with the filter's output and what stream is going to be linked with the filter's input. All of those reasons are why we had the motivation to implement an automatic or "smart" audio filter policy in WirePlumber 0.5.

The smart filter policy is defined by a set of properties that can be set in the virtual device node of an audio filter (The one with Audio/Sink or Audio/Source media classes). Some of those properties are the following:

  • filter.smart: If set to true, WirePlumber will apply the smart filter policy on this filter
  • filter.smart.name: A unique name identifier for this filter
  • filter.smart.before: list of filters that are meant to be chained before this filter
  • filter.smart.after: list of filters that are meant to be chained after this filter
  • filter.smart.target: the matching properties of this filter’s final target node. WirePlumber will use this property to know if it has to chain multiple filters together respecting the before and after properties when they have the same target.

As you can already guess, thanks to those properties we can control and automate how filters will be linked and chained together. For example, let's say we have 3 output filters, A, B, and C, and 2 output devices, D and E; we can instruct WirePlumber to chain filters A and B together and only use them with device C. On the other hand, we can also configure filter C to only use the device E. To do that, we must set their properties like this:

Collabora - WirePlumber Smart Filters A B

Collabora - WirePlumber Smart Filters C

Note that by default, if the filter target is not found, the filter will automatically be linked to the default device. This is because all client nodes have the same behavior in WirePlumber, even virtual clients from filters. However, it is possible to change this behavior so that filter client nodes are not linked to anything if their target does not exist, and stay like that until their target device is available. To do this, we need to set 2 client node properties:

  • node.dont-fallback: If set to true, the node's target won't fallback to the default node if its defined target could not be found. Instead, it will send a "defined target not found" error to the client and the node will be destroyed.
  • node.linger: If set to true, WirePlumber won't send a "defined target not found" error to the client or destroy the node when its defined target was not found. Instead, it will leave the node unlinked. This is only useful when the 'node.dont-fallback' is set.

Note that these properties must be set in the virtual client node of the filter (The one with Stream/Ouput/Audio or Stream/Input/Audio media classes), instead of the filter virtual device node. With that in mind, setting both of those properties to *true* will instruct WirePlumber to treat those filters exactly as we want.

I recommend checking the WirePlumber smart filters documentation for a list of all the available properties here, and also the linking properties for a list of all properties that can be used in client stream nodes here.

Hopefully this small example will give you an idea of how the smart filter policy works in WirePlumber 0.5. The important thing to remember here is that these feature filters insert themselves in between client streams and devices automatically and transparently for you. In contrast, the client stream still targets the real device. This is huge as it allows defining filter configurations tied to specific devices without having to change the default device nodes in the metadata.

In fact, as a real world application, this solution was embraced by gaming users because it was very convenient for them to not configure filters manually while playing games, especially if they had to enable or disable their microphones constantly.

Even better for users (gaming or otherwise) their systems can be pre-configured to apply the correct filters (such as echo or noise cancellation) only when devices, for which it makes sense, are selected: The game or application does not have to know what the input or output is and doesn't have to request specific filters (or even know that anything has changed).

To conclude, the smart filter policy feature is still experimental and might get further improvements in the future. I recommend playing with it and giving feedback upstream if you think it can be improved even more.

Comments (0)


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest Blog Posts

Building a Board Farm for Embedded World

27/06/2024

With each board running a mainline-first Linux software stack and tested in a CI loop with the LAVA test framework, the Farm showcased Collabora's…

Smart audio filters with WirePlumber 0.5

26/06/2024

WirePlumber 0.5 arrived recently with many new and essential features including the Smart Filter Policy, enabling audio filters to automatically…

The latest on cmtp-responder, a permissively-licensed MTP responder implementation

12/06/2024

Part 3 of the cmtp-responder series with a focus on USB gadgets explores several new elements including a unified build environment with…

A roadmap for VirtIO Video on ChromeOS: part 3

06/06/2024

The final installment of a series explaining how Collabora is helping shape the video virtualization story for Chromebooks with a focus…

Hacking on the PipeWire GStreamer elements

05/06/2024

Last week I attended the GStreamer spring hackfest in Thessaloniki to work on the PipeWire GStreamer elements and connect with the community.

Transforming speech technology with WhisperLive

28/05/2024

The world of AI has made leaps and bounds from what It once was, but there are still some adjustments required for the optimal outcome.…

Open Since 2005 logo

Our website only uses a strictly necessary session cookie provided by our CMS system. To find out more please follow this link.

Collabora Ltd © 2005-2024. All rights reserved. Privacy Notice. Sitemap.