Helvum and EasyEffects: Two great applications for PipeWire users

For a piece of software infrastructure, nothing is more important than good applications to showcase it’s features and demonstrate the possibilities it enables. Fortunately there is a very large body of existing applications because PipeWire implemented the PulseAudio and Jack APIs. This article highlights some of the great efforts undertaken in the community around creating interesting applications for PipeWire in an interview with the maintainers of two of the most popular applications built with PipeWire. They are Wellington Wallace, maintainer of EasyEffects for PipeWire and Tom Wagner, maintainer of the Helvum patchbay application for PipeWire.

Christian Schaller: First of all thank you both for taking the time to talk with me. I am really excited about your projects and I know a lot of people in the PipeWire community are too. So let’s start with introductions, tell us a little about yourself and how you got involved with open source development.

Tom Wagner: I am a 7th semester Computer Science student at the Technical University Berlin, currently working on my Bachelors degree. My first real contact with Linux was in my school, where the computers ran Linux Mint.

I started using Linux on my personal devices a few years later, because at that point I already had some experience with it and because I became increasingly annoyed by Windows.

After some smaller contributions to different open source projects, looking for a project to work on, I found PipeWire and started working on Helvum and PipeWires Rust bindings.

Wellington Wallace: I am a physicist currently working as a professor in a Brazilian federal education center called Centro Federal de Educação Tecnológica Celso Suckow da Fonseca. My first contact with Linux happened in  the computational physics lectures I attended at the beginning of my studies. From that moment on Linux  became the only operating system installed on my computer. But it was only some years later that programming became part of my day to day life.

The focus of my PhD was experimental physics. In order to understand the measurements I did about magnetic nanoparticles I started to do Monte Carlo simulations. First in Python and after some time in C++. But, as it was the kind of simulation that took many days to finish, I eventually started to use OpenCL.

My first open source project was a tool to make graphs in my experimental physics classes. But the one where I really felt I was contributing with something unique to the open source community was PulseEffects. This later became EasyEffects when I switched to PipeWire. The project started one day when Pulseaudio suddenly stopped playing audio in one of the channels. I could have tried to fix the problem but the truth was that I wanted more than just an equalizer. That was how PulseEffects was born. Although I expected it to be useful for some people I did not think this project would get so big.

Christian Schaller: Could both of you give us a short introduction to your project and what it does, for those who might be unfamiliar?

Wellington Wallace: EasyEffects(formerly known as PulseEffects) is a tool that allows the user to add global audio effects for applications. Among the possibilities are dynamic range compression, equalization, automatic volume control, noise reduction, bass enhancement, etc. Besides applications output, users can add effects for microphones too.

Tom Wagner: PulseAudio is mostly designed to have applications receive audio input from one single input device and/or send audio to a single output device, which makes it difficult to adapt to more complicated use-cases.

Contrary to PulseAudio, PipeWire runs a full media graph. This means that audio and other kinds of media can be freely routed between any number of devices and even to other applications at the same time.

For example, you could route the audio output of your music streaming software to your speakers as well as your voice-chat software to listen with others, while routing the audio and video of a game to a recording software to make a recording without the music in it.

Helvum is a GUI to do exactly this: create and delete connections between devices and/or programs to route data through the system in exactly the way you want it.

Christian Schaller: Wellington, so EasyEffects started out as an application targeting PulseAudio, but you switched it over to PipeWire relatively early on. What made you decide to jump on and target PipeWire with EasyEffects?

Wellington Wallace: Actually, between the first time users asked for PipeWire to be supported and my move to do that, took about 2 years. It may seem, for some people, that it was early. But almost all that we needed was already available.

Besides the usual “low latency” argument people give when talking about PipeWire, there were other  reasons that were important from an application implementation point of view. Pulseaudio does not offer a way to naturally do the kind of processing we do. There is no support for third party plugins in Pulseaudio that could be used to write filters. What is done in PulseEffects, in a way, is a hack. I had to “take the audio away from the server” so I could process it in GStreamer. After everything was done I had to send the result to Pulseaudio. It may seem simple but that approach had lots of problems with clock drift in the beginning. Audio got out of synchronization so fast that I almost gave up on the project. It took lots of attempts with different GStreamer configurations until I found a way to make the clock drift negligeable.

In PipeWire the whole process feels natural and the filter API is super simple. Way easier than writing GStreamer plugins. Handling the pipeline is a lot easier too. I like GStreamer but sometimes removing or inserting plugins from/to its pipeline on the fly can be a nightmare….

Another thing we get for free on PipeWire is realtime priorities in the filter thread. On Gstreamer I had to handle that myself. That was mostly fine. But, since Flatpak did not have RealtimeKit support, people using PulseEffects through Flatpak could not have realtime priorities in our audio thread.

Another reason that indirectly contributed to the early adoption was the release of gtk4(gtkmm4). The combination of  Pulseaudio and GStreamer did things in a very different way when compared to PipeWire. Moving from  gtkmm3 to gtkmm4, while keeping Pulseaudio and GStreamer, would make me work twice in some places once the move to Pipewire was made. If I had to change the graphical interface anyway it is better to do it once using the new audio server.

Christian Schaller: Tom, so Helvum is a brand new application written explicitly for PipeWire. What was your motivation for starting this project?

Tom Wagner: I used to play World of Warcraft and wanted to record my guilds raids with separated game and voice chat audio tracks so I could mute the voice chat for some clips and keep them for others.

This was fairly complicated and painful with PulseAudio, where I had to do some complicated setup of creating multiple virtual loopback sinks to first record these audio streams separately and later combine them to send them to my headphones.

I was looking for some open source project to contribute to, and became interested in PipeWire, where this could be achieved much easier by rerouting the media graph.

Back then, there was no native PipeWire patchbay, so you had to rely on JACK tools.

While those could be used to route audio and MIDI through a compatibility layer, JACK and therefore any JACK tools can’t handle video, so as my contribution to the open source community I started work on PipeWire Rust bindings and, using those, came Helvum.

Christian: Wellington and Tom, what has your experience been working with Wim Taymans and the PipeWire community so far?

Wellington Wallace: I consider it a good experience. Whenever I need help to get things done in PipeWire Wim Taymans responds reasonably fast. Especially when considering the amount of work he has on his hands now.

Tom Wagner: It’s been great so far. The IRC channel (#pipewire on OFTC), including Wim, is always able to answer any questions that come up quickly, and we already had many great contributions to both Helvum as well as the Rust bindings.

Christian Schaller: Wellington, so what are your plans for EasyEffects going forward? What do you see as the next steps for EasyEffects?

Wellington Wallace: At this moment I am in the process of using gtk4 API directly. GTKMM has served us well but now that I want to use LibAdwaita it will be more straightforward to talk to gtk4 directly. It is not the first time I do something like that. I started this project in Python. Its gtk wrapper is good but eventually it got in the way. And now the same thing is happening with gtkmm.

Besides that there are 2 more things that are on the horizon but for which I do not have a solution yet. The first is adding support for multiple instances of a given filter. For example some people would like to have more than one equalizer instance in the pipeline. At this moment this is not possible. Another rough edge in EasyEffects is the lack of upmixing from stereo to surround devices. But I still think that some of the work required for that would also have to be done on PipeWire.

Christian Schaller: And Tom, where do you plan on taking Helvum from this point forward?

Tom Wagner: At the moment, to configure things such as volume, default audio devices etc. via GUI, the best way is still via PulseAudio applications such as pavucontrol.

I’d like for Helvum to become a full PipeWire management interface, able to handle all these use cases natively.

There are also many quality-of-life improvements such as zooming and better scrolling that are still missing.

Christian Schaller: Tom and Wellington, what do you like the most about PipeWire and do you have any wishes or feature improvements you would like to see from PipeWire?

Tom Wagner: The best thing about PipeWire for me is definitely its flexibility and versatility.

It allows routing data through the system any way you want, even between PulseAudio and JACK applications, without any trouble. It lets power users run their audio with minimal latency, and still remains a drop-in pulseaudio replacement even for casual users.

From my perspective, PipeWire is already in really great shape, and while there can still be some issues, especially with less common setups, it keeps getting better with each release.

Wellington Wallace: Some of my thoughts on that are scattered in my previous comments. It has a simple API that fits our needs and the latency is low enough. Another thing I consider important is a short release cycle. Pulseaudio takes too long to make new releases. I understand that it is an old project by now but I saw in the past some annoying bugs being fixed in Pulseaudio’s development branch that took forever to get to the users. I hope that PipeWire keeps the current approach where releases are done more frequently.

About the improvements, it depends on the point of view. Thinking about it as a user I think PipeWire still has some rough edges when it comes to on-the-fly device connection/disconnection. Like headphones, for example. It still resets volumes to 100% or mutes them in cases that Pulseaudio did not do. It is not often that this happens. But it still happens.

From the EasyEffects point of view I think that the process available to link filters needs some love. Right now if a filter has to link to a surround, or any other device that has a different number of channels, the whole upmixing/downmixing process becomes the audio clients (EasyEffects) responsibility. As I do not have the knowledge for that, the current situation is that there is no upmixing or downmixing being done in these cases. We did a few attempts with the loopback module but it does not fit very well with how we do things. It also generates additional streams that will only confuse the user and increase the chances our audio routing is broken by something that decides to move the loopback streams somewhere they should not be. In an ideal world I would like to have a way to just tell PipeWire “link this filter’s channels to this input/output device and do upmixing/downmixing if necessary”. 

Christian Schaller: So if anyone wants to contribute to either Helvum or EasyEffects what is the best way to get started and get involved?

Wellington Wallace: Up to now 65 people have contributed in some way to EasyEffects(PulseEffects). In all these cases either a pull request was done at https://github.com/wwmm/easyeffects or an issue was opened to discuss things first. In many cases people contributed only with an idea. If it is something that can be done and is compatible with how EasyEffects works, I or someone else usually implements it when time is available. 

Tom Wagner: Similar to EasyEffects, reporting any problems or pet peeves to the issue tracker already goes a long way towards improving the application.

From there, anyone can start work on a Merge Request to fix the issue. Any questions and requests for help are also welcome there, and I’ll do my best to answer them.

Christian Schaller: Are there any projects you would like to see started with PiperWire that you feel are clearly outside the scope of your application?

Tom Wagner: Anything that involves processing the actual data that is being sent is definitely out of scope. For example, people have already asked for support for hosting plugins such as VST and OFX plugins. These would be better handled by a separate app, as the focus for Helvum is on controlling the PipeWire server, not the data being routed through it.

Wellington Wallace: I think that what Helvum is trying to achieve is exactly the kind of thing I have always considered out of EasyEffects scope. I want to focus on providing as many useful effects as possible in a way that is flexible enough for advanced users without making new users too scared. Managing devices, like Pavucontrol does for Pulseaudio and to some extent to PipeWire, is something that another application should do.

Christian Schaller: Tom, from what I know, you are a Fedora Linux user. What has made it your distribution of choice?

Tom Wagner: Yes, after a while of using Arch, I wanted something that works well out of the box, and after trying a number of different distros I stayed with first Fedora Workstation and now Fedora Silverblue.

I love how up-to-date yet stable and polished everything is in Fedora Linux. For Silverblue, specifically, I love how I can let it update in the background without any fear of things breaking and then reboot into the new updated version later without any waiting, and rollback to a previous deployment or even reset to a pristine state if I break something.