Wim Taymans has a long track record in the Linux community. He was one of the two original developers of the GStreamer multimedia framework and he was the principal maintainer for most of the project’s initial existence. He joined Red Hat in 2013 and has helped maintain GStreamer and PulseAudio for Red Hat since. In 2015 he started working on PipeWire: a project that has come to full fruition in Fedora Workstation 34, where it handles both audio and video. In addition to that, it also merges the world of pro-audio with mainstream Linux. In this interview we will talk about where PipeWire came from, where it is at and where Wim sees it going from here.
Christian Schaller & Wim Taymans testing PipeWire video filters at Flock
Christian Schaller: What was the origin of PipeWire in terms of what problems you wanted to resolve?
Wim Taymans: PipeWire really evolved out of two earlier ideas. The first one was PulseVideo, which was written by William Manley back in 2015. It was a small server that would send the video from a v4l2 camera to one or more other processes. It used GStreamer, DBus and file descriptor (fd) passing to do this fairly efficiently. It resulted in a bunch of patches to GStreamer regarding fdmemory.
Around that time we started to think about screen capture for Wayland. I was asked to investigate options. The idea was then to take the PulseVideo idea and implement the possibility for clients to provide streams as well (not just v4l2 devices). Another requirement was to make this secure and work well with Flatpak and Flatpak’s concept of portals to handle things that have potential security concerns.
CS: Ah right, because when PipeWire was originally introduced to Fedora in Fedora 27 it was only dealing with video right? Providing a way to do screen sharing in GNOME Shell?
WT: Yes, there were only wild ideas about trying to handle audio as well. The version that ended up in Fedora 27 needed another rewrite to make that happen, really.
CS: Can you talk a little about how PipeWire interacts with things like Wayland and GNOME Shell?
WT: GNOME Shell will send a stream to PipeWire when screen sharing is activated. PipeWire will route this stream to the applications like Firefox or the screen recorder. We have some more advanced features implemented such as DMABUF passing and metadata for the cursor and clipping regions when sharing a single window. There is also the volume control that interacts through the PulseAudio API with PipeWire to manage the volumes.
CS: So there was no real PipeWire precursor for video, as most stuff just interacted directly with v4l, so I assume it must have been a big task porting over things like GNOME Shell and the web browsers to start using it?
WT: There was nothing for screen sharing, it was just X11 calls to grab the screen content. Jan Grulich worked with the upstream WebRTC project to add code to interact with the new portal APIS defined for Wayland, to negotiate screen sharing options and then native PipeWire support to fetch the screen content. Then Martin Stransky backported that work into the Firefox copy of WebRTC and Jan Grulich and Tomas Popela ensured the changes got merged into Chromium/Chrome.
For webcams there is not much progress yet. Browsers still access the v4l2 camera directly. There is a portal to negotiate webcam access through PipeWire but that has not been implemented in browsers as far as I know.
CS: Talking about porting and developers, first question developers are likely to ask themselves when they hear about a new project like PipeWire is ‘Oh no, do I need to rewrite all my multimedia applications now?’. How is PipeWire dealing with that challenge?
WT: PipeWire provides compatibility with ALSA, PulseAudio and JACK applications with an ALSA plugin, a replacement PulseAudio server and a JACK replacement client library respectively. Theoretically this should provide a way to run all existing applications without modifications.
With PipeWire, we should now start thinking about those audio APIs as Audio toolkits. It’s a bit like GUI toolkits such as GTK or Qt: both of them talk to the underlying display subsystem (Wayland/X11) and no application thinks about implementing raw Wayland backends in their applications.
It’s the same with JACK/PulseAudio, they provide applications with a model of the Audio subsystem and you select the Audio toolkit best suited for your use case. I don’t see this change unless someone comes up with the ultimate Audio Toolkit.
CS: How did your thinking about the problem space evolve as you worked on it?
WT: As the project went forward, I started to investigate if this framework could also support audio. It would need a substantial rewrite to make this work efficiently. GStreamer and dbus needed to be replaced with something more low level to make audio viable, especially pro-audio. At the same time both GObject and DBus started feeling heavy for the low level system I was designing.
I started experimenting with a new small media plugin API at around mid 2016. It was still all very GObject like but I started to reimplement the v4l2 and audiomixer plugins in this new framework. By the end of 2016, I moved away from DBus as well to a more Wayland like protocol.
Early 2017 was when I seriously started to think about implementing the features of an audio server as well. I started to investigate JACK and its processing model and audio plugin APIs such as lv2. This is also when we came up with the name PipeWire. By the end of 2018 I had a working audio server with a JACK-like graph model, well… at least working in the context of my basic test case.
After some discussions with members of the Linux Pro-Audio community they convinced me that I needed to make some more drastic design changes in the way scheduling and mixing worked if this was ever going to be able to replace JACK for them . This is when the final re-architecting started and eventually became, after 2 years of development, the first 0.3 version in early 2020.
CS: I know the Pro-audio support you mention has got a lot of buzz in the community, so who did you initially talk to and what has the reception been so far from the wider pro-audio community?
WT: As mentioned, I had some discussions with them back in early 2018. Robin Gareus and Paul Davis were instrumental in driving the changes that lead to the current implementation.
I think everybody would love to have a seamless, integrated and user friendly experience that can be used for both Pro and Consumer Audio use cases and there is definitely interest in how PipeWire will evolve to make this happen. We’re not there yet in terms of feature parity although we are moving quickly. For instance, just this week I landed Freewheeling support in PipeWire, which should be out in Fedora by the time you read this. Beyond that latency reporting is the big TODO item remaining. Also, while PipeWire can manage the same latency as JACK we are not yet as reliable. So there is some more work to do.
CS: And what about the PulseAudio developers? How have they taken the arrival of PipeWire? Does Lennart Poettering hate you now?
WT: I think they are fine with it. We organized a hackfest in October 2018 with some of the PulseAudio developers to talk about PipeWire so it was not a surprise. In fact, Arun Ragahavan who is a long time PulseAudio contributor is currently working on PipeWire. I also talked with Lennart about it back in the early days and he was all for the idea of unifying Pro and Consumer Audio so I don’t think he hates me 🙂
CS: You are also the creator of GStreamer, how do you see the two projects in terms of use cases?
WT: I see PipeWire as a much lower-level framework to move data around between apps and devices. It’s very good at handling raw audio and video and interfacing with devices. It’s not so good at muxing and demuxing and it does not want to do some of the higher level multimedia tasks such as implementing an RTSP server or handle transmuxing formats. GStreamer still remains ideally suited for those higher level tasks, muxing, demuxing, encoding, decoding, etc.
CS: So you see them compliment each other more than compete?
WT: They absolutely complement each other. I don’t see one overtaking the other. It’s still early to know exactly where things will go but I can see that things like audio or video effect chains are better implemented in PipeWire. While the plumbing and post processing is better done in GStreamer.
CS: Any community contributors you want to highlight so far beyond yourself?
WT: Absolutely! Almost all of the new exciting Bluetooth work has been done by community contributors.
Pauli Virtanen has been doing fixes all over the place such as many Bluetooth improvements and general fixing and stability improvements to the SCO plugins, implementing codec switching and delay reporting. He also has his hands in other areas such as the PipeWire IPC connections and the default-node and policy in the session manager, as well as some object management improvements.
Huang-Huang Bao (eh5) who maintained a pulseaudio-modules-bt has been contributing a lot of changes such as LDAC ABR support, Hardware volume support and numerous stability and compatibility fixes all over the place to bring the bluetooth support to the same level as the pulseaudio module.
We also have Collabora contributors George Kiagiadakis and Frédéric Danis regularly contributing Bluetooth, build and other fixes as part of their AGL involvement. They have also been working on an improved session manager called WirePlumber, which we will try to include in Fedora 35.
Dmitry Sharshakov implemented the Bluetooth battery status reporting, which is a relatively new feature in bluez and now also supported by PipeWire.
While not directly tied to PipeWire itself ,the work I mentioned earlier by Jan Grulich, Martin Stransky, and Tomáš Popela getting PipeWire support into the web browsers was also a major step forward. The same goes for all the work Jonas Ådahl did to create the screen capture portal and implement it in GNOME Shell. I also want to give a special mention to Georges Stavracas for his great work on getting PipeWire support into OBS Studio. Jan Grulich has also done a lot of work getting PipeWire support into KDE.
There also also a lot of people active on the issue tracker that try to help triage bugs, provide help and improve the wiki pages.
CS: As you’ve been testing and using PipeWire has there been applications you didn’t know about before, but which you discovered due to people reporting they didn’t work with PipeWire or you found when looking for test cases?
WT: Most of the midi tools, really. I never really used midi before I started to add support in PipeWire. I got fascinated by the various synths, like Helm, zynaddsubfx, and more recently Vital and the free Vitalium application.
There is a whole world of music creation tools that become available when you have midi and JACK compatibility that were previously little or unknown. I didn’t know about any of the lsp or calf plugins before.
I love the idea of Inge and I would love to see it developed some more. I imagine that a tool like this can be used to model and tweak the effect chains in PipeWire.
CS: In terms of pro-audio and midi, are you a musician yourself and are these things you see yourself using personally going forward?
WT: I play a little guitar myself but I’m old school, I plug into a real tube amp without effects and I jam. I did some recording of guitar and voice in Ardour using PipeWire to test things out. I’m really more interested in creating code so that other people can make music you actually want to listen to 🙂
CS: What do you feel are the remaining items that need to be tackled in PipeWire?
WT: There is a long TODO list of pending items…
For desktop use cases, we need to reach reasonable feature parity with PulseAudio. We’re missing the automatic detection and setup of network streams along with passthrough of compressed formats such as DTS and AC3 over HDMI.
For the PRO audio use cases we need to implement what in Jack is known as Freewheeling and then latency reporting.
After that, we can start to look at all the exciting new things we can do now with PipeWire. We’re probably looking at a redesign of the sound control panel at some point.
On the video front, a lot can be improved. We don’t have a video processing pipeline yet, let alone the tools to manage such a video pipeline.
CS: Are there any specific areas you would love to see more contributors to in PipeWire?
WT: Sure! I think there are so many exciting things you can make now. For example, we don’t really have a native patchbay. We rely on JACK tools, but those don’t handle the video streams. I would say a simple curses based patchbay would be a nice contribution.
In PipeWire it is relatively easy to write new external sinks or sources. I would love to see a native implementation of a good general purpose network protocol like ROC or so.
CS: You recently started a new job inside Red Hat, can you tell us a little about that and what that means for PipeWire?
WT: Yes, I’m part of the new Infotainment group inside Red Hat that will initially focus on providing the software stack for the sutomotive sector. This is about enabling Audio and Video in cars and PipeWire will play a major part in realizing that. PipeWire is already part of Automotive Grade Linux, together with WirePlumber.
One of the challenges is to be able to route all the audio capture and playback streams in a car in a flexible way. Modern cars also have a large amount of video cameras that need to be managed. Part of the plan is to improve PipeWire for these use cases.
The expectation is that some of these use cases will also benefit desktop users eventually.
S0rvin
The Linux audio pipline infrastructure is a pathetic mess. It’s an accrued tech debt of 20 years. When people talk about open source and online collaboration, the good example you give is the Linux kernel. The bad example is the Linux audio subsystem.
I’m automatically suspicious and wary when people mention “audio” and “Linux” at the same time.
Christian Fredrik Schaller
Well we do hope PipeWire will be a big part of answering some of those issues and over time as PipeWire becomes the de-facto standard it would also allow the ALSA layer to refactor knowing that PipeWire is there and handling a lot of issues.
ventYl
This time, things look rather good. Unlike with PA, which started by bold statements and introduced new, incompatible API, things are being merged and PRO audio is not omitted.
Dutchy
You are overreacting. Please point out specific issues, else your comment is just unwarranted ranting.
Linux audio can actually be remarkably good. But for low latency recording studio quality you need to get your hands a bit dirty and setup jack.
For casual desktop usage, pulseaudio is more than good enough. Even when using bluetooth. It is correct that when things get complex pulseaudio can become annoying. For example when conflicting resampling settings come into play when using wine.
ventYl
Exactly due to the need of PA / JACK co-existence, if you want to do low latency recording, Linux audio is a mess. PA is mostly ignored by studio tools (for variety of good reasons) and JACK is PRO-only. Once you set up JACK you end up PA acting weird to the extent you rather turn it off completely.
I think, that this is rather fitting definition of mess.
Dutchy
I don’t know what your problem was, but for me Jack and PA can coexist perfectly (well, could because I’m still on Fedora 33 and I’m not using pipewire yet).
Perhaps your problem was with specific clients that got confused or maybe PA was claiming the audio interface.
It might not be everybody’s cup of tea, but this is actually the beauty of Linux: the option to choose what fits best for your use case, even if the fit isn’t 100% like a glove (reality is that’s also not the deal on the competing platforms, yet they don’t allow you to use alternative implementations).
Finally having a audio server that combines both the robust and low latency Jack and
the flexibility of PA sounds very exciting to my ears.
I do hope that the regular Jack server can easily coexist with pipewire.
Luiz Carlos
Too many problems. If didn’t was manadtory, good, but I have some problemas after upgrade Fedora 33 to 34 like crackle audio, lostin network communication and after suspect, looking for pipewire on web, see many reports about others problems like wifi and bluetooth.
Christian Fredrik Schaller
We did extensive testing during the beta phase and very few problems showed up. Wim jumped in and fixed a lot of them throughout the process. Of course with something new there are a few birthing issues, but we have been pushing a lot of updates already in F34 to PipeWire and Wim is still very much focused on fixing any bug he is able to reproduce. There is also a growing community of PipeWire contributors so hopefully they can also help us close any gaps found. But it is worth nothing that overall the feedback we get is overwhelmingly positive, a lot of people have found the switch to PipeWire to either be transparent or provide them with a better experience.
Mark
I had troubles on Fedora 34 with audio popping, it turned out to be due to power management turning the intel audio hardware on and off. I was able to fix the pops with:
adding line:
options snd_hda_intel power_save=0
to file
/etc/modprobe.d/alsa-base.conf
david
how to disable pipewire completely when fedora is used on a server without audio card?
Ranomier
It shouldn’t even start. At least it is like this with archlinux. There a systemd-socket is triggered and then it will start.
But you can disable the service or socket
systemctl disable pipewire.socket
systemctl disable pipewire.service
If its running as your user profile:
systemctl –user disable pipewire.socket
systemctl –user disable pipewire.service
Mark
On a server he wouldn’t have even gone out of his way to manually run the –user commands on his user in systemctl, so the service wouldn’t even be starting
Renich Bon Ćirić
Even if it’s working progress, Pipewire really does its part when it comes to pro audio. Latencies are super low and it worked out of the box in Fedora 34; even with flatpak apps. The Jack compatibility was a key feature IMHO.
Well done, Pipewire team!
hammerhead corvette
Congratulations !
skierpage
Nice interview. “So you see them compliment each other…”
GStreamer, you are so good-looking? 🙂 You mean complEment, filling in what the other lacks. (Both your English skills are excellent.)
I’ve had zero audio issues since upgrading to F34, well done.I assumed PipeWire’s compatibility libraries for ALSA, PulseAudio, and JACK were temporary until applications are rewritten to talk to it directly; it’s interesting Wim Tayman doesn’t envision a replacement API.
Feda
Big thank you to everyone working on Pipe Wire. Pro-Audio was sorely lacking in Linux and Pipe Wire is doing a great job of addressing that.
Willdrick
Thanks a lot for PipeWire, and further thank you for mentioning DTS over HDMI. Since updating to F34 this was not working and I was going crazy thinking I must have broken something on PW!
If you need a guinea pig for testing DTS passthrough via HDMI you know who to poke now 😀
Király István
I have a setup that alwas had some video glitches, and that is gone now!
Awesome!
Hm
Awesome, I started using pipewire 6 or 7 months ago it had cracking issues at first but now everything works perfectly. amazing work Dev team
Nathan Myers
I used to use the graphic equalizer module in PulseAudio to compensate for my high-frequency hearing loss. That worked great. Then it went away, for reasons I never discovered.
Is there hope for a first-class graphic equalizer that can be interposed before the audio output in Pipewire, without much pain?
wof.
PulseEffects, ignore the name!
Fabian
Will this make it possible to share a screen or app and have the right audio? Discord for example can’t transmit the audio because there is basically just one source: desktop audio. Same thing with OBS. Had to manually create sinks and loopbacks, then add the apps i want the sound from to these. It was awful, i hope PipeWire will make it possible to manage apps individual audio streams. (Wondering why KDE Plasma can change per app volume but OBS can just get desktop audio as a whole aswell)
Kendy
Thanks developers for try make new sound system. It need next lot of work. Why?
This week I was install Fedora 34 and pipewire was first software what I deinstall and return to pulseaudio.
Why? Because I’m playing games (wine + teamspeak) and with Pipewire sound was crackling, game in wine had lags and low fps. After switch to pulseaudio, lags was gone, fps in normal and sound works perfectly…
Before deinstall I was try change config, but sound still crackling (in log I saw many, many errors about Alsa buffer underrun).
Regards…
James
Hi. My personal compliments for the job. When do you think that PipeWire will be able to replace the legacy audio stacks? Can PipeWire contribute to integrate Dolby digital and DTS audio codifications via browsers, such as Firefox, in order to get audio surround on playing videos on Netflix, as example?
Be
Huge thanks to Wim! I maintain a low latency audio application (Mixxx) and configuring audio I/O is most common and persistent type of user support question we get. It’s no fun to explain how the complex web of PulseAudio, JACK, and ALSA work to someone who just wants audio to work. Getting PulseAudio and JACK to play nice together reliably is no fun either, even for experienced users.
Finally with PipeWire, audio Just Works on Linux for the first time. With PipeWire, I now think Linux has the best audio infrastructure of any OS. Windows is still a mess of different APIs like Linux was, but Windows is worse because not all device drivers support all of the APIs. macOS does not have arbitrary routing of audio between applications and devices built into the audio infrastructure and tools to manage this (there are third party audio servers on macOS, but that’s not the same as having it integrated into the infrastructure and having it Just Work on any system with any application). PipeWire does it all and it works. It’s like that XKCD comic about creating one universal standard to cover everyone’s use case, except it actually does.
Thorsten
Yay, OBS is now finally working as it should. Not so Yay – I haven’t been able to figure out how to get high quality audio out to my headphone DAC/AMP. With pulseaudio this was as easy as adding the following entry to /etc/pulse/daemon.conf
default-sample-format = s24le
default-sample-rate = 9600
But I still haven’t figured out how to do something similar using Pipewire and am wondering if I should just revert to pulseaudio again.
Steffe
I can’t wait until i can remove Pulsaudio, i never liked it.
I find pipewire very interesting and every time i try it it works better and better.
I remember when i fist heard about pulsevideo and i didn’t care for it but the code and features has developed to something new and exciting that is actually useful.
Mattias
Fantastic work Wim and all!
PipeWire is already a great QOL-enhancement in Linux audio for me. And the rate of bug fixes and features – awesome. Looking forward to latency reporting!
Zaro
Thank you guys for your amazing work!
Anthony_325
I upgraded from Fedora 33 and already had jack installed. I found I had to do:
sudo dnf install –allow-erasing pipewire-jack-audio-connection-kit
since pipewire-jack-audio-connection-kit conflicts with jack-audio-connection-kit.
Evert Mouw
Great job. I’ll probably switch my Arch installs to PW this winter.
Question: Are network sources/sinks possible? Like with PA you can distribute a sound stream over multiple computers on your LAN, so your music plays using multiple speakers in various rooms at the same time. From what I’ve read it looks like it is possible, but probably it also can be improved upon (PA play over LAN is often out of sync between computers), is that ware ROC might come in?
Boris
I have upgraded my Fedora 33 to Fedora 34. Since the update, audio is a mess.
Issues: Sound is on “max volumne” by chance, moving the settings slider a litte bit normalizes it again. Setting is not persistent, issue can reappear during the same session.
My audio use case is a) Firefox sounds b) Microsoft Teams audio c) System sounds. Output is played via HDMI in Monitor boxes.
In other words: I have the opposite of a “Pro” usecase.
I followed the recommendation in https://fedoraproject.org/wiki/Common_F34_bugs#Audio_may_not_work_after_upgrade_to_Fedora_34_if_pipewire_was_previously_installed but I had to revert, afterwards there was no sound. I reverted to the previous config.
I’m missing a clear HowTo what “You will then need to re-apply any customizations you had made to the configuration files.” means (I did not make any)
@localhost ~]$ ls /etc/pipewire
client.conf client-rt.conf jack.conf media-session.d pipewire.conf pipewire-pulse.conf
Klaatu
Pipewire is amazing so far, and this is a great interview about it and what’s in store.
Thank you both for this exciting news about pro (and consumer!) audio and Linux!
Felix calderon
Las lecturas siempre son apasionantes, siempre enseñan algo. No se si deba hacer aqui las consultas respecto a pipewire y como hacer que funcionen mis altavoces (5.1). Los componentes de mi equipo estan descritos en el neofetch abajo.
Aqui Las dificultades:
Fedora 34 trabaja muy bien mientras no utilice alsamixer para hacer reconocer los 6 canales, es decir interactua muy bien con firefox google-chrome youtube, etc. y puedo ir escuchando musica simultaneamente. pero una vez que activo un reproductor de musica y configuro para que funcione mi sistema de sonido 5.1, Youtube no arranca o el sistema se CUELGA si ejecuto algun juego, es preocupante esta situacion. he tenido que reinstalar Fedora ante serias dudas y modificaciones realizadas.
No soy un experto en el tema, pero es la primera vez que me pasa esto, cuando era mas joven podia lidear con algunos inconvenientes por que vengo usando linux desde la version de REDHAT 8.
felix@fedora
OS: Fedora 34 (Workstation Edition) x86_64
Host: Z390 UD
Kernel: 5.11.20-300.fc34.x86_64
Uptime: 3 hours, 58 mins
Packages: 2148 (rpm), 2 (flatpak)
Shell: bash 5.1.0
Resolution: 1680×1050
DE: GNOME 40.1
WM: Mutter
WM Theme: mcOS11-Shell-Dark
Theme: Ice-Dark [GTK2/3]
Icons: Mint-X-Purple [GTK2/3]
Terminal: gnome-terminal
CPU: Intel i7-9700KF (8) @ 4.900GHz
GPU: NVIDIA GeForce RTX 2060 Rev. A
Memory: 2012MiB / 15933MiB
Roland
It would be great if some of the multi-speaker setups are supported (5.1, 7.1, Dolby Atmos and more) by Pipewire. Then you could use a Linux box as a media center and use Netflix to play movies with Dolby Atmos sound.
I wonder how far from that Pipewire is. Windows 10 is capable of doing this. Also my Nvidia Shield Pro does this, which runs Android. Android uses a Linux kernel, so at least the kernel part is capable of these multichannel audio protocols
Brecht
I love that I can now simply play my MIDI keyboard while my Spotify is playing and it just works. macOS’s CoreAudio is still unparalleled here, but this feels like a really big step. I’m a fan!
Norbert
Sounds really nice. Develop a systemd replacement next please 😀
newton
realmente nosso sistema Linux deixa ser feito grandes trabalhos dando enfase em grandes sistemas ,sou apaixonado por tantos sistemas bons e de grande validade parabéns