Introduction
The PipeWire project is slowly getting popular as it matures. Its documentation is still relatively sparse but is gradually growing. However, it’s always a good idea to have people from outside the project try to grasp and explain it to others in their own words, reiterating ideas, seeing them from their own perspective.
In a previous
posts
I went over the generic audio stack on Unix and had a section mentioning
PipeWire. Unfortunately, because at the time I didn’t find enough docs
and couldn’t wrap my head around some concepts, I think I didn’t do
justice to the project and might have even confused some parts.
In this post I’ll try to explain PipeWire in the most simple way possible,
to make it accessible to others that want to start following this cool
new project but that don’t know where to start. It’s especially important
to do this to open the door for more people to join in and follow the
current development, which is happening at a fast pace.
Disclaimer: I’d like to preface this by saying that I’m not a developer
involved in the project, only an internet traveler that got interested. I
might still have made mistakes and do not cover everything, so be sure to
leave comments or emails so that I can correct or add info.
PS: If you like deep dives and similar discussions then check the
nixers community, it’s full of people that love
to do that.
PPS: I’ve used a similar name for this
article as the fabulous PulseAudio Under the
Hood but I
don’t think I go in as much details as the author did for PulseAudio.
Table of Content:
- Introduction
- What’s PipeWire — Quick Test Run
- Relation To Gstreamer
- The Blocks: POD/SPA
- The PipeWire Lib, An Inspiration From Wayland
- Configuration
- Tools & Debugging
- Conclusion
- References
What’s PipeWire — Quick Test Run
PipeWire is a media processing graph, this might not ring a bell so let me rephrase. PipeWire is a daemon that provides the equivalent of shell pipes but for media: audio and video.
What lives in this graph are nodes that can represent multiple things,
from real devices such as headsets or webcams, to virtual ones such as
audio filters.
These nodes have ports, and these ports can be linked together, the
media flowing from the first node’s source in the direction of the next
node’s sink. What happens in each node is up to them and the interfaces
or functionalities they provide.
Practically, the nodes, links, ports, and others, are all different objects extending a basic type, and live in this graph. These objects don’t necessarily have to be media-related either, they can do a lot of other things. We’ll see that they use a special type system, plugin system, and marshalling format/storage.
All to say, that PipeWire is a graph, “it’s mechanism and not policy”.
That means that to create and interact with that graph we need another
piece of software. The standard way is to rely on what PipeWire calls
a session manager or policy manager. The role of such software is to
create and manage the entities in the graph depending on the environment,
such as when a device is plugged in, or a restore volume policy is set,
or permissions needs to be checked before allowing a client to access
a device.
Currently, there are two implementations of such session manager:
the default one called pipewire-media-session, and a work-in-progress,
but extremely promising and interesting one, called WirePlumber.
Yet, you can choose to build your own session management workflow by
relying on external tools to manage what is in the PipeWire graph.
Let’s cut this short, and have a look at how to run PipeWire.
Get it from your package manager, and do the following in a terminal.
$ pipewire
And in another one do:
$ pipewire-media-session
In some cases, as we’ll see from pipewire configuration, you might not
need to execute the second command, because pipewire could be set to
automatically start pipewire-media-session
.
Try a ps
beforehand just to be sure.
Still, doing the above won’t get you anywhere if no application can
speak with PipeWire — only a handful do at the moment. Furthermore,
PipeWire GUIs and toolings are still lacking, as we’ll see.
That is why, PipeWire offers three compatibility layers: with ALSA
through a PCM device, with PulseAudio through pipewire-pulse server,
and with Jack through the pw-jack
command.
To make sure you got the ALSA compatibility, check
/etc/alsa/conf.d/50-pipewire.conf
which should define a pcm
for pipewire, often created when installing the pipewire-alsa
package. Then check if this has become the default pcm by
dumping ALSA configuration through alsactl dump-cfg
or aconfdump and
verifying the pcm.default
entry (usually through something like
99-pipewire-default.conf
). However, it doesn’t have to if you still want
to rely on the PulseAudio shim, in that case it would be type pulse
.
For the PulseAudio layer, which allows using PulseAudio tools with
PipeWire, install pipewire-pulse
and start it. Your distribution package
might even start it automatically along with the session manager through
the init/service manager.
Finally, for jack, install the pipewire-jack
package and issue
pw-jack
before any jack related command and they will automatically
use PipeWire. For example:
pw-jack qjackctl
Additionally, for the video functionality you should install a desktop portal, which is a dbus service implementing the xdg-desktop-portal specifications and whose job is to check if the client requesting access to the video is allowed. There are many such software:
- xdg-desktop-portal-gtk
- xdg-desktop-portal-kde
- xdg-desktop-portal-wlr
You should be familiar with these if you are running a Wayland compositor as this is the only way to access video on Wayland (webcam, screen sharing, screenshot).
Here you go, PipeWire is running!
Yet, there’s a lot more to see than that, such as how to configure
PipeWire and the session manager, how to write software that relies on
PipeWire, the thinking that goes into it, how to manipulate the graph
and rely on tools, and plenty of other examples.
Relation To Gstreamer
The best way to understand the ideas behind PipeWire is to take a look
at GStreamer, which it inspires itself from and which is maintained by
the same lead developer (a nice and welcoming person).
Even though the author wants to make the comparison with JACK
instead.
You could take a look at JACK and extract the concepts you want from it,
but we’ll go for GStreamer in this article.
GStreamer plays around the concept of a pipeline created from objects added in a “bin”, its version of a graph. Like PipeWire, media flows from one end to another. In Gstreamer’s world, instead of nodes there are GstElements, instead of ports there are pads attached to GstElements, and links are connections between pads.
The resemblance goes deeper, GStreamer relies on GObject, which, if you are unaware, is a C framework/programming model to make development easier. It allows it to do multiple things such as registering for asynchronous events (called signals) that are triggered by GstElements present in the pipeline. GObject also has a main loop management, a type system, introspection mechanisms, and other goodies.
GStreamer offers different types of GstElements as plugins created
through “factories”. In fact, these are GObjects extending the
common GstElement type to provide useful functionalities. There’s a
list of them available in the GStreamer documentation. For example
aasink.
Some are for videos, some for audio, some for logging, some for conversion
between format and negotiation, etc..
These elements can also be introspected on the command line via the
gst-inspect-1.0
tool. This makes it wonderfully easy to program with
GStreamer.
Similarly, PipeWire has plugins that extend a basic node element
type. However, it doesn’t rely on GObject/GstElement but on its own
simpler plugin system, appropriately named SPA (Simple Plugin API). It
also has factories, a loop management system, asynchronous events, and
a message passing format called POD (Plain Old Data).
Think of PipeWire as a simpler Gstreamer running as a daemon, with a fully
controllable loop, and that relies on a session manager to automatically
create the graph/pipeline based on streams and devices that appear.
Yet, PipeWire is still missing documentation and introspection tools as excellent as GStreamer. Plugins are barely documented yet. I have to say, GStreamer is a really well-done project and I hope that PipeWire will soon be the same.
You can take a look at the following useful command line tools:
gst-discover-1.0
gst-inspect-1.0
gst-launch-1.0
An example usage of the creation of a pipeline:
gst-launch-1.0 videotestsrc pattern=1 ! optv ! videoconvert ! autovideosink
GStreamer is still relevant with PipeWire, as GStreamer can now integrate with it. “GStreamer is intended to be a Swiss army knife of multimedia”. New plugins for PipeWire are available under the names:
pipewiresrc
: PipeWire sourcepipewiresink
: PipeWire sinkpipewiredeviceprovider
: PipeWire Device Provider
Learning about GStreamer is a great way to better understand PipeWire, at least it was for me.
The Blocks: POD/SPA
The GObject, the GLib Object System, is well-known and battle-tested,
but it’s not what PipeWire uses because of its heaviness. PipeWire relies
on SPA and POD.
So what are SPA (Simple Plugin API) and POD (Plain Old Data)?
POD — Plain Old Data
POD, the Plain Old Data (not to be confused with Perl’s Plain Old Documentation) is a generic data container format for marshalling/unmarshalling, serialization/unserialization, storage, and transfer. It’s the usual flat passive data structure.
Think of it as yet another format, similar to XML, JSON, ASN.1, protobuf, and others. It inspires itself from formats such as D-bus Variant and LV2 Atom.
Practically, it’s an LTV (Length-Type-Value) format, thus using
octet-counting framing, where the length and type are fixed 32bits values
and the frames are always 8 bytes aligned. So padding is often added to
values that don’t align on it. (NB: Framing refers to the concept of start
and end of value, how to delimit.)
The type system, what the 32 bits T
refers to, is called the SPA type
system and has compound/container and basic/primitive types. These range
from containers such as a array, struct, object, sequence, pointer,
file descriptor, choice, to primitives such as bool, int, string, etc..
The advantage of such format is that it’s an “as-is” format, it can
directly be transferred on the network, read from memory, stored on the
stack or on disk without extra marshalling.
NB: If you want to know more about protocol/format design in general refer to RFC 3117.
The POD library wasn’t designed to be specifically used for PipeWire,
it can be used in any other project, though the question remains of why
use this format instead of another.
The library is a small
header-only C library
without dependencies, which makes it a breeze to test with.
I’ve published an example of its
usage
based on the official
tutorial but let’s still go over the general aspect of it.
The best documentation for POD is to consult the headers themselves, as
a lot of helpers aren’t documented anywhere else. You can usually find
them in /usr/include/spa/pod/
or /usr/include/spa-<version>/pod/
(just be sure it’s the latest version). It comes bundled in
the same directory as SPA as it relies on the type ID system in
/usr/include/spa/utils/type.h
and defs.h
.
The pod structure struct spa_pod
is defined in /spa/pod/pod.h
along
with the primitive and container values. The builder to construct them are
found in /spa/pod/builder.h
, while the parser is in /spa/pod/parser.h
,
and the manipulation can be done with helpers in /spa/pod/iter.h
,
/spa/pod/filter.h
, and others.
Practically, to create a pod we initialize any part of memory, be it in the heap or stack, and use a builder helper to initialize it. Then we have to rely on a frame to set the start of a container object and its end, basically setting the final size of the object (LTV as we said) when we’re done. The frame acts as a sort of push and pop of value on the memory segment we chose.
Here’s a taste of what it looks like, you can consult my
example,
the official docs, or the headers directly to know more.
Compilation should be as simple as cc pod-test.c -o pod-test
.
We define any sort of storage, here 256B on the stack and tell the pod builder we’ll use it to store pods.
uint8_t buffer[256];
struct spa_pod_builder b = SPA_POD_BUILDER_INIT(buffer, sizeof(buffer));
We can then define a frame for a container object, here a simple struct, but it could be objects (key/value, called properties which also have their own IDs) or other complex types.
struct spa_pod_frame f;
spa_pod_builder_push_struct(&b, &f);
Once the frame starts we can add values to the struct.
spa_pod_builder_int(&b, 5);
spa_pod_builder_float(&b, 3.1415f);
Finally, we close the frame to say we are done with the struct, and it
returns the struct spa_pod
basic type which can later be casted to
its appropriate type.
struct spa_pod *pod = spa_pod_builder_pop(&b, &f);
struct spa_pod_struct *example_struct = (struct spa_pod_struct*)pod;
This will look like this in memory (little-endian formatted):
length = 00000020 = 32
type = 0000000e = 14 = SPA_TYPE_Struct
value =
length = 00000004 = 4
type = 00000004 = SPA_TYPE_Int
value = 00000005 = 5
_padding = 00000000 (always align to 8 bytes)
length = 00000004 = 4
type = 00000006 = SPA_TYPE_Float
value = 40490e56 = 3.1415
_padding = 00000000 (always align to 8 bytes)
The types can be found in /spa/utils/type.h
and others files. We can
also notice the padding for alignment, which was automatically added by
the builder.
Once we have a pod of a specific type created, we can handle it with
related helpers. In the iter.h
header we can find ways to loop over
struct spa_pod_struct
or ways to verify that a pod is indeed a struct.
struct spa_pod *entry;
SPA_POD_STRUCT_FOREACH(example_struct, entry) {
printf("field type:%d\n", entry->type);
// two ways to get the value, casting or using spa_pod_get_..
if (spa_pod_is_int(entry)) {
int32_t ival;
spa_pod_get_int(entry, &ival);
printf("found int, pod_int: %d\n", ival);
}
if (spa_pod_is_float(entry)) {
struct spa_pod_float *pod_float = (struct spa_pod_float*)entry;
printf("found float, pod_float: %f\n", pod_float->value);
}
}
As you noticed, struct spa_pod
are generic and only contain the
first two 32bits value of a pod, namely size and type. Thus, we have to
interrogate its type to find out what it actually is, and then cast it
manually or using helpers such as spa_pod_get_*
found in iter.h
.
Instead of doing that, we can instead rely on an spa_pod_parser
and
other helpers to do the validation and inspection of raw data (see in
iter.h
spa_pod_from_data
for example).
Yet, it would be annoying to manually check values when you only want
to glance at its format. That’s why debugging functions are present in
/spa/debug/pod.h
. The spa_debug_pod_value
is particularly useful
to print what’s in a pod. We could also use /spa/utils/json.h
and /spa/utils/ansi.h
to print structures as json, like is shown
here, but there’s no simple function to do that yet.
There’s a lot more to POD than this, it’s a whole object format, but let’s stop and move to SPA.
SPA - Simple Plugin API
While POD is just about data representation, SPA is about functionality. SPA, the Simple Plugin API, is a header-only and no dependencies framework that gives the ability to load libraries that have a specific format, enumerating factories, creating them, and the interfaces they provide that can be introspected and used, all at runtime.
Like POD, it is not necessarily tied to PipeWire, but can be used anywhere, though PipeWire got its own plugins using SPA.
The objects that are created by the factories in the SPA have a specific format, often internally relying on POD, and the SPA factory interfaces let us know the functionalities associated with them.
Practically, plugins using SPA take the form of dynamically loadable
libraries (.so), usually found under /usr/lib/spa/
, or the env
SPA_PLUGIN_PATH
, and opened on the fly using dlopen(3)
.
The SPA libs have at least one public symbol spa_handle_factory_enum
,
defined in /spa/support/plugin.h
, which is loaded as follows.
#define SPA_PLUGIN_PATH "/usr/lib/spa-0.2/"
void *hnd = dlopen(
SPA_PLUGIN_PATH"/support/libspa-support.so",
RTLD_NOW);
spa_handle_factory_enum_func_t enum_func = dlsym(
hnd,
SPA_HANDLE_FACTORY_ENUM_FUNC_NAME);
I’ve also written a simple
example
to show SPA usage, but let’s review the mechanism quickly as
this is important to grasp PipeWire configuration.
Compiling should also be as simple as cc test-spa.c -ldl -o test-spa
.
A plugin is composed of a list of factories, and in each factory we have a list of interfaces available. The idea is that factories are a set of methods used around a struct of a specific type, from its creation, to different interactions with it. Interfaces are about anything that can be bundled together.
Factories have well-known names and the interfaces in them have names
too, they are defined in /spa/utils/names.h
and the interfaces names are
stored in the plugin header themselves in the form SPA_TYPE_INTERFACE_*
.
Factories also have additional information such as version, author,
description, and more.
This is what we need to know to use the library after loading
it. Yet, we have to consult the header files and the library location
/usr/lib/spa-<version>
to know which factories are currently available and
how to use them. In that library directory, folders are present displaying
the categories of plugins, most of them related to PipeWire.
If we take a look at support/libspa-support.so
, we see that it relates
to the support
headers we got. We can grep on SPA_TYPE_INTERFACE_
to find the headers that have plugins in them to be sure. Let’s see
support/log.h
as an example.
The factory name is SPA_NAME_SUPPORT_LOG
and the only interface defined
is SPA_TYPE_INTERFACE_Log
. Internally, we see that all plugins using
SPA define two things: a struct containing a struct spa_interface
and
another struct containing the methods related to that interface along with
the version, additionally it can define events and callbacks. If you’re
curious struct spa_interface
is defined in /spa/utils/hook.h
. However,
the inner workings are not important, what’s important is that we can
take a look at which functions are in an interface which dictates what
we can do with the plugin.
A series of command line tools starting with spa-
are present to
interact with the plugins. One in particular called spa-inspect
can
be used to dump the factories and interfaces present in a .so file. Yet,
it’s not very solid and doesn’t confer that much information on what
we can do with the plugins.
Example output:
factory version: 1
factory name: 'support.log'
factory info:
none
factory interfaces:
interface: 'Spa:Pointer:Interface:Log'
factory instance:
interface: 'Spa:Pointer:Interface:Log'
Thus, after loading the .so
and getting the enum_func
function,
we can loop over the factories, find the name we are interested in,
then loop over its interfaces to see if it has the one we want, and get
an instance of the factory to use it.
uint32_t i;
const struct spa_handle_factory *factory = NULL;
for (i = 0;;) {
if (enum_func(&factory, &i) <= 0)
break;
printf("factory name: %s, version: %d\n",
factory->name,
factory->version);
if (strcmp(factory->name, SPA_NAME_SUPPORT_LOG) == 0) {
const struct spa_interface_info *info = NULL;
uint32_t index = 0;
// get interface at position 0
int interface_available =
spa_handle_factory_enum_interface_info(
factory,
&info,
&index);
if (strcmp(info->type, SPA_TYPE_INTERFACE_Log) == 0) {
// allocate a handle (struct) pointing
// to this factory's interfaces
size_t size = spa_handle_factory_get_size(
factory, NULL);
struct spa_handle *handle = calloc(1, size);
spa_handle_factory_init(factory, handle,
NULL, // info
NULL, // support
0 // n_support
);
// fetch the interface by name from the factory handle
void *iface;
int interface_exists =
spa_handle_get_interface(
handle,
SPA_TYPE_INTERFACE_Log,
&iface);
// finally get something useful by casting it
struct spa_log *log = iface;
// use the methods in the interface of the factory
spa_log_warn(log, "Hello World!");
spa_log_info(log, "version: %i", log->iface.version);
// clear the handle to the factory
spa_handle_clear(handle);
}
}
}
This long example has all the parts we discussed and something more:
- Loop over the factories present in the library and match by name
- Get the name of the first interface (though we can loop over
spa_handle_factory_enum_interface_info
until it returns 0) and match by name. - Allocate a
struct spa_handle *
which is a pointer to all interfaces in the factory, a way to construct it. (We have to fetch its size to do that, I’m sure there could be a better helper to make this easier in the future). The factory creation could possibly take parameters. - Get our interface from the created factory using
spa_handle_get_interface
, then cast it to the struct we talked about earlierstruct spa_log
. - Finally, use the methods associated with that struct.
There are a lot of plugins and different ways to use them. Some interfaces are asynchronous and use callbacks through registered event handlers and hooks, some are synchronous.
This is a general idea of what SPA is. It might seem cumbersome and kind
of underwhelming: fixed factory names defined in header files, along with
fixed interfaces and a common way of fetching them manually from dynamically
loadable libraries, to then use the methods in it based on the info.
Now let’s see how POD and SPA are actually used within PipeWire.
The PipeWire Lib, An Inspiration From Wayland
PipeWire uses POD for encoding messages (methods and events) and some properties of the objects that live on the server. Meanwhile the SPA libraries are used to create objects in the server’s graph.
Objects that live in the graph are plugins created through the SPA
factories: objects that implement specific interfaces. Some of these
objects are always present — singletons such as the core and the
registry — while others appear dynamically — be it because they get
created by modules or by the session manager.
PipeWire, like PulseAudio, has a plugin architecture: the core is small
and everything else is a module, in this case SPA plugins. That makes
it very extensible and dynamic.
These objects created through SPA factories all have IDs. One object, the
core singleton, has a fixed ID of 0 so that clients can always find it.
The client can then “bind” — that’s what we call getting a proxy to
a remote object — to the registry singleton object to be able to list
all objects living server side (to then also bind to them if needed).
Binding is needed to be able to call methods on objects or register to
events they emit.
Some objects are related to the media processing, such as nodes, links,
and ports — yes, these are all separate entities in the graph. Others
are modules to extend the core, or factories to create other objects
(factories can live in the graph too), or management related objects
for the session manager.
PipeWire objects also have permissions attached to them (read, write,
execute, metadata) so that when clients attach to PipeWire they can only
access methods and states that are allowed (a module called
libpipewire-module-access
is responsible of that).
Think of the factories present in the PipeWire graph as the equivalent
of GstElement factories in GStreamer.
Some examples of interfaces that are implemented by objects created by
these factories:
- PipeWire:Interface:Core (
struct pw_core
) - PipeWire:Interface:Registry (
struct pw_registry
) - PipeWire:Interface:Module (
struct pw_module
) - PipeWire:Interface:Factory (
struct pw_factory
) - PipeWire:Interface:Client (
struct pw_client
) - PipeWire:Interface:Metadata (
struct pw_metadata
) - PipeWire:Interface:Device (
struct pw_device
) - PipeWire:Interface:Node (
struct pw_node
) - PipeWire:Interface:Port (
struct pw_port
) - PipeWire:Interface:Link (
struct pw_link
)
The PipeWire daemon offers an event loop where events are processed sequentially, as they arrive. Clients can interact with the remote objects living in the graph, calling methods and registering to receive events, the ones implementing the interfaces listed above, through proxies (bind) that allow calling the methods related to the interface that they follow.
If you know something about Wayland, this way of doing IPC would ring a
bell. You’d be right because PipeWire event loop, proxies, registry
mechanisms have all been inspired by Wayland asynchronous IPC design.
However, unlike Wayland, the interfaces are still simply in header files
and not defined in XML files, so you have to consult them like we did
in the previous SPA example.
So far this is the explanation we have:
- A daemon, what we call PipeWire service/daemon, implements a global graph (there’s also the idea that part of the graph can run in clients but I haven’t seen it in the wild yet, the concept is interesting)
- Clients operate on this graph
- Media processing elements, and others, live in the graph
- The elements are created by SPA plugins, have specific IDs, and permissions
- Clients bind (create a proxy) to these elements to be able to call methods and register for events (hooks) that the elements implement
- Message sent from client to server is a “method”
- Message from server to client is an “event”
- The method data, event data, and some of the elements properties are encoded using POD.
As a client, this concretely takes the form of an initial connection to
the PipeWire server, which gives back a proxy to the core object (ID=0),
and then a continual event loop to poll for new activities.
I’ve written a simple example
here
but let’s go over how the usual flow is done in five steps:
- Create a
struct pw_loop
, there are multiple kinds of loop depending on the criteria. The gist is that it’s an abstraction over poll(2). Each have specific ways to run and stop, control points. (somewhat similar to PulseAudio loops concept)
The available ones are:struct pw_main_loop
, defined in/pipewire/main-loop.h
struct pw_thread_loop
, defined in/pipewire/thread-loop.h
struct pw_data_loop
, defined in/pipewire/data-loop.h
-
After getting a loop we pass it to
pw_context_new
(casting it tostruct pw_loop
) to create a context object,struct pw_context
, which manages the environment related resources. -
Then we connect to the PipeWire server using
pw_context_connect
which will return a proxy to the singleton core object, astruct pw_core
. -
After getting our initial connection to the core we can start running our loop using the method specific to the type of loop we got. If we are using a
struct pw_main_loop
then we can callpw_main_loop_run
.
This particular loop only stops whenpw_main_loop_quit
is called, so it’s a good idea to register for some interesting events before starting this loop or to call methods on the core proxy. The usual method to call first ispw_core_get_registry
to fetch a proxy to the singleton registry object. - Finally, when the loop ends, we have to cleanup object, destroy the proxies we’ve created, disconnect from the core, destroy the context, and destroy the loop.
As you can notice, step 4 is the one that will have the actual logic into
it. This is when we setup things before our loop and will dynamically
handle events.
Like we said, normally we bind to get a struct pw_registry
proxy and
register for the “global” event, which will emit an event for each object
that exists in the graph. In the event callback hook to the “global”
event, we can choose to bind to any of the object we’re notified about,
to do more things: inspect elements properties, call methods on them,
or register for events that they can emit.
Additionally, apart from the registry object, the core object itself got interesting events to get notified of new clients binding, object creation, errors, etc.. A useful event is the “done” event that is emitted after the core handles the “sync” method (along with the same sequence number set). It might sound benign, but because the core processes everything sequentially, that means that anything sent before the “sync” will be finished. It’s a great way to know asynchronously that what was previously sent has reached the server and is indeed “done”.
That’s about it for the PipeWire library. The useful things that we can
do on objects living in the graph depend on which object we are
talking about and which interface it implements.
Some of the properties on these objects only make sense to the
session manager, others to the node itself, and others to the core.
For example, the node object has properties and params,
all having different meanings and fetched at different times through
different events that we can register to. The volume(s) is part of the
params of a struct pw_node
.
Again, there are a couple of examples in the official docs, and one I left here.
Configuration
Not everyone will write clients, but a lot of people are interested in
configuring the PipeWire server and the session manager.
There are three or four blocks in the PipeWire equation that can be
configured:
- The PipeWire daemon running the core and hosting the processing graph
- The clients which get their features automatically set according to the conf
- The session manager to add nodes to the graph, discover, and set them up appropriately
- The PipeWire PulseAudio server and JACK backward compatibility layer and their configurations
The PipeWire daemon configuration, the clients configuration, and the
pipewire-media-session, pipewire-pulse, and WirePlumber configuration
all have the same format. While the manpage pipewire.conf(5)
explains
it briefly, it can still be confusing so let’s try to make sense of it.
The format is a series of assignment in the form of name = value
. The
value can be either another simple assignment, a dictionary { key1=val1
key2=val2 }
, an array [ val1 val2 ]
, or a composite such as an array of
dictionaries. As you might have noticed, there’s no comma in this format.
The initial list of names present and their meaning depends on what we are configuring. However, as I found out, most of them got some of the following:
context.properties
A dictionary containing generic properties.context.spa-libs
A dictionary that tells the program where to find the.so
when matching a SPA factories names.context.modules
An array of dictionaries containing modules to load on startup.context.objects
An array of dictionaries containing objects that will be created automatically using an SPA factory.context.exec
An array of dictionaries (not available for client configuration) with additional commands to execute sequentially after launch.
PipeWire Server Configuration
The PipeWire daemon configuration is available in multiple
places, either globally in /etc/pipewire/pipewire.conf
and
/usr/share/pipewire/pipewire.conf
, or locally in the $XDG_CONFIG_HOME
,
usually .config/pipewire/pipewire.conf
.
So be sure to copy it to the local user directory before doing
modifications, as the global configuration might change rapidly with
the current development speed.
Before that, it’s good to know PipeWire respects a couple of environment
variables including PIPEWIRE_DEBUG
which takes a level of verbosity
for debugging between 1 and 5 (5 is the most verbose) and could also
have an optional category next to it to filter what is being logged,
PIPEWIRE_LOG
and PIPEWIRE_LOG_SYSTEMD
to log in a specific file and
to disable or enable systemd logs respectively, PIPEWIRE_LATENCY
to configure the default latency (buffer-size/sample-rate
or samples in buffer/samplerate
, see the previous
article
to know what that means).
pipewire.conf
contains all the configuration names we’ve mentioned
before. In particular, its context.properties
has information about
the default aspect of the processing graph.
Let’s go over each section and the particularities of each one.
context.properties
contains generic aspects of the core: logging level,
scheduling settings, default global sample rate, default quantum min and
max value (buffer), and more. The data in the pipeline will default to
this sample rate and each node will negotiate automatically their own
latency accordingly (buffer size within the quantum set in the config),
and when finally reaching a device the signal will be converted to its
sample rate.
Nodes can set a desired buffer size themselves by setting the property
node.latency
.
context.spa-libs
contains, as we said before, a dictionary
mapping factory names as regex to the library location on disk (.so
files). Remember how we did the dlopen(3)
in the previous section
and looked for the available factories. Here it’s useful to quickly
find how to create elements using the factory in the right library,
either directly from the configuration in context.objects
, through
the command line tools, or others.
For example, we notice support.* = support/libspa-support
, which
says any call to use a factory that starts with support.
will use the
library in /usr/lib/spa-0.2/support/libspa-support.so
.
context.modules
contains an array of modules loaded sequentially
as a series of dictionaries that has at least a name
and optionally
args
and flags
. The name
is used to point which library will be
loaded from the system, usually in /usr/lib/pipewire-<version>/
. The
args
is a dictionary containing specific per-module settings, and the
flags
currently is an array that can have two values in it: ifexists
,
to only load the module if the library is on disk, and nofail
, to not
stop loading other modules or crash if there was an error.
There are quite a lot of modules available, each doing different things,
some extending the core functionality, others providing ways to create
filters, some changing the scheduling, some adding profiling, some doing
access control, some providing the adapter around nodes for resampling,
some providing factories, and much more.
You can usually find the description and usage of the already loaded
modules, though very sparse, by doing pw-cli dump Module
and checking
the module.description
and module.usage
properties. I’m currently not
aware of a way to dump the possible arguments of a module without first
loading it, similar to what spa-inspect
does. You can always consult the
source
and find the PW_KEY_MODULE_USAGE
, like here for
module-filter-chain
.
NB: Technically, modules are dynamic clients that have an exported
function with the signature: SPA_EXPORT int pipewire__module_init(struct
pw_impl_module *module, const char *args)
.
This interesting plugin, the libpipewire-module-filter-chain
, described
here
allows to create a sub-graph of LADSPA and built-in plugins (SPA), then
later exposing the sources and sinks of that sub-graph to the global
PipeWire graph (Is this an instance of part of the graph running in
a client, I’m not sure). This makes inserting filtering easier than
to rely on separate software but you need to be familiar with how to
configure LADSPA library controls (which I’m not).
context.objects
, contains a list of objects that will be automatically
created when PipeWire starts. It takes at least the factory
which is the
SPA factory name, and optionally the args
passed to it as a dictionary,
and flags
which can be set to nofail
to ignore errors when creating
the object.
This section of the configuration is useful to autocreate virtual nodes
in the graph or to manual set devices.
For example:
{
factory = spa-node-factory
args = {
factory.name = videotestsrc
node.name = videotestsrc
Spa:Pod:Object:Param:Props:patternType = 1
}
}
NB: Be sure to uncomment the SPA factory videotestsrc
for this
example.
This creates a sample video with a fixed pattern,
similar to GStreamer videotestsrc
. There’s a page in the
Wiki
that goes over the concept of virtual devices, it explains some of the
usual properties that can be set on playback and capture.
NB: Even though it should be the same as the gst-launch-1.0
example
from earlier, using gst-launch-1.0 pipewiresrc client-name=hello
! videoconvert ! autovideosink
and connecting the videotestsrc
to hello
just displays an empty screen (yet connecting the webcam
directly does indeed display the screen properly). It does work
with audiotestsrc
though, using gst-launch-1.0 pipewiresrc
client-name=hello3 ! audioconvert ! autoaudiosink
.
Lastly, context.exec
contains an array of programs that will be
launched by PipeWire after startup. It takes at least a path
with
the program along with optional args
passed to it. I’m not actually
sure why this section is present when everywhere I look it seems like
it’s recommended to rely on the service manager instead to start other
software. I don’t think it should be the role of the PipeWire daemon to
manage other services. Yet, it’s there, so good to know about.
Let’s note that pipewire-pulse
is a copy of the pipewire
binary with
the only difference that it has a different name. PipeWire loads its
configuration based on its name, so it will load pipewire-pulse.conf
,
which activates the libpipewire-module-protocol-pulse
.
PipeWire Client Configuration
As with the daemon configuration, the client configuration is found in
the same directory under the names client.conf
and client-rt.conf
.
The difference between these two is that one loads the
libpipewire-module-rtkit
module and the other doesn’t.
PipeWire clients either load a specific config file or load client.conf
by default. They do this configuration step before joining the graph and
being connected to other nodes by the policies of the session manager.
These files contain the usual sections we’ve seen before but also has
two other sections fitler.properties
and stream.properties
which
configure how filters and streams should be handled internally. The
clients are the ones doing most of the sample conversion, mixing, and
resampling, so these sections are about how it will be done.
From buffer size, to sample quality, to channel mixing, etc..
A quick explanation of some of the properties is found in the official docs here.
PipeWire Session Manager Configuration
The session manager is the piece of software that is responsible for the
policy: to find and configure devices, attach them appropriately to the
graph, set and restore their properties if needed, route streams to the
right device, set their volume, and more.
It can create it’s own objects in the PipeWire graph related to session
management such as endpoints and links between them, a sort of abstraction
on top of PipeWire nodes.
There are currently two implementations of the session manager:
pipewire-media session and WirePlumber. Each have a different
policy format, pipewire-media-session having static matching
rules while WirePlumber providing dynamic lua scripts for policy.
Yet the global idea is the same: Finding and setting up devices
depending on where they come from (ALSA, JACK, Bluetooth), set their
profiles/ports/name/volume, add them to the graph, connect devices and
streams as they appear according to defined rules or restoration info,
also do the same for properties of streams.
Think of the session manager configuration as the rulebook for what happens in the graph.
pipewire-media-session Configuration
pipewire-media-session ships alongside the pipewire daemon, at least for
now with most package managers. Its configurations are found in the
same place as the client conf and the daemon conf but under a directory
named media-session.d
. In this directory we find the entry-point config
called media-session.conf
along with three other configurations for the
different device drivers that the software manages: alsa-monitor.conf
,
bluez-monitor.conf
, v4l2-monitor.conf
.
The configuration format in the media-session.conf
should look
familiar by now with all the usual section. The additional one here
is the session.modules
which is a dictionary of module categories
(bundles) that are enabled when specific files with the same name as the
key of the dictionary entry exists in the media-session.d
directory
(yes, it’s an interesting way to enable features). There is a default
key which is always enabled, as the name implies.
Each module activates a specific functionality, here’s a glimpse of the
ones listed when issuing pipewire-media-session --help
:
- flatpak : manage flatpak access
- portal : manage portal permissions
- metadata : export metadata API
- default-nodes : restore default nodes
- default-profile: restore default profiles
- default-routes : restore default route
- restore-stream : restore stream settings
- streams-follow-: move streams when default changes
- alsa-seq : alsa seq midi support
- alsa-monitor : alsa card udev detection
- v4l2 : video for linux udev detection
- libcamera : libcamera udev detection
- bluez5 : bluetooth support
- suspend-node : suspend inactive nodes
- policy-node : configure and link nodes
- pulse-bridge : accept pulseaudio clients
- logind : systemd-logind seat support
There isn’t anything else in the config file, however, some of the
modules give us the ability to have matching rules to set
additional properties on devices and clients, apart from the usual
client.conf
. These rules are stored and read in separate files such as:
alsa-monitor.conf
, bluez-monitor.conf
, and v4l2-monitor.conf
,
which should be consulted when a module with the same name is
set, for example alsa-monitor
module. (Though the last two are
not mentioned in the list above, but somehow hinted at here for
Bluetooth)
These files have two sections: properties
which are
properties that are always set on such device, and a
rules
section. For example, in the alsa-monitor.conf
the
properties
has values related to alsa device reservation, more on that
here.
The rules
configuration is an array of dictionaries, the dictionaries
containing two parts: matches
, a list of regex and matching
rules (AND/OR conditioning too), and action
, which currently only has an
update-props
entry to set properties or remove them (when setting to null) on the
device.
The properties that can be set are either generic node properties or dependent on the type of device we are handling. For example, ALSA got some properties listed in this doc. I’m currently not aware if there’s a way to list all of the properties that could possibly be set for a specific device driver, looking at the code they seemed to be defined and read in different places. This is something you should remember, that properties on nodes are read and interpreted differently by multiple software/modules/libraries handling them.
Another set of modules from the pipewire-media-session that are
interesting are the ones related to restoration rules. These are
similar to PulseAudio restoration mechanism, and actually reuses its
code, however it stores the rules in a JSON format in the home directory
(usually under ~/.config/pipewire/media-session.d/
). This saves you
from the binary format of PulseAudio, which I had to create a custom db
editor to be able to edit them.
The files are the following:
default-nodes
: Stores the default sink and source.default-profile
: Stores the default profiles for devices.default-routes
: List profiles and their settings/volumes for each device.restore-stream
: List of output and input stream names along with the last volume set on them. Additionally, it could have information such as thetarget-node
which should, in theory, re-attach the stream to the right node when it appears (though I had issues with this not being respected).
Apart from these we also have the streams-follow-default
configuration which will keep moving streams to whatever the default
device is, even when it changes. This can be extremely annoying
though, but my guess is that it’s somewhat a replacement for PulseAudio
concept
of module-always-sink
and module-always-source
which create a
virtual device that streams are moved to when all other devices are
disconnected. Regardless, this breaks the usual restoration rules flow
for devices and streams, and is unintuitive.
You can always consult these PulseAudio restoration flowcharts when
in doubts:
WirePlumber Configuration
WirePlumber is a more advanced session manager that is based on
Glib/GObject to wrap PipeWire types/functionalities and relies on lua
plugins for most of its extra logic.
That means, on one side that you can debug WirePlumber using
the usual Glib stuff, such as the environment variables such
as G_MESSAGES_DEBUG=all
, though it is deprecated in favor of
WIREPLUMBER_DEBUG
, and on another side that you can extend its
functionality to suit your need by adding lua plugins.
The project is still considered in early stage and changes
at a rapid pace. The documentation is slowly taking form
here.
Like pipewire-media-session, it also has mechanisms to set policies,
discover devices, and plugins to extend functionalities, however it does
it in its own way by creating wrappers over the native PipeWire library.
The configuration files are found either in the local user directory
~/.config/wireplumber
or globally in /etc/wireplumber
. The main
configuration file wireplumber.conf
follows the same format
as the ones we’ve previously seen and has as an additional
property the wireplumber.components
which, as far as I
understood, is currently used to bootstrap the lua scripting
engine through libwireplumber-module-lua-scripting
(found in
/usr/lib/wireplumber-<version>
).
Then, sub-directories with lua scripts will be loaded in alphabetical
order (bluetooth.lua.d/
, policy.lua.d/
, main.lua.d/
). The last
script in the list usually enables and activate whatever was set in the
previous ones.
For example, policy.lua.d
will pass through the following files,
in this order:
00-functions.lua
10-default-policy.lua
50-endpoints-config.lua
90-enable-all.lua
The last file 90-enable-all.lua
calls a function defined in
10-default-policy.lua
, default_policy.enable()
which loads modules
and scripts from /usr/share/wireplumber/scripts/
to set the policies
in place. This is a bit confusing, partly because it’s not well
documented, and partly because the flow is not clear. Yet, the gist
is that it’s loading the policy set in these lua files similar to what
pipewire-media-session was doing.
A better example would be to look at
/etc/wireplumber/main.lua.d/50-alsa-config.lua
which is the equivalent
of alsa-monitor.conf
. It sets properties and rules in the alsa_monitor
object that is created in the file 30-alsa-monitor.lua
, and then calls a
function defined in that file alsa_monitor.enable()
in the final file
90-enable-all.lua
to load the monitor script and make it all fall
into place… Again, not so obvious but still makes sense when doing
the comparison with pipewire-media-session.
While lua is used for configuration, it can also be used for scripting
additional functionalities or side programs that rely on WirePlumber,
all running in their own sandboxed environment.
As we said, WirePlumber extends PipeWire objects,
mapping them to GObjects which it documents here in its C
API. It
also implements the Endpoint session related abstraction over objects
in the graph. As with all GObjects, you can call methods and register
for signals on them. Some are globally accessible when using the library
and provide a gateway to the WirePlumber daemon.
The Lua
API
is a map of the C API unto LUA, along with helpers found in
/usr/share/wireplumber/scripts/
. Methods are called with the
object:method
notation and the signals are registered using
object:connect("signal_name", function()...)
. Let’s show a couple of
examples of these global GObjects available through Lua.
The
Core
and
ObjectManager
are currently the most interesting to
take a look at, along with debugging log
functions.
The Core
provides a wrapper around WirePlumber core, which, as I could
see, is very useful to load modules via the
Core.require_api
and call methods on them. There is
currently no documentation for the available plugins and
which calls and signals they provide, but you can always check the
source
for *_api_class_init
functions, like here for the mixer
api
An example of this can be found here, which I reproduce under:
#!/usr/bin/wpexec
--
-- WirePlumber
--
-- Copyright © 2021 Collabora Ltd.
-- @author George Kiagiadakis <george.kiagiadakis@collabora.com>
--
-- SPDX-License-Identifier: MIT
--
-- Load the necessary wireplumber api modules
Core.require_api("default-nodes", "mixer", function(...)
local default_nodes, mixer = ...
-- configure volumes to be printed in the cubic scale
-- this is also what the pulseaudio API shows
mixer.scale = "cubic"
local id = default_nodes:call("get-default-node", "Audio/Sink")
local volume = mixer:call("get-volume", id)
-- dump everything
Debug.dump_table(volume)
-- or maybe just the volume...
-- print(volume.volume)
Core.quit()
end)
As you can notice, these scripts are run by passing it to the wpexec
command line tool.
It can also be used to create nodes on the fly:
local props = {
["media.class"] = "Audio/Sink",
["factory.name"] = "support.null-audio-sink",
["node.name"] = "ExampleNode",
["node.description"] = "ExampleNode",
["audio.position"] = "FL,FR",
}
node = Node("adapter", props)
node:activate(Features.ALL)
The ObjectManager
is somewhat the equivalent of the PipeWire registry
but made easy, letting you listen and filter nodes that are of interest to
interact with them, possibly modifying their properties.
The ObjectManger
takes a list of
Interest
,
which are filters.
obj_mgr = ObjectManager {
Interest {
type = "node",
Constraint { "media.class", "matches", "*/Sink" }
}
}
Then we can add a listener for the “installed” signal.
obj_mgr:connect("installed", function (om)
for obj in om:iterate() do
local id = obj["bound-id"]
local global_props = obj["global-properties"]
print("\n")
print("Obj ID: ".. id)
Debug.dump_table(global_props)
print("\n")
end
end)
Finally, we get the ObjectManager
running so that it fires the signal:
obj_mgr:activate()
This script should dump all the properties of the sinks in the PipeWire graph.
More examples can be found here.
With that let’s move to other tools and debugging.
Tools & Debugging
Native PipeWire Tools
PipeWire comes with a series of tools that can be used to do common tasks, interact with the server, and debug or profiling. If you are familiar with the set of tools that come with PulseAudio then you’ll find a similarity.
Here’s a list of some interesting ones:
pw-cat
: used to play audiopw-play
,pw-record
,pw-midirecord
,pw-midiplay
: symlink topw-cat
pw-record
: used to record audiopw-loopback
: create a dummy loopback nodepw-link
: A port and link manager, used to list ports, monitor them, and create linkspw-dump
: used to dump nodes in the graph or the whole graphpw-dot
: similar to pw-dump but dumps it in a graphviz formatpw-top
,pw-profiler
: used to monitor traffic efficiency between objects in the graphpw-mon
: used to monitor any events happening in the graphpw-metadata
: used to modify metadata in the graph, this is currently used for storing default nodes information.pw-cli
: generic command line to interface with PipeWire daemon, allowing dump, loading and unloading modules, listing objects, creating links and nodes, setting params, and more.
These tools, and in particular pw-cli
, can be used to create objects
and manage the graph on the fly or inspect things happening in the graph.
Here’s an example of create a link between two front-left ports of two nodes:
pw-cli create-link "TestSink" 'monitor_FL' \
"alsa_output.usb-C-Media_Electronics_Inc.device.iec958-stereo" \
'playback_FL'
Or dumping all available factories:
pw-cli dump Factory
Or creating a virtual node:
pw-cli create-node adapter { \
factory.name=support.null-audio-sink node.name=my-mic \
media.class=Audio/Duplex object.linger=1 \
audio.position=FL,FR }
Apart from the command line tools, there currently
exists a single native Rust-Gtk-based GUI under the name of
helvum. It is still
a WIP project, offering the basic functionality of connecting node by
clicking on ports of both sides.
Other than this, there is a lack of frontend for PipeWire, especially
those that would allow manipulating/representing properly a graph of
any type of media, both audio and video.
Here’s what helvum looks like:
(UPDATE: There’s now a new QT GUI called qpwgraph )
I’ve attempted myself to build something using Rust, however the rust
binding,
especially related to POD and SPA, is still a work-in-progress. It is
hard to bind to rust as the PipeWire library mostly only has static
inline
functions.
Yet, there exists fun tools such as PulseEffects that allow creating filter nodes to add audio effects on the fly.
On the other hand, when it comes to the session manager, the pipewire-media-session currently doesn’t have clients while WirePlumber does offer a bundle of command line tools.
We’ve already seen wpexec
to execute lua scripts. There’s also wpctl
which is another introspection tool to interrogate WirePlumber about
available devices, nodes, their properties, and their volume. It can be
used to interface with the notion of “endpoints” which is the abstract
representation of where media start and ends for the session manager.
wpctl status
can be used to list known endpoints, showing default
devices with a star character prepended. This default can be changed
using the set-default
option, passing the node ID. The other options
such as set-volume
, set-mute
, set-profile
are straight forward
to understand.
Tools Relying on PulseAudio
pipewire-pulse, which as we said is a wrapper to load the module
libpipewire-module-protocol-pulse
, allows to use most PulseAudio
features and software on top of PipeWire compatibility layer.
The module itself can be configured in
pipewire-pulse.conf
with certain options, as shown
here. It even covers the modules for network protocol support.
It thus let us use any user interface that we got accustomed to use with
PulseAudio such as pavucontrol, pulsemixer, pamixer, and more. This
also includes command line tools such as pactl
which then gives us
yet another way to interface with PipeWire instead of pw-cli
.
$ pactl info
Server String: /run/user/1000/pulse/native
Library Protocol Version: 34
Server Protocol Version: 35
Is Local: yes
Client Index: 106
Tile Size: 65472
User Name: vnm
Host Name: identity
Server Name: PulseAudio (on PipeWire 0.3.30)
Server Version: 14.0.0
Default Sample Specification: float32le 2ch 48000Hz
Default Channel Map: front-left,front-right
Default Sink: alsa_output.pci-0000_00_14.2.analog-stereo
Default Source: alsa_input.pci-0000_00_14.2.analog-stereo
Cookie: daad:fd8f
While pactl
is supposed to be used with PulseAudio, the functionality
is somewhat mapped to PipeWire. This includes the loading of modules,
which can be used to create nodes in the graph.
pactl load-module module-null-sink object.linger=1 \
media.class=Audio/Sink \
sink_name=my-sink \
channel_map=surround-51
As you can notice, the syntax of the module-null-sink
isn’t
anything like the syntax you’d use for PulseAudio. This
same module can also be used to create duplex and
source nodes. More on the creation of virtual devices
here.
The mapping even works with listing modules through pactl list modules
,
but lists PipeWire modules instead. Obviously, we can do listing of
objects, or specific ones such as sinks with pactl list sinks
.
There’s this useful guide that shows features mapped between PulseAudio and PipeWire, including the pactl we showed and certain configuration specific to PulseAudio.
Here’s a screenshot of pulsemixer running on top of PipeWire:
Tools Relying on Jack
As with PulseAudio, PipeWire offers a compatibility layer with JACK. It
is initiated by prepending JACK commands with pw-jack
, for example:
pw-jack jack-plumbing
pw-jack qjackctl
NB: you can find jack-plumbing
here,
a very nice tool to create jack rules connection rules.
pw-jack
is a shell script that sets some environment variables and
modifies the LD_LIBRARY_PATH
to point to PipeWire’s jack lib before
the global one. You can notice this by doing the following:
pw-jack ldd /usr/bin/qjackctl| grep -i libjack
libjack.so.0 => /usr/lib/pipewire-0.3/jack/libjack.so.0 (0x00007f65c9fa0000)
Similarly to PulseAudio shim, this allows us to use most JACK based
GUI tools such as qjackctl, carla, catia, and all the beautiful
visualizers. The advantage over PulseAudio tools is that JACK tools
already display a graph by default which maps well to PipeWire concepts,
however it only maps audio devices and not video.
PipeWire also allows modifying the graph on the fly, the effect being
instantaneous, unlike JACK, which is nifty.
The documentation also
lists
specific configurations related to JACK, usually found in jack.conf
.
Here’s a screenshot of qjackctl running on top of PipeWire:
Tools Relying on GStreamer
Lastly, as we’ve mentioned before, we can rely on any program that uses
gstreamer and the gstreamer command line tools to play with the media.
Again, through the plugins:
pipewiresrc
: PipeWire sourcepipewiresink
: PipeWire sinkpipewiredeviceprovider
: PipeWire Device Provider
We can create a PipeWire sink that will provide audio from the internet:
gst-launch-1.0 \
uridecodebin 'uri=http://podcast.nixers.net/feed/download.php?filename=nixers-podcast-2020-07-221.mp3' ! \
pipewiresink mode=provide \
stream-properties="props,media.class=Audio/Source,node.description=podcast"
And then link the ports to an audio device, either via the command line tools or GUI such as helvum.
pw-link gst-launch-1.0:capture_1 \
alsa_output.usb-C-Media_Electronics_Inc._Microsoft_LifeChat_LX-3000-00.iec958-stereo:playback_FL
pw-link gst-launch-1.0:capture_2 \
alsa_output.usb-C-Media_Electronics_Inc._Microsoft_LifeChat_LX-3000-00.iec958-stereo:playback_FR
In theory, that should allow us to manipulate videos too, however, as
I’ve mentioned above I got some issues with it.
I’ve tested the following and tried to open it with cheese
webcam app
but the video only showed for 2s and stopped:
gst-launch-1.0 \
uridecodebin uri=https://venam.net/workflow-compil-venam-2020.webm ! \
pipewiresink mode=provide \
stream-properties="props,media.class=Video/Source,node.description=compilation"
Still the possibility is fascinating: to easily be able to modify and add filters to videos (including cameras) on the go.
Conclusion
This review should help kick-start people on the PipeWire journey, give the key to unlock more knowledge around it — to get started.
We reviewed the basic ideas behind PipeWire, the concept of the graph,
“mechanism not policy”, we also saw the relation with Gstreamer.
We had a look at the building blocks such as the POD format and the SPA
to build factories of objects following an interface. We’ve also glimpsed
at the PipeWire library loop and it’s inspiration from Wayland with its
singleton objects such as the registry.
Afterwards, we explained the configuration format, what can be configured,
for the client, PipeWire server, and the session manager, be it the
default pipewire-media-session or WirePlumber with its flexible lua
scripting.
Then we’ve had a quick overview of the tooling around PipeWire, from
native, to the ones relying on compatibility layers with PulseAudio
and JACK.
Yet, this is only the tip of the iceberg, there’s a lot of things that weren’t covered such as how to compile PipeWire from scratch, how to create audio streams/filters and manage their buffers, diving into the metadata and how its used by different pieces, how the session manager can use the concept of Endpoints, the policy mechanism and the link with desktop portals, etc..
This post is descriptive but the real story happens as we speak, the
development is quick and the discussion is lively on IRC (currently on
irc.oftc.net
in the #pipewire
channel). The developers are nice fellows
that are very active in the discussion.
This post will probably age badly because the tech is relatively
new, but there’s a need for such blog post to get acquainted with the
project through new eyes.
I’ve only touched a small part of PipeWire but that should get anyone
started, at least I would’ve personally loved to have such content when I
first heard of PipeWire.
Let me know if it helped!
References
https://pipewire.org/
https://gstreamer.freedesktop.org/
https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2016/Wim%20Taymans%20-%20Simple%20Plugin%20API%20(SPA).pdf
https://wiki.gentoo.org/wiki/PipeWire
https://lwn.net/Articles/847412/
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-ALSA
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/SPA
https://www.youtube.com/watch?v=1w6yVqU0lkU
https://www.youtube.com/watch?v=yjh5Eg3Efjg
https://gitlab.freedesktop.org/ryuukyu/helvum
https://wiki.archlinux.org/title/PipeWire
https://docs.pipewire.org/page_pipewire.html
https://en.wikipedia.org/wiki/Passive_data_structure
https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/master/doc/spa-pod.dox
https://pipewire.github.io/pipewire/page_remote_api.html
https://www.collabora.com/news-and-blog/blog/2020/03/05/pipewire-the-media-service-transforming-the-linux-multimedia-landscape/
https://wiki.debian.org/PipeWire
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/home
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Access-control
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/FAQ
https://docs.pipewire.org/page_spa.html
https://docs.pipewire.org/modules.html
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-PipeWire
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Filter-Chain
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Virtual-Devices
https://www.guyrutenberg.com/2021/03/11/replacing-pulseaudio-with-pipewire/
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Config-pipewire-media-session
http://old-docs.automotivelinux.org/docs/en/halibut/apis_services/reference/audio/audio/pipewire.html
https://pipewire.pages.freedesktop.org/wireplumber/daemon-configuration.html
https://docs.pipewire.org/page_registry.html
https://github.com/PipeWire/pipewire/blob/master/doc/tutorial2.md
https://github.com/PipeWire/pipewire/blob/master/src/tools/pw-dump.c
https://pipewire.pages.freedesktop.org/pipewire-rs/pipewire/index.html
https://gitlab.freedesktop.org/pipewire/pipewire-rs/-/blob/main/pipewire/examples/
https://pipewire.pages.freedesktop.org/pipewire-rs/
https://www.antixforum.com/forums/topic/pipewire-to-manage-audio-in-antix-21/
https://blog.linuxplumbersconf.org/2009/slides/Paul-Davis-lpc2009.pdf
If you want to have a more in depth discussion I'm always available by email or irc.
We can discuss and argue about what you like and dislike, about new ideas to consider, opinions, etc..
If you don't feel like "having a discussion" or are intimidated by emails
then you can simply say something small in the comment sections below
and/or share it with your friends.