For those of you who were interested but couldn’t make it to the GStreamer Conference this year, the cool folks at Ubicast have got the talk videos up (can be streamed or downloaded).
Among these is my talk about recent developments in the PulseAudio world.
Longish day, but I did want to post something fun before going to sleep — I just pushed out patches to hook up the WebRTC folks’ analog gain control to PulseAudio. So your mic will automatically adjust the input level based on how loud you’re speaking. It’s quite quick to adapt if you’re too loud, but a bit slow when the input signal is too soft. This isn’t bad, since the former is a much bigger problem than the latter.
Also, we’ve switched to the WebRTC canceller as the default canceller (you can still choose the Speex canceller manually, though). Overall, the quality is pretty good. I’d do a demo, but it’s effectively had zero learning time in my tests, so we’re not too far from a stage where this is a feature that, if we’re doing it right you won’t notice it exists.
There lot’s of things, big and small that need to be added and tweaked, but this does go some way towards bringing a hassle-free VoIP experience on Linux closer to reality. Once again, kudos to the folks at Google for the great work and for opening up this code. Also a shout-out to fellow Collaboran Sjoerd Simons for bouncing ideas and giving me those much-needed respites from talking to myself. :)
As I’d blogged about last week, we had a couple of Audio BoF sessions last week. Here’s a summary of what was discussed. I’ve collected items in relevance order rather than chronological order to make for easier reading. I think I have everything covered, I’ll update this post if one of the attendees points out something I missed or got wrong.
Surround: There were a number of topics that came up with regards to multichannel/surround support:
There seems to be some lack of uniformity of channel maps, particularly in non-HDA hardware. While it was agreed that this should be fixed, we need some directed testing and bug reports to actually be able to fix this.
Multichannel mixers, while theoretically supported, are not actually exposed by any current drivers. It was suggested that these could be exposed without breaking existing applications by having new MC mixers exposed with device names corresponding to the surround PCM device (like “surround51”).
We need a way to query channel maps on a given PCM. This will be implemented as a new ALSA API which could be called after the PCM is opened. (AI: Takashi)
It would be good to have a way to configure the channel map as well (if supported by the hardware?). The suggestion was to do this as was done in PulseAudio, where an arbitrary channel map could be specified. (NB: is there hardware that supports multiple/arbitrary channel maps? If yes, how do we handle this?)
Routing: Unsurprisingly, we came back to the problem of building a simplified mixer graph for PulseAudio.
The current status is that ALSA builds a simplified mixer for use by userspace, and PulseAudio further simplifies this by means of some name-based guessing.
PulseAudio would ideally like a simplified version of the original mixer graph, but something more complete than what we get now
However, since PulseAudio has fairly unique requirements of what information it wants, it probably makes sense to have ALSA provide the entire graph and leave the simplification task to PulseAudio (discussion on this approach continues)
There was no consensus on who would do this or how this should be done (creating a new serialisation format, exposing what HDA provides, adding node metadata to ALSA mixer controls, or something else altogether)
As an interim step, it was agreed that it would be possible to provide ordering in the simplified ALSA mixer (that is, add metadata to the control to signal what control comes “before” it and what comes “after” it). This should go some way in making the PA mixer simplification logic simpler and more robust. (AI: Takashi)
HDMI: A couple of things came up in discussion about the status of HDMI.
There was a question about the reliability of ELD information as this will be used more in future versions of PulseAudio. There did not appear to be conclusive evidence in either direction, so we will probably assume that it is reliable and deal with reliability problems as they arise.
It was mentioned that it might be desirable to only expose the ALSA device if a receiver is plugged in. This had been mooted earlier as something to do in PulseAudio as an alternative. One thing to consider with this approach is dealing with a device switch on the receiver side. Doing this without a notification to userspace was generally agreed to be a bad idea.
Jack detection: The long-standing debate on exposing jacks as input devices or ALSA controls came to a conclusion, with the resolution being that jacks would be exposed as ALSA controls. This requires a change in the kernel (and potentially alsa-lib?) which should not be too complex. Actual buttons (such as volume/mute control) will continue to be input devices. Once this is done, the pending jack detection patches will be adapted to use the new interface. (AI: Takashi (patches are in a branch already!), David)
UCM: Another long-standing issue was the merging of the ALSA UCM patches into PulseAudio. Most of the problems thus far had been due to an incomplete understanding of how ALSA and PA concepts mapped to each other. Some consensus was arrived at in this regard after a lengthy discussion:
As is the case now, every ALSA PCM maps to a PA sink
Each UCM verb maps to a PA card profile
Each combination of UCM devices that can be used concurrently maps to a PA port
Each UCM modifier is mapped to an intended role on the corresponding sink
The code should (as is in the patches currently submitted) be part of the PA ALSA module, and there will be changes required to use the UCM-specified mixer list instead of PA’s guessing mechanism. (AI: ???)
(NB: It was mentioned that PulseAudio needs to support multiple intended roles for a sink/source. This is actually already supported — the intended roles property is a whitespace-separated list of roles)
(NB2: There was further discussion with the Linaro folks this week about the UCM bits, and there’s likely going to be an IRC/phone gathering to clarify things further in the near future)
GStreamer latency settings: We currently do not actually use PulseAudio’s power saving features from GStreamer for several reasons. Suggestions to over come this were mooted. While no definite agreement was reached, one suggestion was to add a “powersave” profile to pulsesink that chose higher latency/buffer-time values. Players would need to set this if they are not using features that break when these values are raised.
Corking: The statelessness of current the corking mechanism was discussed in one session, and between the PulseAudio developers later. The problem is that we need to be able to track cork/uncork reasons more closely. This would give us more metadata that is needed to make policy decisions without breaking streams. Particularly, for example, if PA corks a music stream due to an incoming call, then the user plays, then pauses music, and then the call ends, we must not uncork the music stream. We intend to deal with this with two changes:
We need to add a per-cause cork/uncork request count
We need to associate a “generation” with cork/uncork requests, so certain conditions (such as user intervention) can bump the generation counter, and uncork requests corresponding to old cork requests will be ignored
This will make it possible to track the various bits of state we need to do the right thing for cases like the one mentioned before.
So that’s that — lots of things discussed, lots of things to do! Thanks to everyone who came for participating.
For those who are in Prague for GstConf, LinuxCon, ELCE, etc. — don’t forget we’ve a couple of interesting audio-related things happening:
If you’re around and interested, do drop in!
Yep, if we keep this up, it could even become a habit!
PulseAudio 1.1 is out. It’s mostly a bunch of bug fixes on top of 1.0. Most important of these are fixes for: a libpulse dependency on libsamplerate (if enabled) which would make our LGPL license invalid, broken Skype audio capture (because we changed from a 3 number version to 2 numbers), broken startup without a DBus session bus running, and not going crazy on USB disconnects.
This should be a very safe upgrade, so grab it while it’s hot!
I’ve just pushed a bunch of patches by Pierre-Louis Bossart that can have a pretty decent CPU/power impact. These introduce the concept of an “alternate sample rate”.
Currently, PulseAudio runs all your devices at a default sample rate, which is set to 44.1 kHz on most systems (this can be configured). All streams running at different sample rates are resampled to this sample rate. Pierre’s patches add an alternate sample rate that we try to switch to under certain circumstances if it means that we can save on resampling cost. This would happen if the stream uses exactly the alternate sample rate, or some integral-or-so multiple of it.
The default value for the alternate sample rate is 48 kHz. So if you’re playing a movie off a DVD where the audio track is typically a 48 kHz stream, and your card supports it, we switch to 48 kHz and avoid resampling altogether. Similarly, while making voice calls, common sample rates are 8, 16, and 32 kHz. These can be resampled to 48 kHz much faster than to 44.1 kHz.
Now for the big caveat — this won’t work if there’s any other stream connected to your sink/source. So if your music player is playing (or even paused) when you get that voip call, we can’t update the rate. This situation can probably be improved by at least allowing corked streams have their sample rate change (so having some random stream connected but not playing — I’m looking at you, Flash! — won’t block rate updates altogether). Hopefully we’ll get this fixed before this feature is released in PulseAudio 2.0.
Thanks to Pierre for all his work on this, and to my company, Collabora, for giving me some time for upstream work!
Prague is an interesting place to be at this time of the year — next week it’s playing host to the Real Time Linux Workshop. The week after that, we have the Kernel Summit, GStreamer Conference, Embedded Linux Conference Europe and LinuxCon Europe. I’m going to be at the last 3, and there’s some great audio stuff happening!
On the evening of Oct 23rd, we’re having an Audio BoF to discuss pending issues that cut across the stack — ALSA, PulseAudio, GStreamer and any other similar bits that people have complaints about.
Then there’s GstConf, where there are going to be a bunch of awesome talks. Also included is a talk by yours truly about recent developments in the PulseAudio world.
At some point during that week, possibly Oct 26th morning, plans are afoot to have an ALSA BoF to discuss the state and future of the HDA driver.
There are also rumours of excellent beer that will need to be scrupulously verified. ;)
It’s going to be a pretty exciting week!
As Colin Guthrie reports, PulseAudio 1.0 is now out the door! There’s a lot of new things in the release, and we should be getting a much more regular release schedule going. Head over to the full release notes for more details.
A lot of people have contributed to this release and thanks to them all. Special props to Colin all the patch-herding, tireless help, and code ninjutsu!
p.s.: Gentoo packages are already available, of course. :)
In an unsurprising turn of events, Adobe completely fails to play well with modern Linux systems. Well done, guys. Well done, indeed.
p.s.: I was quite happy to see that the Google Talk plugin has proper PulseAudio support (thanks to the WebRTC née GIPS code, it looks like).
I’m going to be at the Linux Plumbers’ Conference next week, speaking about the things we’ve been doing to make passthrough audio on Linux kick ass.
If you’re around and interested, do drop by!