Some of you might’ve noticed that there has been a bunch of work happening here at Collabora on making cool open source technologies such as GStreamer, Telepathy, Wayland and of course, PulseAudio available on Android.
Since my last blog post on this subject, I got some time to start looking at replacing AudioFlinger (recap: that’s Android’s native audio subsystem) with PulseAudio (recap: that’s the awesome Linux audio subsystem). This work can be broken up into 3 parts: playback, capture, and policy. The roles of playback and capture are obvious. For those who aren’t aware of system internals, the policy bits take care of audio routing, volumes, and other such things. For example, audio should play out of your headphones if they’re plugged in, off Bluetooth if you’ve got a headset paired, or the speakers if nothing’s plugged in. Also, depending on the device, the output volume might change based on the current output path.
I started by looking at solving the playback problem first. I’ve got the first 80% of this done (as we all know, the second 80% takes at least as long ;) ). This is done by replacing the native AudioTrack playback API with a simple wrapper that translates into the libpulse PulseAudio client API. There’s bits of the API that seem to be rarely used(loops and markers, primarily), and I’ve not gotten around to those yet. Basic playback works quite well, and here’s a video showing this. (Note: this and the next video will be served with yummy HTML5 goodness if you enabled the YouTube HTML5 beta).
(if the video doesn’t appear, you can watch it on YouTube)
Users of PulseAudio might have spotted that this now frees us up to do some fairly nifty things. One such thing is getting remote playback for free. For a long time now, there has been support for streaming audio between devices running PulseAudio. I wrote up a quick app to show this working on the Galaxy Nexus as well. Again, seeing this working is a lot more impressive than me describing it here, so here’s another video:
(if the video doesn’t appear, you can watch it on YouTube)
This is all clearly work in progress, but you can find the code for the AudioTrack wrapper as a patch for now. This will be a properly integrated tree that you can just pull and easily integrate into your Android build when it’s done. The PA Output Switcher app code is also available in a git repository.
I’m hoping to be able to continue hacking on the capture and policy bits. The latter, especially, promises to be involved, since there isn’t always a 1:1 mapping between AudioFlinger and PulseAudio concepts. Nothing insurmountable, though. :) Watch this space for more updates as I wade through the next bit.
Robert Swain
April 30, 2012 — 4:30 pm
Very shiny stuff. Wouldn’t it be great if that kind of responsive music streaming were available in lots of devices and automagically discovered in some list in the app so you could easily upgrade from your phone / headphones to an AV receiver with decent speakers or wherever else.
Of course, it’s nice to have pulseaudio running and capable of doing all the Androidy stuff anyway. Nice work!
Robert Piasek
April 30, 2012 — 7:46 pm
Very impressive! Keep up the good work.
liam
May 1, 2012 — 6:25 am
Hi Arun,
Amazing work! Does this version of PA have the orc/arm optimizations? Have you done any latency tests yet? From your previous post, that was the single biggest revelation to me. Also, did you have to bring over rtkit or is that not needed until you work on the policy section?
Best/Liam
Arun
May 1, 2012 — 10:44 am
Hey Liam,
This vesion of PA does have the ARMv6 optimisations, but not Orc. In some experimentation, I found that the Orc work I did (which was for volumes) was actually less efficient than the hand-rolled armv6 code. This is because ARM offers a 32bit x 16bit multiplication primitive directly, which is what the entire x86 optimisation revolved around. There are still potential improvements to be made here, and s bit of hacking on Orc to have a generic 32×16 mul instruction would do the trick. Not been able to get to this, unfortunately.
I didn’t have to bring over rtkit stuff yet. The PA side of things automatically gets RT prio via the old sched_setscheduler() mechanism. Android apps would be using the standard Android APIs to set their priority, anyway, so we might not even need rtkit.
As for latency tests — my first focus has been to get the functionality going, so I haven’t done any stringent latency tests. There is a noticeable difference in latency in the dialler app between the stock image and my code, but this is hard to capture in audio/video.
That said, here’s a very basic test that I ran: http://soundcloud.com/arunarunarun/sets/af-pa-comparison — I intend to do some testing and reporting with a more realisitc test case (like an app that generates sounds) before too long.
Gutz
May 1, 2012 — 9:25 am
Hi Arun ,
Great progress so far! You mentioned the ability to control the policy for audio routing/volume , Does this involve in-depended application routing/volume control also ( Like in Vista/Win7/8 ) ?
I’ll keep watching your blog for updates!
Arun
May 1, 2012 — 10:45 am
Hiya, this is something we’ve had in PulseAudio for a while now. Typically, though, this is not something that is used very often on phones and tablets, to keep the UI simple. So while it’s unlikely that you’ll be able to use this feature in a stock image, there would not be anything stopping someone from implementing this if they so desired.
Alexy
May 1, 2012 — 4:24 pm
Good stuff Arun. Keep the good work going
Craig
May 1, 2012 — 6:22 pm
Hi Arun,
This work is looking great! While I’m impressed with the ability to stream the audio from the handset via PA. Will the same be possible in the opposite direction? It would be great if I could carry the phone around, with the possibility of it outputting audio from the PC (more for having a portable jack to plug headphones in).
Arun
May 2, 2012 — 9:27 am
Yep, since you’re running a PulseAudio server on both sides, anything you can do in one direction can almost certainly be done in the other direction.
There might be issues to resolve such as preventing WiFi from sleeping on the phone when a client might connect, but these should be solvable.
Freddie
July 4, 2013 — 12:20 am
That’s something that I’ve been wondering about … basically turning the phone into a wireless audio receiver for the XBMC box. I got used to using Bluetooth headphones for late-night movie watching, but they’ve since died and I can’t afford (or find any worth buying) new ones right now. Would be neat to be able to use my Android phone as a wireless audio receiver and be able to listen to the audio from the XBMC box via headphones on the Android device. :)
Basically, replace Bluetooth with PA-over-WiFi.
Unless someone knows of another way to do this with existing Android/Linux apps …
Pedro Alves
May 2, 2012 — 8:07 pm
Android also uses the Linux kernel, and PulseAudio is not a kernel subsystem.
You meant:
“replacing AudioFlinger (recap: that’s Android’s native audio subsystem) with PulseAudio (recap: that’s the awesome GNU/Linux audio subsystem)”
Jonathan Abbey
May 3, 2012 — 2:05 am
Really? FSF is doing PulseAudio now?
Pedro Alves
May 4, 2012 — 3:22 pm
That’s a common misunderstanding.
GNU/Linux means GNU-based userspace, with a Linux kernel; the GNU part being glibc, and all, where everything else in your distro’s userspace sits uppon. The advent of Android should make clear that just calling regular distro’s “Linux” is wrong. Android is also Linux, but the userspace is very different.
I strongly recoment reading this, for example, before having such reactions:
http://www.informit.com/articles/article.aspx?p=1339466&seqNum=6
Pedro Alves
May 4, 2012 — 3:31 pm
Or start from page 0:
http://www.informit.com/articles/article.aspx?p=1339466
Arun
May 4, 2012 — 3:32 pm
I know what I meant, and my meaning is not unclear to anybody. Please keep the useless pedantary off my blog.
Ian Meredith
May 3, 2012 — 8:23 pm
Hi Arun, this looks amazing, the decrease in audio latency is very significant. Sorry for some more amateur questions but anyway: As a beginner app dev at what stage will I be able to utilize PA in one of my applications? Eventually you will be looking to get PA into a release build of android is that right? Its just I’m so impressed by the results I’d love to implement PA as soon as I’m able to!
Arun
May 4, 2012 — 9:25 pm
Hey Ian. For getting this out in usable shape, there’s still at least 2-3 months of work left to get a alpha-level build out. At some point in between, I’ll take a detour to write up some NDK support, so you could build PA apps using the NDK. Initially, this will only be useful to talk to PA servers on other machines, but eventually, hopefully we can get some traction on getting PA on more Android devices.
Yong Chang
May 4, 2012 — 6:50 am
This is great!! I’ve been interestingly keep watching your blogs about this work since I’m trying to implement Bluetooth A2DP sink function in an Android device. Is it possible to make an Android device as an A2DP receiver (like headset) using Pulseaudio? I’ve tested it in Ubuntu, and your work makes me confident that it can be done in Android.
Best, Yong
Arun
May 4, 2012 — 9:26 pm
Yi Yong, Thanks! :)
Once all the basic ALSA bits are done, I would like to take a look at using the built-in copy of BlueZ that Android runs talking to PA. Once that’s done, what you’re saying should be very easy at the PA level.
More system-level support for this (that is, support for doing this via the standard Android stack) is a bit harder, since the pieces are a bit more complicated, but this is still doable.
Anthony Maciel
October 20, 2012 — 12:47 am
Have you had a chance to work on getting BlueZ and your port of PulseAudio communicating? I’d be very interested in getting A2DP working as well.
And is there any way you could give a high-level overview of what might be required to get A2DP working with Android’s native AudioFlinger APIs? I know of a lot of people who would be interested in getting this functionality working, and I think that’d be the best way to do it in a cross-device friendly manner (unless PulseAudio becomes standard with Android). If it is straight-forward enough I plan on taking a stab at it, but I don’t have much experience with audio subsystems so if you could provide a starting point that’d save a ton of time.
Thanks!
Jesse Burt
May 28, 2012 — 8:39 pm
Arun,
This is great…it’s one of the first things I thought of when I got my GN :) Yong, is your work concerning using an Ubuntu desktop as an A2DP sink public by any chance? This is something I’ve sought for a long time, and only had very limited success with.
Thanks to both of you! Cheers, Jesse
Dmitry
October 1, 2012 — 11:29 pm
i successfully implement it with a2dpd. not in ubuntu, but in SLAX – not too much differ anyway. source available on the net is buggy and needs to be fixed, but after fixing it worked really well for me.
vitriolix
May 4, 2012 — 10:05 pm
Very exciting stuff. Nice work.
Christoph
May 4, 2012 — 11:23 pm
Very impressive work.
Is it possible to have the current Android sound system stream sound to a linux computer? This functionality would be useful even without pulseaudio.
Arun
May 7, 2012 — 9:56 am
If you want, say, to have all the audio from a single app streamed to a remote machine yes. There are likely apps to do this already, and if not, you could use PA in the NDK to build an app to do this (I’ll make my work usable from the NDK at some point in the not-too-distant future).
If you want to stream audio from arbitrary apps, I don’t know of a simple way to do this other than the stuff that I am currently working on.
skierpage
May 8, 2012 — 9:35 am
You can send audio over Bluetooth using Advanced Audio Distribution Profile, supported by most Android devices. Here’s a post about setting up Linux PulseAudio to receive it. There’s an apt-X codec for better audio quality supported by some Bluetooth implementations.
Another way to stream audio is the complexity and interoperability roulette of DLNA. I had some luck sending audio from my Android phone using the subpar iMedia Share DLNA app to a PlayStation3 (Digital Media Player -> DM Receiver), but had lockups trying to browse the phone’s tracks from the PS3 (DM Server -> DM Controller?). There are DLNA feature packages for GStreamer and PulseAudio that supposedly can do similar things, but it wasn’t obvious to me how to configure Kubuntu for them so I stuck with the low-tech cable with 3.5mm jacks :-)
Philip Mather
September 6, 2012 — 1:26 am
Skierpage or Arun, Any idea what it would take or how to get Apt-X on a Nexus? I have a couple of campatible devices but from everything I can find the GNEX doesn’t support it. My technical level is high enough that I can probably blunder my way through to coding at the NDK level (badly probably) but I’m concerned about the availablity of the codec seeing as it’s proprietary which is what I assume keeps it off the GNEX whilst it’s available on the S3? If I use PA is there a path to load Apt-X as a alternative codec somehow?
Frank Simon
May 7, 2012 — 4:49 am
I’m working on an app project that requires the need to send 44.1khz 16 bit stereo audio that is live streaming audio content from a PC or Mac and then transmitted over a WAP that is standard WI Fi at 2.4 and or 5 ghz, a, b, g, or n, and then it must be received, decoded and reproduced in the earphones of the cell phone in less than 80 msec from the time of being received by the cell phone with high quality audio reproduction.
If Pulse Audio has the ability to possibly perform this task with concealed packet loss with as little compression and buffering as possible please contact me as soon as you can as your product may be able to used as a demonstration for a product we are developing in the United States. An opportunity exists for future development of a new and exciting product and service.
Arun
May 7, 2012 — 9:40 am
Hi Frank. This depends a bit on the actual cell phone on the other side (do you control the stack?) and the various pieces in between on your stack as well as how stable the wifi connection is.
There are various pieces that need some attention to get them to work better. For example, latency reporting for the tunnel sinks is a bit broken. And while we do have an RTP module for streaming remotely, there is no packet loss concealment as part of this.
So to answer your question — if you have control over all the bits from start to finish, it is doable, but everything is not production-ready today.
Frank Simon
May 9, 2012 — 8:37 am
Arun,
Thank you for the quick response.
We do have the possibility to control all the bits on at least one device as long as the codec works with standard Wi -Fi hardware. We will be serving on a VPN and can develop protocol to suit the application. Our Wi-Fi delivery is already suited for high density and we can dedicate gigabit AP’s as required.
The delivery of the data via Wi Fi is under our complete control and we also have the ability to control certain functions of the handsets circuitry and software from a dynamic application controlled by our server in real time depending upon the ability of certain handsets.
I believe that you would find this product and service possibly an exciting opportunity to develop Pulse Audio in supporting a newly granted and unique United States patent for delivering streaming live music content directly to cellphones at a concert in real time.
Please contact me directly so we may discuss the possibilities. I look forward to speaking with you.
Kind regards Frank Simon Sound at TheAudioAuction dot com
Neeraj
May 8, 2012 — 2:33 pm
Great work Arun!..plz do post the latency results once get around to it..audioflinger latency is killing me right now..can’ wait to try dis out once the PA access via NDK is up..Awesome work again!
Rodrigo Carvalho
May 18, 2012 — 10:07 pm
I hope this work be successful! Google should use it so Android could have some nice audio applications as iOS has…
I was voting for this issue (http://code.google.com/p/android/issues/detail?id=3434) and I saw a comment pointing to this link: http://forum.xda-developers.com/showthread.php?t=1621914. Have you already seen it? I hope it is helpful for you!
Thank you for the good work!
Josef
June 7, 2012 — 3:44 am
Great work but it seems that the patches are down :/ Can someone (maybe Arun itself) please tell me where to find the patches or reupload them? Would be very sad if this great piece of work gets lost…
Josef
June 7, 2012 — 3:38 pm
Ignore my post, they’re up again :)
eric
June 14, 2012 — 6:43 am
Would love to see some ROMs integrate this. Hopefully it solves the audio lag problem. There are videos on YouTube demonstrating the lag via a Sonic The Hedgehog game in an emulator
Hansel
June 15, 2012 — 6:22 am
Great app..and I’m using it for almost a month now.
Meynard
June 16, 2012 — 9:33 am
It makes my phone sluggish maybe there are some glitches in ICS update so I switched to NewsAce Multimedia News Reader and find some great feature and good user interface it is the best app so far to replace Pulse and Taptu.
https://play.google.com/store/apps/details?id=com.tecace.newsace
The Android Love
July 8, 2012 — 2:31 am
Thanks for this great post. Pulse Audio System is definitely going to bring in a fresh change.
volcacius
July 15, 2012 — 12:42 pm
This is simply beautiful. Would it be possible to stream to a DLNA device coupling PA with a Rygel port? It would be such a great open airplay alternative.
Melanie
July 17, 2012 — 8:52 am
You better switched to NewsAce it is the same with PULSE but it has a better user interface and features.
Market Link:https://play.google.com/store/apps/details?id=com.tecace.newsace
CommenterInNeed
September 19, 2012 — 10:56 am
Have you possibly turned this into an apk? Does it have to be part of the rom to be used? I am no rom developper but what you have demonstrated is exactly what i need.
Thank you.
Dave Ferguson
October 11, 2012 — 5:00 am
Has there been any collaboration between PulseAudio and the OPUS codec?
William Smith
April 2, 2013 — 1:43 pm
Hi Arun,
Unfortunately, PulseAudio doesn’t build any longer with 4.2 AOSPs using the instructions you have posted.
Pulseaudio for Android would provide developers with exciting possibilities. Are you planning on continuing this work? This would be great.
Arun
April 2, 2013 — 1:49 pm
I haven’t had much time to continue the work, but the basic build should still work (someone did get it running recently). The build is a bit quirky – if you have an error message you could share, I can try to help.
Ramya
April 18, 2013 — 4:23 pm
Hi Arun,
I am trying to stream audio from my PC to OMAP4 based board running ICS on it using BT. I have enabled BT audio sink profile & able to connect it. But no audio.
So I tried the patch you have given on Android 4.1 ICS source. It gives some compilation error. ” Unable to locate format.h file. Tried googling, still no luck. ” How do I resolve this?
Any inputs? Thanks in advance.
Regards, Ramya.M
http://hackitch.
May 31, 2013 — 3:54 am
Everyone loves what you guys tend to be up too. This kind of clever work and reporting! Keep up the superb works guys I’ve you guys to my personal blogroll.
ubercoolguy
October 23, 2013 — 11:32 pm
hi arun i tried to install linux distro in android for this app with no use.. i want to play music badly when im chatting in skype or chatrooms from my android.. so pls i want an apk file which would have a checkbox to activate stereomix :/ like in windows or pulse audio in linux.. pls send me apk or tell me how to build it .. pls pls pls /\ i m in bad need of it..
ubercoolguy
October 23, 2013 — 11:48 pm
course we can play it from mic.. but users r abusing me for the worst quality :'( like mic should catch the output from speakers not surroundings.. i donno the terminology and technology but hope u understand my problem.. pls understand my feelings.. tried linuxonandroid which runs a vm inside android but its too slow operaating and eating up the battery.,
paul
November 19, 2014 — 1:34 am
Hello,
I’m trying to compile the app but it crashes on startup. I really want to use this for audio streaming :). Could u help me further with this, or send me a working compiled version? :)
Greets
Venkat
April 18, 2016 — 1:54 pm
Hi Arun,
Looks like the git link http://cgit.collabora.com/git/user/arun/pa-output-switcher.git is broken. If it is still available somewhere on the Internet, please update the URL.
Thanks Venkat
Arun
May 10, 2016 — 1:46 pm
Hi Venkat — sorry about the delay. There is now a copy of the repo at https://github.com/ford-prefect/pa-output-switcher
Michaël
April 19, 2020 — 6:59 am
The link for the patch is dead now; any idea where I could find it and whether it would still apply? Cheers!
Nikhil
September 13, 2020 — 10:43 am
Hi,
I have a different question from a end user perspective. I don’t know where else I should ask the question. I am using Microsoft teams on my mobile and then use wifi direct to share the content to my TV. But in this case, I continue to receive the audio from my phone. But if I play any recording either from Microsoft stream or from YouTube, I can hear the voice from my TV. I have used pulseaudio on linux and I thought if there is something like that for android, probably I can do something. That’s when I found your blog. Could you please provide me any pointers?
-Nikhil
Arun
September 25, 2020 — 12:17 am
Hey Nikhil — unfortunately, this is a question of Android routing on your phone. There isn’t an easy way to use the work from the post to do what you want at the moment.