I have a secret to confess. I’ve spent a great deal of time over the last few months talking to myself. I can’t say I haven’t enjoyed it — it turns out my capacity to entertain myself is far greater than initially suspected. But I hear you ask … why?
Here at Collabora, I’ve been building on Wim’s previous work on adding echo cancellation to PulseAudio. Thanks go to Intel for supporting us in continuing this work. Before too long, all this work will be trickling down to your favourite Linux distribution and all your friends will stop hating you.
First, a quick recap on what acoustic echo cancellation (AEC) is. If you already know this, you might want to skip this paragraph and the next. Say you’re on your laptop, and you receive a voice call from your friend. You don’t have a pair of headphones lying around, so you’re just going to use your laptop’s built-in speakers and mic. When your friend speaks, what she says is played out the speakers, but is also captured by the microphone and she gets to hear herself speak, albeit a short while (a few hundred milliseconds or more) later. This is called acoustic echo, and can be frustrating enough to make conversation nigh impossible. There are other types of echo for phone systems, but that’s not interesting to us at the moment.
This problem is common on pretty much all devices that you use to make phone calls. Astute readers will ask why they don’t actually face this problem on their phone. That’s because your phone (or, if you have a cheap phone, your phone company) has special software hidden away that removes the echo before sending your signal along to the other end. On laptops, which are general-purpose hardware, the job of echo cancellation is left to either your operating system (Windows XP onwards, for example) or your chat client (Skype, for example) to provide.
On Linux, we implement echo cancellation as a PulseAudio module (code-ninja Wim Taymans wrote this last year). We use the Speex DSP library to perform the actual echo cancellation. The code’s quite modular, so it’s not very hard to plug in alternate echo cancellers (we even include an alternate implementation, which isn’t quite as effective as Speex).
Recently, we plugged in some more bits from the Speex library to do noise suppression and digital gain control (so you can quit twiddling with your mic volume for the other end to be able to hear you). We also added a bunch of fixes to reduce CPU consumption significantly — this should be good enough to run on a netbook and reasonably recent ARM platforms.
While all this sounds nice, I think a demo would sound (haha!) nicer …
Without AEC: /downloads/pulseaudio/aec/call-no-aec (or download ogg, aac)
With AEC: /downloads/pulseaudio/aec/call-with-aec (or download ogg, aac)
This is a recording of a call between my laptop and N900. The laptop is playing audio out the speakers and recording with the built-in mic. What you hear is the conversation as heard on the N900.
All this echo cancelling goodness will come to a Linux distribution near you in the upcoming 1.0 release of PulseAudio. The next version of the GNOME IM client, Empathy (3.2), will actually make use of this functionality. In due time, we intend to make it so that all voice applications will end up using this functionality (so if you’re writing a VoIP application and don’t want to use this functionality, you need to set a special stream property to disable this — filter.suppress="echo-cancel").
For the impatient among you, you can try all this out by getting recent testing versions of PulseAudio (I know packages are available for Ubuntu, Debian, Gentoo and Mageia at least). To force your phone streams to use echo cancellation, just run pactl load-module module-echo-cancel, and you’re done.
There’s still some work to be done, refining quality and using other AEC implementations (in the short-term, the WebRTC one looks promising). Things don’t work at all if you’re using different devices for playback and capture (e.g. laptop speakers and webcam mic). These are things that will be addressed in coming weeks and months.
August 25, 2011 — 2:44 pm
Nice stuff! Glad to see that this works its way into Pulseaudio, so that all applications will benefit.
August 25, 2011 — 7:32 pm
So, could I use this to have a combination of baby monitor and white noise generator? Right when I listen to our baby monitor receiver, I mostly hear the white noise generator in the room.
August 27, 2011 — 5:02 am
Presumably you could do this with standard tools in Linux. Unless I misunderstand what you need, you’re problem isn’t related to echo-cancellation.
August 25, 2011 — 10:06 pm
This is really nice work. Can’t wait to get advantage form this in Empathy for SIP calls. :-)
August 26, 2011 — 4:11 pm
The release of this feature makes me deeply pleased. Good work chaps.
September 10, 2011 — 10:34 pm
Asif Ali Rizwaan
September 28, 2011 — 6:39 am
Great Work Arun,
Is there a noise cancellation in pulseaudio also? because recording using mic, catches too much fan noise, clicks etc., in Linux.
Hope to see noise cancellation in pulseaudio.
September 28, 2011 — 8:32 am
Hey Asif — as I mentioned the Speex preprocessing bits also include noise suppression, and I’ve seen this be quite helpful with fan noise. Clicks, on the other hand, might either be your system not being able to keep up, or driver bugs.
Asif Ali Rizwaan
September 29, 2011 — 2:38 am
I want to record Hindi pronunciations at forvo.com, but with Flashplugin and Microphone there is a constant ‘static’ noise.
That noise is not there in any other sound applications, arecord, sound-recorder, empathy, skype, etc.
Please see these 2 audio clips (2.5 seconds each) http://www.forvo.com/word/mukherjee/#hi
In the second recording (mine), there is a noise with flashplugin.
The same ‘trrrrrr’ noise is not there in windows.
Is there a solution? The opensource gnash and lightspark do not have mic support when I used them.
September 29, 2011 — 8:30 am
Maybe Adobe’s directly accessing things and picking the wrong mic? https://bugbase.adobe.com/index.cfm?event=bug&id=2968177
Asif Ali Rizwaan
September 29, 2011 — 7:24 pm
I am thinking of buying a noise cancellation mic, like Logitech USB Desktop Microphone http://www.logitech.com/en-gb/speakers-audio/microphones/devices/221 http://cgi.ebay.in/Logitech-USB-Desktop-Microphone-Black-Silver-/150631222113?pt=LH_DefaultDomain_203&hash=item2312520b61
Once I get it I’ll share the noise issue with you. :)
Asif Ali Rizwaan
October 19, 2011 — 9:29 am
I got the Logitech noise cancellation Microphone from US of A.
Here is the recording with Logitech USB microphone: http://www.forvo.com/word/arun_raghvan/
the fan noise is reduced considerably.
Moreover, I got my nokia 5800 xm to work as microphone which has lesser echo and noise recording. Using Picobrothers Mobile Microphone and 3.5mm Stereo Audio Cable (both sides male) to connect 5800 to pc mic port. http://www.forvo.com/word/asif_ali_rizvan/
October 21, 2011 — 8:58 am
I’m not able to play stuff on that site unfortunately, but the capture path is probably directly from ALSA (that was the bug I linked to). It’s something that should be fixed in future versions of Flash.
November 5, 2011 — 7:01 pm
Hi Arun, I tried the WebRTC test programs recently. Echo cancellation does not work if using different devices for playback and capture ( a USB microphone for example). What is your experience with it?
November 6, 2011 — 7:24 am
It does work on the short tests that I did. Some more extensive testing is on my todo list.
January 16, 2012 — 5:45 pm
Just in case, you didn’t try adopting WebRTC echo cancelation to frame sizes other than 10ms, did you? (They require frame sizes to be 10ms, but I need 48ms.)
January 19, 2012 — 4:35 pm
Nope, I just broke up my blocks into 10ms chunks.
May 5, 2012 — 7:18 am
Hi, I am trying to use this module for my social robot. The problem is that the robot has buil-in its body the microphone which is used to Automatic Speech recognition, but the robot voice is collected for the microphone too. I want to substract its own voice of the signal received for the built-in microphone, because if not substract this signal the ASR give me misunderstandings.
If the user doesn’t talk but the robot is speaking I want to received a silence signal… but if users start to talk (even at the same time that the robot) I want only received the user voice. Previously I used a wireless headset microphone to avoid the robot voice signal but it is more handy and natural to use a built-into microphone.
Do you believe that this echo-cancel module is suitable for me purpose?
May 7, 2012 — 9:32 am
Sounds like the echo-cancel module should suit your purpose. The current RCs have significant improvements in the quality of echo cancellation so you might want to try that out.
May 8, 2012 — 6:52 am
mmm RC’s? the new algorithms implemented in Pulseaudio 1.1? I spend all day “fighting” with Pulseaudio 1.1 in Ubuntu 10.10 (by default is installed Pulseadio 0.9, and it hasn’t echo cancellation). But really I still have some problems… I have many sound cards devices connected to the laptop, in ALSA always start them in the right order (by /etc/modprobe.d/alsa-base.conf), but in Pulseaudio some times, some sound cards are not detected, and in the other hand, the another problem is that I write in /etc/pulse/default.pa the module-echo-cancelation for loading it… but the module is not loaded… I need to write in a terminal the instruction later have been started Ubuntu. I guess the problem is that Pulseaudio 1.1 is not the default of Ubuntu 10.10.
July 25, 2012 — 5:11 pm
I downloaded and converted call-no-aec.oga to wav format. Should I notice echo cancellation if I play this audio file with module-echo-cancel loaded with appropriate sink and source?
August 3, 2012 — 2:52 am
Hi Fahad, this isn’t how echo cancellation works. If you play a wav or ogg or mp3 file that exhibits a recorded echo, it will just play back that stream, echo and all.
The echo cancellation module is meant to work with the hardware combination of speaker and mic.
May 28, 2014 — 8:19 pm
i’m implementing a voip application using gstreamer, i use the example of the rtp in the plugin-good! i want to implement echo cancellation, i couldn’t use the speex echo canceller with gstreamer because the input and the output are not in the same process. So, i want to use pulse audio to make echo cancellation? would you help me how to deal with? thanks