Downsampling multichannel audio (5.1) into stereo (2 channels) with ffmpeg

Wed, 2009.02.25 - 23:50 — müzso

I googled quite a bit to find patches for ffmpeg that will allow conversion from 5.1 audio channels into stereo. This is necessary if you want to convert all sorts of videos into FLVs with mp3 audio channel (eg. if you operate a video streaming service), since mp3 supports only mono and stereo channels.

Here're the patches:

[FFmpeg-devel] [PATCH] Electronic Arts Decoder
There's another thread ([FFmpeg-devel] Decoder-independent multichannel downmixing) about the patch that you might want to read. The original patch deals with a number of issues and not just 5.1->stereo downsampling. But somebody has extracted the essential part as a separate patch ([PATCH] merge 4 or 6 channels to stereo). I've attached the latter (resample.diff) to this post so it won't vanish in case something happens to that mailing list archive.
There's another approach that I like better ([FFmpeg-devel] [PATCH] 5.1->stereo resample). This is far from perfect too, but still better than the first one, because that alters the audio volume levels and this one does not. I've attached a copy of this patch too (6to2channel-resample.patch).

Unfortunately neither of these two are as sophisticated as the algorithms in various media players (like Quicktime, VLC, Mplayer, etc.).

P.S.: I'm aware of the fact that Flash Player supports AAC audio codec (and H.264 video codec), but only since v9.0.115 (see the respective article in Adobe's Wiki). There're still many computers (at the moment of writing) that run a lower version of the Flash Player. So depending on your goal (on percentage of supported clients), you might want to choose to encode your FLVs with H.264 and AAC (which supports 5.1 audio).

Attachment	Size
resample.diff	1.59 KB
6to2channel-resample.patch	2.79 KB

Comments

Wed, 2011.02.09 - 11:51 — Jules (not verified)

Link change

The location of your second link has changed, for some reason. The patch can now be found at http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-January/040472.html

Anyway, thanks for posting this. Just tried this patch, and it works brilliantly.

Wed, 2011.02.09 - 14:40 — müzso

Re: Link change

Thanks for the feedback. Apparently most of the referenced links have changed.

Fixed them for now. Having quoted the title of the posts helped a lot in tracking them down.

Sun, 2011.05.01 - 09:12 — darkdragn (not verified)

The world changes

The resample.c file has been altered considerably over the three years since these patches were put out, and having said so these patches do not work anymore. However, they core of the code for the file remains just about the same, so I was able to extract the hunks and make a new patch if anyone is interested.

diff -rupN ffmpeg/libavcodec/resample.c ffmpegMod/libavcodec/resample.c
--- ffmpeg/libavcodec/resample.c        2011-05-01 03:31:32.897900334 -0400
+++ ffmpegMod/libavcodec/resample.c        2011-05-01 03:32:03.097900335 -0400
@@ -53,6 +53,14 @@ struct ReSampleContext {
     unsigned buffer_size[2];         ///< sizes of allocated buffers
 };



+static short clip_short(int v) {
+    if (v < -32768)
+        v = -32768;
+    else if (v > 32767)
+        v = 32767;
+    return (short) v;
+}
+
 /* n1: number of samples */
 static void stereo_to_mono(short *output, short *input, int n1)
 {
@@ -104,14 +112,43 @@ static void mono_to_stereo(short *output
     }
 }



-/* XXX: should use more abstract 'N' channels system */
-static void stereo_split(short *output1, short *output2, short *input, int n)
-{
+/* XXX: make this better.  channels will always be >= 2.
+ - Left = front_left + rear_gain * rear_left + center_gain * center
+ - Right = front_right + rear_gain * rear_right + center_gain * center
+ where rear_gain is usually around 0.5-1.0 and center_gain is almost always 0.7 (-3 dB) if I recall correctly.  */
+static void multi_to_stereo_split(short *output1, short *output2, short *input, int n, int channels) {
     int i;
+    short l,r;



     for(i=0;i<n;i++) {
-        *output1++ = *input++;
-        *output2++ = *input++;
+        if (channels == 2) {
+            /* simple stereo to stereo. Input is: l, r */
+            l = input[0];
+            r = input[1];
+        } else if (channels == 6) {
+            /* 5.1 to stereo. l, c, r, ls, rs, sw */
+            int fl,fr,c,rl,rr,lfe;
+            fl = input[0];
+            c = input[1];
+            fr = input[2];
+            rl = input[3];
+            rr = input[4];
+            lfe = input[5];
+
+            l = clip_short(fl + (0.5 * rl) + (0.7 * c));
+            r = clip_short(fr + (0.5 * rr) + (0.7 * c));
+        } else {
+            /* channels must be 3-5, or >= 7. l, c, r, ? */
+            l = input[0];
+            r = input[2];
+        }
+
+        /* output l & r. */
+        *output1++ = l;
+        *output2++ = r;
+
+        /* increment input. */
+        input += channels;
     }
 }



@@ -151,9 +188,9 @@ ReSampleContext *av_audio_resample_init(
 {
     ReSampleContext *s;



-    if ( input_channels > 2)
+    if ((input_channels > 2) && (input_channels != 6))
       {
-        av_log(NULL, AV_LOG_ERROR, "Resampling with input channels greater than 2 unsupported.\n");
+        av_log(NULL, AV_LOG_ERROR, "Resampling with input channels other than 1,2, or 6 is unsupported.\n");
         return NULL;
       }



@@ -302,7 +339,7 @@ int audio_resample(ReSampleContext *s, s
     } else if (s->output_channels >= 2) {
         buftmp3[0] = bufout[0];
         buftmp3[1] = bufout[1];
-        stereo_split(buftmp2[0], buftmp2[1], input, nb_samples);
+        multi_to_stereo_split(buftmp2[0], buftmp2[1], input, nb_samples, s->input_channels);
     } else {
         buftmp3[0] = output;
         memcpy(buftmp2[0], input, nb_samples*sizeof(short));

Sun, 2011.05.01 - 09:20 — müzso

Re: The world changes

Thanks for the updated patch! I've added [code]...[/code] formatting codes around your patch so it keeps whitespaces (indentations). I wonder however why did this patch not make it into ffmpeg?

Sun, 2011.05.01 - 09:43 — darkdragn (not verified)

Not sure, and good point

I couldn't tell you. In all seriousness, it is a bit of a tragedy that it hasn't. There are literally thousands of questions, comments and threads floating around talking about this and wishing it was implemented. I'll see what I can do about talking to some of the developers. For now, I'm a bit of a geek, so I'll stick with my weekly routine of pulling from the git repo, applying the patch, compiling and building a deb. If you want a copy I can stand up a ppa or something to sync to, ^_^.
I do want to thank you, while I'm here. Without your post, and the links to the original patches I'd never have been able to make the new one. Considering that I know absolutely nothing about audio, let alone that tidbit about the proper way to calculate left and right on stereo with the correct gains, it was actually pretty cool, ^_^. All I did was seek, cut and paste... yet it was still fun, and made me feel accomplished, ^_^.... *squees like the anime fanboy I am*
Have a good evening, and one last time... Thanks. If it wasn't for you keeping this post, and keeping these links up to date I'd still be figuratively beating my head against a wall, trying every ffmpeg switch combination known to man in an attempt to get it to work. ^_^

Sun, 2011.05.01 - 09:54 — müzso

Re: Not sure, and good point

You're welcome.

I'm not an experienced ffmpeg-developer either. Back in 2009 I did just the same as you: searched for a solution on the web. And while at it, I documented it on my blog, so later on I could find it easier (and possibly help others finding it). However, recent years showed that the development of ffmpeg is quite a mess.

It's quite disorganized and chaotic from what I see. I'm seriously considering a switch to MEncoder for my video+audio conversion needs.

Sun, 2011.12.25 - 17:04 — Tom (not verified)