Splitting podcasts on the introductory jingle

Question

So I have a very large number of podcasts in mp3 format. Every podcast has the format:

[1-2 minutes of adverts] [10 second jingle] [Around 14 minutes speech]

The 10 second jingle is always exactly the same in every podcast.

What I am trying to do is to split the audio on that jingle, and thus isolate the 14 minutes of speech. I am thinking that this should be a simple matter of searching the audio for the jingle snippet to find out where it occurs, and then splitting off the speech from this point.

Because there are so many files, I can't do this in a program like Audacity. I need a solution that I can run in a script for all of the files. My platform is Ubuntu so I have access to command line tools such as 'sox' and 'ffmpeg'.

Any help would be very much appreciated!

Answer 1

I will answer on the assumption that the solution:

• can be given for only one file: you will know how to apply it to a bunch of files using any scripting language,
• has not to be sample-wise: a precision of around 100 ms is acceptable for the use case you describe,
• has to be able to deal with around 30 minutes long audio material,
• has to be able to locate the jingle inside the audio file even if small variations exist compared to the reference signal: quality loss (MP3), phase difference, sample rate difference,
• should limit extensive programming and take advantage of freely available tools: you mentioned sox, ffmpeg, etc.

Let's go straight to the burning question now:

How to locate a piece of audio signal inside a longer audio signal, with possible difference in quality and phase, without diving into tedious signal processing programming?

A similar problem exists in the imagery field and fortunately anyone can solve it using the following ImageMagick command:

compare -metric <metric> -subimage-search <full-image> <sub-image> <output>

In the following illustrated example, a part of the Stackoverflow logo, highly JPEG-compressed, is searched inside a bigger image made up of 16 different logos.

The metric used here is RMSE which gives good results. Fuzz, MAE and MSE give the same correct result, contrary to AE, MEPP, NCC, PAE and PSNR which should be avoided in our case.

The relevant information here is the precise location of the sub-image inside the full image. It is provided by ImageMagick as (X,Y) coordinates written to the standard output just after the @ symbol:

1031.85 (0.0157451) @ 108,178

Figures on the left can be ignored since they just indicate how "intense" the match is.

So, to locate the jingle inside the full audio file, it is now a matter of:

• producing a relevant graphical representation of an audio signal,
• converting pixel coordinates (in the graphical representation) to milliseconds coordinates (in the audio signal).

We could first think of the classical wave form representation of sound, but upon reflection, it does not fit our needs. Usually, audio is broadcasted at a sampling rate of 44100 Hz. This would lead to a 79 380 000 pixel wide image for a 30 minutes audio file if all samples were preserved, which is really too wide to be dealt with.

So reducing the width if the image would be equivalent to removing high frequencies. Thus, depending on the spectral content of your jingle, this method could fail.

A better representation of sound to use here is a spectrogram. The horizontal axis is still for time, but vertical axis is now for frequency, and each frequency bin is filled with its magnitude using a color scale (black to white, cold to hot, etc.).

This representation is rich and convenient for what we have to do. Indeed, condensing information can be done on both axes: loosing frequency definition is not that bad for common sounds comparison, and loosing time precision does not lead to removing some frenquency range. It's a deal!

It's thus SOX's turn to take the stage, with this hunky-dory one-liner:

sox <audio> -n spectrogram -X <time-precision> -y <freq.-precision> -r -o <image>

• <time-precision> is expressed in pixels/second (ensure -X is uppercase)
• <freq.-precision> is the number of frequency bins (ensure -y is lowercase)
• -r suppresses the display of axes and legends
• <image> is the name of the output PNG file

Be careful: since the spectrogram computation relies on the FFT algorithm, <freq.-precision> should be chosen as a power of 2 plus 1 to ensure fast computation. Thereupon, the lowest such value allowed by SOX is 65.

Here is a spectrogram example of the first minute of D.A.N.C.E. by French band Justice:

By doing the same with an audio query, it is now possible to use the sub-image search technique on audio files. Here are a few examples with different <time-precision> parameters (1px/sec and 10px/sec):

As seen in a previous paragraph, the position of the sub-image is given expressed in pixels. In our specific use, the Y coordinate will always be zero and the X coordinate can be parsed as follows (bash syntax):

pixel_position=$(echo $result | cut -d@ -f2 | cut -d, -f1 | sed 's/ //g')

Using simple math, time position can be deducted from pixel position and time precision, and trim time can be determined from time position and audio query length:

DECIMALS=6
TIME_PRECISION=<time-precision> # in pixels per second
QUERY_LENGTH=$(sox <query> -n stat 2>&1 |grep ^Length |cut -d: -f2 |sed 's/ //g')

time_position=$(echo "scale=$DECIMALS; $pixel_position / $TIME_PRECISION" | bc)
trim_time=$(echo "scale=$DECIMALS; $time_position + $QUERY_LENGTH" | bc)

Finally, the interesting audio part (from trim time to the end) can be obtained this way:

sox <input-audio> <output-audio> trim $trim_time -0s