1

I've just done a general pass over the 'net looking for ways to automatically synchronize audio.

This seems to be a nontrivial topic due to the way subtle circumstantial variations seem to demand fundamentally different solutions.

In my case, I have several copies of the same streamed audio broadcast that was published on multiple online services, which I want to perform differential spectrographic analysis on so I can highlight and quantify poor encoding quality.

To achieve the above, I need to correct for the fact that the various copies of the broadcast are out of sync by between 500ms and 50 seconds due to some copies having pre-stream leadin and some not.

Furthermore, since some of the copies were streamed to services that perform remote encoding (like YouTube), subsample-level shifting may be required to get perfect phase alignment, to (IIUC) correct for encoding drift induced by network jitter during the stream upload. So phase analysis with subsample accuracy would be needed as a prerequisite.

The reason I want to automate this process is that the streams are between one and five hours long, and there are approximately two hundred of them, and I want to "perpetuate" the synchronization so everything stays perfectly in sync throughout the length of the track (ie to reverse the effects of network jitter).

I don't expect there's software out there for Linux that will achieve the above. (Especially not freeware, or better, open-source.)

Perhaps there are audio toolkits (with eg Python bindings) that can be scripted to achieve the results I'm looking for though? I think all I really need is 1) a way to get two audio tracks to mostly within a few milliseconds of each other, then 2) some kind of contraption that can direct a phase-realignment process.

Caveat emptor: I barely understand the concepts I'm describing above. If there are black-boxes out there that completely solve the subsample phase-alignment problem but need complete handholding everywhere else (eg to actually then do the synchronization/reencoding/everything else), that would be within my attention span. If I needed to look over literature, or go drown in DSP theory for a month to figure this out... I might, um, be taking a raincheck. :)

Improve this question

1 Answer 1

Reset to default
1

If you decide on the DSP bunny warren, then you'll be looking at autocorrelation and fft.

the problem with trying to an autocorrelation alignment is that any changes to one of the streams will then render any further measurements moot as you will then have to consider the changes made by the autocorrelation routine on top of the codec artifacts you want to detect and measure.

Also, to make any useful measurements and/or comparisons, you will need access to the original un-encoded audio stream prior to broadcast. You won't be able to do anything useful simply be comparing multiple codec versions without having a known reference point.

Raincheck?

Improve this answer
5
  • Thanks for the headsup regarding autocorrection, that looks like it'd get me to the 99%-aligned mark. I only grok FFT up to the point it turns audio into waveforms :) but I can see how that would help with the phase analysis bit.
    – i336_
    Commented Nov 15, 2019 at 3:17
  • Good point re the "realignment feedback" from autocorrection. One idea I had was to perform realignment, consider the offset my new "base", track all audio sources from that point until I detected I was out of phase again, then interpolate the speed of one or both tracks so they're at the new realignment point.
    – i336_
    Commented Nov 15, 2019 at 3:22
  • ...Reading the last comment back, I can't help but remember the joke of the mathematician who notices his apartment is on fire, solves for the exact minimum amount of water to pour, location(s) to pour, and angle and velocity... and then goes back to sleep. Of course in this case my apartment isn't on fire, which sort of inverts the outcome of going back to sleep, as it were. Heheh.
    – i336_
    Commented Nov 15, 2019 at 3:42
  • And yeah, regarding the original goal of comparison... I was hoping I'd be able to do something useful by comparing each of the poor-quality copies to each other. Chances are I'd probably see something, but given all the comparisons it would probably not be a very intuitive result. The other thing I was hoping to do was align everything and play all the tracks as a single merged whole, because (in back-of-envelope experiments with hitting Play at the right millisecond) this seemed to produce subjectively better-sounding audio.
    – i336_
    Commented Nov 15, 2019 at 3:51
  • 1
    FFt is an utterly wonderful tool which on it's own is useful for turning time-domain data into frequency-domain data, but with a bit of quite simple additional maths, can be used for convolution and correlation functions as well. Check it out - it's awesome!
    – Mark
    Commented Nov 16, 2019 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.