What's audio equivalent of FPS in video, and what's the current limit of capturing the maximum?

Question

In video you have "frames per second" which defines the number of individual pictures that are taken in a second and when strung together they make up 1 second of video footage. The upper limit of capturing frames per second currently seems to be about a million, and it's called super slow-mo.

What is the audio equivalent of all this?

Also, I see there are softwares that can stretch a given audio to 50 times, but they always sound synthetic or weirdly electronic. How close are they to if the audio were actually captured in slow-mo?

Answer 1

The audio equivalent would be the sample rate which is the number of times per second a discrete sample is taken. The higher the sample rate, the more time-resolute the result.

To sort-of answer your second question, the higher the sample rate the more likely that a time-stretched signal would resemble an original at the stretched rate.

Answer 2

The important thing to realise is that time-stretching audio works in a fundamentally different way from video slow-motion. For video, it's really simple: you just record at a higher frame-rate and slow that down to something more usual.

If you apply the same technique to ordinary audio, what you get is not just slower, it is in particular also lower in pitch, because the frequencies themselves are slowed down (lower pitch simply means, the oscillations are slower). The well-known chipmunk effect was originally created by just speeding up tape tracks. If you just slow down, you get a cheesy monster-voice kind of effect.

Modern software avoids this, roughly speaking, by splitting up the audio into its frequency components for each time¹, slowing down the stream of these frequency coefficients, and transforming that back. Trouble is, “frequency at a given time” isn't really well-defined: you have to give the signal some time to “swing”, to be able to measure frequency at all. The longer you make these time windows, the more accurate the frequency spectrum will be, but at the same time you're losing the time resolution, and that's what you particularly need for strong slow-downs.

It's a nontrivial tradeoff. Time-stretch algorithms basically have to guess what the “real” evolution of the spectrum would be – there is no way to determine it exactly. Hence strong slowmo audio sounds so synthetic: is is synthetic; the algorithm has to guess/synthesize information that isn't really in the recorded signal. This has nothing to do with the sample rate, just with the physics of audio as such.

Some algorithms can actually do 50× slow down without sounding obviously artificial, but this really just means they are particularly clever at synthesizing the inferred part in a way that fits in “naturally” with the real sound components. It's still synthesized though.

---

¹_{Incidentally, you could say the same happens also in video, and well before the signal is even digitalised: the camera sensor of a colour camera separates different frequencies of light – that's just what colours are! – and stores only the information about the amplitude of each frequency component, rather than recording the light waves as such. Light waves oscillate much, much faster than audible sound (hundreds of terahertz), therefore its no problem do make the time-windows very short, like a microsecond. For sound, the absolute minimum length of a time window is the reciprocal of the highest audible frequency, i.e. 1/20 millisecond; in practise, you have to make it much longer to actually get any useful frequency resolution.}

Answer 3

To try to answer the second part of your question. If you want your audio to sound right when it is slowed down, you will probably have to capture at a much higher bitrate. You may still get some pitch shifting but you should not get the robot effect as that is caused by your DAC trying to process incomplete information.