5

If I have the spectrogram and waveform of a clip of audio what sort of information can I gather by looking at those two things?

2 Answers 2

6

I'll start from the very beginning: sine signals.

(from wikipedia)

Also known as pure tones, these kind of signals are periodic (i.e. repeat the same pattern over and over again). Anything that is periodic can be said to have a frequency, which is in fact f=1/T, being T the period. If you look at the image, you can see that the period is 2*pi and therefore frequency is 1/(2*pi) Hertz (Hz).

This one is, in fact, a time domain signal. A waveform. In this examples it's pretty easy to compute the frequency, but it isn't always like that. So the only real information you can extract out of it is the amplitude at each moment.

Sine waves contain a single frequency and, in fact, sound like this. If we apply the Fourier transform to a 200 Hz, we would get the following spectrum:

from mathworks

This means, roughly, that the signal's whole power is at 200 Hz. I will get more deeply into that later.

Speech, or music, are far from featuring just one frequency. You can easily distinguish a note played by an electric guitar from that very same note played by a trombone. Same musical note means same fundamental frequency, so there must be more frequency components. That would not be a pure tone, but a some of some of them. Look at what the signal looks like when you sum to sines (first row, a is time domain and c is its spectrum):

enter image description here

So now we have a time signal that seems periodic, yet more complicated. If we look at its spectrum, we now see two frequency components.

So let's generalize. According to Fourier's theories, any signal can be decomposed into infinite a sum of sine signals. If a sine signal had one component and its spectrum had one impulse, two sines had two impulses... infinite sines means a real spectrum, showing the most powerful frequencies as higher components. Infinite vertical bars mean "so many that you can't distinguish one from the other", so the plot appears to be a single one.

enter image description here

Now, let's get to practice. You can open any song in Audacity to have it plotted in time domain. If you select the whole song and plot its spectrum (Analyze > Plot spectrum...), what do you see? Well, you will probably see some sort of plot that means almost nothing. What would you hear if you played all the notes of a song at once? In the bottom row of the previous image, the two distinct sine signals happen at different moments but, as the spectrum of the whole signal was taken, it seems that it happened at the same time. So the same happened here: it's like printing a book in a single page.

So the real interest is taking a small sample of the audio (let's say, 0.1 s) and plotting its spectrum. Now we will see what's happening in that time lapse. You can, for example, see if low frequencies (bass) are too loud.

An interesting experiment is to take a brass instrument and plot its spectrum. You have samples here. As you are analyzing a single undistorted instrument playing a single note, you can see its harmonic structure: if you have taken a Tuba playing F2 (87 Hz), its harmonics would be 87*2=174, 87*3=272, etc.

But, in general, as I said, the most interesting information appears when you take a little time lapse. Wouldn't it be great if some sort of software looked into every little fragment and plotted it to you? That's the spectrogram, which shows how much power is into every frequency at every moment.

enter image description here

An example: this is the spectrogram of someone saying "Rice University". The spectrogram has a bunch of info impossible to obtain just looking at the waveform, i.e. the frequency components contained in each phoneme.

  • "R" sounds bassy, so has strong low frequency components.
  • "i" is a vowel. Those have an harmonic structure similar to the tuba's previous example.
  • "c" sounds like "s" which contain pretty high frequencies.

etc. This is this way up to a point that if an experienced phoneticist looks at an spectrogram can indeed "read" what the person said.

1
  • this answer is absolutely amazing. I had great pleasure when reading it! Thanks a lot!
    – Aidos
    Sep 8, 2020 at 10:29
2

Here below is an image of a waveform (top part) and a spectogram (lower part).

enter image description here

Waveform

This is a two-dimensional graph where the amplitude of the signal (your audio) is shown over time. The amplitude is interpreted as the volume, so you see your total volume progression over time.

Spectogram

This graph has three dimensions. X-axis is the time, Y-axis is the frequency and the colour represents the amplitude of the signal. Here below is a simple example to show you how the spectogram works.

First a 400Hz tone over the duration of 8 seconds: enter image description here You see that the lightest colour (highest intensity) is on the 400Hz line of the Y-axis. Some other frequencies slightly higher and lower than 400Hz also exist in the tone, but those have a lower intensity (darker colour) and are not the main tone.

Now a 200Hz tone shows the same behaviour. Highest intensity at the 200Hz frequency: enter image description here If I play the 400Hz and 200Hz tone simultaneously I see on the spectogram that both the 400 and 200Hz have a high intensity. enter image description here

4
  • I am a bit puzzled by this one - a pure 400Hz tone should not show other frequencies above and below the 400Hz line at all. Your spectrogram output shows frequencies of up to 100Hz either side of that midpoint - this is not normal!
    – Rory Alsop
    Jan 20, 2013 at 20:47
  • This might be caused by the way I produced the 400Hz tone. It was through Ableton Live (with which I'm not very familiar). But it was more to show an example of what a spectogram would look like and how to read it, and I think that is achieved in this answer. Jan 20, 2013 at 21:14
  • @DrMayhem I've looked into it and apparently the other frequencies are not there at all, my spectral resolution was just to low. I've changed it now from 512 to 8192 in Adobe Audacity and this gives me more precise spectograms. I will update the answer, thanks for pointing out the error. Jan 20, 2013 at 21:29
  • Lovely - the graphs look like I'd expect now :-)
    – Rory Alsop
    Jan 20, 2013 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.