Suppose that we have a microphone-recorded note, say played on a guitar, and we want to write a computer program which will detect the note frequency. There are already plenty of such programs. How to write a new one from scratch? And, as a sidenote, how does a human ear (maybe for those with absolute pitch) detect the note frequency?
It may seem that the question already has an answer "read on Wikipedia about Fourier transform / FFT", but what is missing here is the window width, this is what the question is about. This question may seem a bit technical, so let me give some background first.
The sound is transmitted through the air with variable pressure. The microphone identifies air pressure at given time for several consecutive time points. Our ear, I suppose, works in a slightly different way, but I am going to return to this later. Everyone interested in acoustics knows that the pure sound is formed by a sine function
f is frequency,
t is time; the real sound with given base frequency is formed by the sum of sine functions. There is a lot of questions and answers here on
music.stackexchange about what are overtones and possible inharmonic frequencies, this is fairly well understood.
In order to decompose the sound "back" into the sum of sines, there is a so-called Fourier transform. Mathematicians know that if your function
f(x) is periodic with period
f(x+T) = f(x), then you can decompose the function into the sum of sines with coefficients, which are all computable.
From a musical point of view this means that if you have some low-frequency note, say
C1 played in a low octave, and also multiple frequencies of it, say
G3 and so on (harmonics), with different amplitudes, then
- from the shape of the waveform you can recover the amplitudes of the harmonics
- from the amplitudes you can recover the musical instrument that played this note, by comparing to already pre-recorded profiles of amplitudes.
But here we had an assumption that the function is periodic. In real life that doesn't happen because many instruments play many different notes and the notes change, so the period also changes. Fourier transform is of little help here, unless we know the true base frequency of the note, and we can throw out other notes as "noise" because they don't fit into the periodic picture. Note that a human ear of a trained musician solves this problem perfectly, detecting the base pitches (or at least, relative pitches), so all the information contained in the sound package is enough to infer the answer. I know that the ear contains some "hair" that vibrate in response to particular frequencies. Maybe this approach is developed to some extent?
Question. How to detect the note or a set of notes which sound at given point/period of time on a sound recording using computer only?
Also, "transcribe!" is a sort of computer program that can do that (I only heard but never launched), so I suppose that this is not so hard to understand on idea level.
Some related links: