How to reproduce any sound

Question

This may be a silly question, but it is one I don't know and I haven't found an exact answer to it.

A speaker can reproduce virtually any sound through vibration sequences, using an electromagnet. This means virtually any sound can be stored on a computer. I would like to know what the exact components are that are used in the production of any distinct sound.

For example, I understand amplitude and frequency. Amplitude, when toggled, changes the loudness of any tone. From what I get, adjusting the frequency would seem to alter the pitch (how high or low) the tone is. But can you reproduce any sound just by adjusting these two points? I don't see how these affect tone. You could have the same amplitude and pitch of a particular sound, though it could have a completely different tone, such as comparing a guitar string to a violin. What am I missing?

I am trying to liken my understanding to, for example, recording colors on a computer. This is simple. By adjusting red, green and blue from 0 - 255 on a scale, you can reproduce nearly every color on the spectrum, to a very fine degree (16,777,216 colors). You can do the same by adjusting HSV.

What is the comparable parameter for reproducing every possible audio tone on a computer? Is it just two sliders - amplitude and frequency - or are there other factors?

Thanks.

Answer 1

To reproduce a sound using a computer, you have to have captured and stored some information about the original sound. There are several approaches to this but I believe the two most generally used are sampling and frequency/waveform analysis.

Sampling

This is what generally happens when you record music using a computer. An analogue signal from a microphone carries a voltage that varies over time according to the varying air-pressure reaching the microphone. This is fed to an analogue-to-digital converter (ADC) which measures the voltage (samples it) at fixed time intervals and converts that analogue voltage into a digital representation - typically by assigning a small integer value to the value of the voltage. Since the precision is not infinite, this can lose some detail. E.g. converting 1.002, 0.998, 0.981 volts to integer values 120, 120, 119 say (numbers made up and are not representative)

^{Image from Planet Of Tunes}

The frequencies present are somewhat immaterial so long as the sampling frequency is much higher than the highest frequency you need to reproduce. In the example above it appears that the sampling frequency is perhaps 15 x the highest frequency in the musical waveform.

Note that human hearing extends roughly from 20 Hz to 20 kHz (much less as you get older) and CD sampling for example is around 44 kHz.

The accuracy depends on both the sampling rate and on the number of bits used to represent the amplitude.

Frequency analysis

Another way to capture and record the sound of an instrument such as a piano is to make a detailed analysis of the sound of striking a single key. An algorithm known as Fast Fourier Transform (FFT) can be used to determine what frequencies are present in the sound. Striking a hammer against piano wire causes the wire to vibrate in a complex mixture of modes which produces a mixture of frequencies whose amplitudes vary over time. The computercan then store information about which frequencies that are present to which extent and how the overall amplitudes vary over time.

Synthesisers generally use one of these two approaches, they either replay a captured sample of a real instrument or they attempt to reproduce the known sound using analogue electronic or virtual oscillators that produce sounds of known frequency (and wave-shape) - these are then passed through various filters and mixed with the output of envelope generators to re-create the intended sound.

the current state of the art seems to be that the most accurate electronic pianos (for example) use sample based synthesis.

Reproduction

Having stored a digital representation of an analogue soundwave, to reproduce it means converting a recreated digital signal to an analogue soundwave. This is carried out by electronic circuits which are essentially the reverse of an ADC and is sometimes referred to as DAC (Digital to Analogue Converter). This produces a signal that can be fed through an analogue amplifier to loudspeakers.

Tone

The difference in tone between a piano and a trumpet is largely one of the mix of harmonics and the dynamics of the note played (e.g. the "shape" of amplitude envelope - how volume varies over time - as seen in the Attack-Decay-Sustain-Release (ADSR) controls on a synthesiser). You can consider the sound as a changing mix of frequencies or as the complex waveform that results from combining them - they travel through the air to your ear in this combined form. It is mechanical parts in your ear plus subconscious analysis in your brain that makes you able to discern the separate frequencies present.

^{(note: the voltage levels in this diagram are not those encountered in typical audio recording but the principles are the same. audio signals are typically millivolts or at most a few volts depending on whether the source is a microphone, line-in or some other source. line-in signal levels are different for consumer and professional audio equipment)}

Adding a fundamental frequency (green) with a third harmonic (orange) produces a complex waveform (red). Only the red waveform need be sampled to reproduce the sound. You could instead, if you have stored information about the frequencies present, synthesize the sound by separately generating the fundamental (green) and the third harmonic (orange) waves using sine-wave oscillators and then adding them together.

Answer 2

Your analogy with colour works in a way, but remember that a picture is millions of pixels. One HSV triple gives you a colour, but it doesn't give you textures. With, say, 16 pixels you could approximate a texture. The more pixels you have, the closer you get to being able to reproduce 'any' texture.

When recording audio as a digital signal, we measure the movement of the microphone diaphragm thousands of times per second. For compact disc quality, that's 44,100 times per second, using a 16 bit number for each measurement. This is called sampling.

That series of numbers contains everything we need about how the air was vibrating - but a single number from the series on its own is of no use.

The numbers we record actually represent the velocity of the microphone diagram at a given moment in time. That's closely related to amplitude - the velocity is zero at the peak of a wave, and at its fastest as the waveform passes through zero. We can use maths to translate the series of velocity measurements into amplitudes.

By converting that series of numbers into voltages, and using those to drive a speaker, we play back the sampled sound.

That tells us how to record and play back sounds, but your question hints at something more subtle. What if we want to generate sounds from first principles, that sound like various instruments. This is called synthesis, and it is what a synthesiser does.

None of those numbers in the sample stream represents frequency. But we can look for frequencies in the data. For example, if we have recorded the purest sound possible - a sin wave - the sequence of numbers will hit a peak at regular intervals. The number of peaks in a given time period is the frequency. If there are 200 peaks per second, that's a frequency of 200 oscillations per second, or 200 Hz.

What you're missing about frequency is that sound usually consists of many frequencies all at the same time. That is, for example, you might find repetitions in the waveform every 131th of a second, and every 165th of a second (this would be the case for a chord of C and E played with sin waves).

The picture below shows a waveform consisting of two frequencies, both of which you can see clearly. The slower frequency repeats every 30 milliseconds - or 33.33Hz, and the faster frequency repeats every 5 milliseconds - or 200Hz.

There is a mathematical process called a Fourier Analysis which will take a sequence of numbers from sampling, and tell us the loudness of each frequency band.

A synthesiser can recreate that combination of frequencies in a number of ways. We can construct sounds by combining the output of a number of oscillators producing various frequencies. We can shape sounds by boosting or cutting frequencies in a filter.

Most sounds have more complicated waveforms, and so you would typically be able to find repetition at a number of frequencies.

But the construction of sound is endlessly complicated. A guitar string has a certain combination of frequencies immediately after it's plucked, then the combination changes continuously as the note decays. Synthesising it completely accurately is a big deal, involving many books' worth of expertise, science, art and craftsmanship.

Answer 3

Your question is difficult, since it consists of some sub-questions each of high complexity on its own. I try to give a few hints.

The first issue addresses digitizing sounds: this is always possible by approximation, which can be as exact as you invest effort. The sampling theorem says, that you need at least to sample with twice the frequency of the highest frequency to record. Since young human ears hear to something between 16 and 20 kHz, you need at least 40 kHz. But a pixeling (exactly: quantization) effect remains, like the difference between vector and pixel graphics. You can reduce it, but not get rid of it. (Possibly a reason for vinyl record fans.)

The second is wave form. The sine wave from physics lessons sounds completely boring, nearest approaching classical instrument is the flute. Violins have more triangle waves, which after fourier analysis can be represented by an overlay of many sine waves of different frequencies, the so-called overtones. This works well for the resonators found in classical instruments. The decomposition to sine waves reaches its limits with unnatural waves like square waves and I remember oscilloscope pictures from high end amplifiers showing small but clearly visible deviations for that waves. So to stay in your sliders comparison: you need something like a variable number of sliders to adjust each the amplitude of each overtone.

The third topic is time variance. Real instruments produce sounds varying considerably over time and therefore this frequency mix also constantly varies: attack, decay and sustain are the phases here, and attack is the most important here to recognize certain instruments. I read once from an experiment, where the attack template of an instrument A was molded on the frequency spectrum of instrument B and most listeners identified the sound of as belonging to A.

Answer 4

As an aside, you can get by with only one bit for audio I/O!

What they do is set up a carrier wave by flipping the value back and forth very quickly 1-0-1-0-1-0..., at a frequency outside of most people's hearing. It is flipped so quickly that the speaker cone doesn't have time to move enough to reach the 0 or 1 position, so the speaker cone stays close to a neutral position - say ½.

Ideally, you won't hear anything from the speaker in this neutral state. Changing some of the 0 bits to 1 brings the speaker cone out, and vice versa with changing 1 bits to 0, brings the speaker cone in. This motion of the speaker cone will be perceived as sound. Any sound can be reproduced adequately by this method. Officially this a variation of pulse width modulation - PWM.

Oddly, it works better on poorer quality speakers like in cell phones, because the low fidelity of the speaker smooths out the "rough edges".