How to make realistic instrument sounds digitally from scratch?

Question

I am looking at basic JavaScript Web Audio API examples, such as building simple synthesizers and such, hoping to find something where they create the sound of a drum or the sound of the human voice singing, or of the dynamic nature of an electric or acoustic guitar, but nothing. From my limited knowledge, these synthesizers are built from simple sound waves which are combined and distorted and manipulated in relatively simple ways to create different "effects" like the phaser or distortion or whatnot. But capturing the nature of the vibrating strummed string on a guitar, or the plow of a drum at different velocities in different rooms/environments/atmospheres seems like a whole different thing.

Wondering if one could just outline what it takes to build such a realistic guitar, drum, or voice tool purely from scratch from basic components. What does it take? (Other than obviously probably a lot of work)? Is it simply a complex wiring/system of oscillators, or is it more like a physics engine for sound? If so, where can I find a reference to such an open source engine for inspiration? What do they entail at a high level -- the key pieces to create such realistic sounds? Not looking for an in depth exposition on such a broad topic, but simply an introduction to how it works, and where I can find inspiration on how to build one.

The key point to this question is I don't want to use samples of real sounds to build up a digital instrument. I want to create some sort of acoustic environment where sounds can be created, but my imagination is not quite equipped in knowing what is possible or how people typically go about doing this.

Answer 1

It sounds like you're asking how synthesis works.

There are a few different types of synthesis. Historically, the earliest synthesisers did just use electronic components to create sounds. Later, digital technology started to make use of microprocessors to generate sound. Soon after that, digital recordings were manipulated to generate the sounds. Most recently, manufacturers have been making use of computer technology to run simulations of the original electronic components of older machines.

In basic subtractive synthesis, an oscillator produces a tone that is then shaped by various processes such as filtering, amplitude modulation, pitch modulation etc. This is also known as "East Coast Synthesis" and was pioneered by people like Bob Moog

With additive synthesis, harmonics are stacked on top of a base frequency to generate a complex waveform which can then be shaped by various processes. This is also known as "West Coast Synthesis" and was pioneered by people like Don Buchla.

Most modern synthesisers use one of these models. Sometimes they are also blended with samples or snippets of waveforms taken from real instruments.

When trying to recreate a sound in synthesis, you must understand the sound you're trying to replicate. For instance, with a snare drum, there are three basic elements: the snap of the stick against the top skin, the resonance of the shell and skins, and the rattle of the snare spring on the bottom skin. The two skins will have pitches, there will be an element of noise from the springs and the whole thing will start very suddenly and tail off quite quickly. A violin will be different because it is made of different elements: the bow rapidly grips and releases the sting thousands of times every second, the cavity of the violin adds resonances and filters some frequencies, the sound can start quiet, rise in volume, hold indefinitely, and fade away as quickly or slowly as the player likes.

Understanding how an instrument makes its particular sound will get you a long way to recreating something similar.

As has been suggested in other answers and comments, there are plenty of resources out there (books, magazines, websites, etc) that will help you get a more in-depth idea of how synthesis works. It's a vast subject and gets more complex as you try for more realism.

Good luck with it!

Answer 2

New Complete Synthesizer by David Crombie, Omnibus Press, 1986, ISBN 0711907013 includes a chapter on which synthesizer aspects (waveforms, envelopes, filters etc.) can be used towards simulation of a few types of acoustic instrument sounds.

Answer 3

Real Sound Synthesis by Perry R Cook is a fairly elementary book with sample code that discusses synthesis of quite a few different physical sounds including some musical instruments.