How can layers of sound be distinguished, and therefore separated?

Question

Let's say I have a file like WAV. It has singing and background music.

What if one was to separate the background music from the background music, if this is even possible? If it possible, and it is not a manual process, how does the computer know what is background sound and what is the singing? How are the "layers" of sound distinguished? Is there some sort of mathematical reasoning to it?

---

If the above is impossible, then what if I have access to a few seconds of the song without singing? Would that affect the feasibility of the above?

Answer 1

The most common vocal removal technique is based on the fact that in most songs the vocal is centred in the stereo spectrum, whereas a significant part of the other song components are not. It's possible, in fact a relatively easy operation in terms of audio signal processing, to compare the two channels of a stereo audio signal and remove what is common between them.

How well this technique works depends on the specifics of each song. Sometimes it does not work at all, either because much of the audio content is equal in both channels, so what remains is insufficient, or because the lead vocals are not precisely centred and so cannot be removed this way. This can happen for example when stereo effects (e.g. reverb or delay) were used or there was layering of multiple vocal takes that are spread over the stereo spectrum.

When that happens, filtering is the easiest next best thing, i.e. applying low and high pass filters excluding the frequency spectrum occupied by the song lead vocals. The problem with this technique is that probably much of the musical content of the piece will go along too. With experimentation it may be possible to narrow the filters in order to remove most (perhaps not all) of the vocals and still leave a useful, albeit not perfect, karaoke track. A filtering band from 200 Hz to 3000 KHz is generally a good starting point.

These two techniques can be combined to optimise results, i.e. only the stereo centred audio is removed within a frequency range.

You can try the combined technique with Audacity, an open source audio application, using the included effect "Vocal Remover".

Additional professional/industrial techniques combine these approaches with more sophisticated ones, like analysing in detail the frequency spectrum and targeting very specific frequencies, or removing spectrum components that do not remain very stable for a certain period of time (as the human voice has natural oscillations that most instruments don't have). These can be done by error and trial and targeting different techniques at different segments of the song to optimize results.

Answer 2

There are no "layers" when it comes to sound files. There is no actual difference between the vocal track(s) and the instrumental track(s). Its all one track. You can hear different things because each thing has its own timbre and set of frequencies, and we are able to distinguish them because we can very clearly detect patterns and differences between different sounds. Computers are just machines and they can't just "learn" these patterns and differences.

The only "layers" that are still present in a stereo track is, of course, the left and right channels. There are very easy ways to remove everything that isn't centered, and because many tracks have the vocals down the center and other instruments that are not centered, this can be an effective way to isolate the vocals in a track.

With most tracks, it is possible to isolate a single instrument or sound manually using tools like Izotope's RX5, where you can look at the spectrum and basically carve out parts of it to get just those frequencies. There are a few ways of doing this automatically, but they are not really dynamic enough, the results will definitely vary, and you'll still get a better result by doing it manually (if you know what you're doing).

By the way, RX5 has a noise removal function in which you can give it a sample of just the vocals and that would improve its ability to isolate the vocals or instrumentals on the rest of the track.

In terms of mathematical reasoning, I think there are a lot of ways one could go about mathematically removing vocals from a track, but the math would vary from track to track. It is very difficult for a computer to tell the difference between vocals and instrumentals. Its like giving a computer a picture of a group of people and telling it to remove Bob when it has no idea what a Bob would look like (and not all Bobs are created equal).

If you want to look into the "math" of changing something so dynamic, I suggest you look into machine learning.