Can I edit sound as fourier transformations?

Question

Classical waveform isn't very practical for editing. For me, it's not even helpful for viewing. All I can tell is amplitude and some guess about pitch:

^{Maybe not even that - one of these is e and other is E, but they look almost the same.}

There's however more suitable format to display sound. It's called Fourier transformation ¹. And computers have implemented a fast algorithm called FFT - Fast Fourier Transform. It looks like this:

^{All strings on my guitar. Produced using this program I made in hope to create nice pure javasript guitar tuner. Relevant question.}

Some description:

Ok, editing it in mspaint would probably not produce beautiful music, but maybe there would be a smarter way to edit this. Don't forget that what you see is a 3D function plot. These look like concave landscapes:

And many of us surely tried to edit 3D landscape in some game map editor. It's not that hard and allows you to do a lot of micro-edits.

Is there any software that provides convenient tools to edit the sound like that? Any tricks on that topic?

Edit:

I keep Googling and I found out that you can display sound as spectrogram (FFT) in Audacity:

But no editing there:

^{1: It doesn't apply on sound though - Fourier transformation is an important mathematics calculation in engineering - for any periodic function, it gives you sine and cosine coefficients to describe this function using sines and cosines only.}

Answer 1

Well amplitude is a pretty important thing to know about. As is waveform shape and DC offset.

Adobe audition has allowed spectral editing for many years now, as does Izotpe RX and Iris. 3d spectrograms for audio currently look like this:

(From Izotope Ozone)

I'm not sure editing in 3d would provide any more functionality than 2d which we already have.

Having said that, more specific selection tools are making this sort of thing more useful, such as being able to select all harmonics in an area.

Melodyne and MetaSynth are also worth looking at. In fact the MetaSynth painting approach might be exactly what you're after.

Answer 2

This is a screenshot I just made from iZotope RX. Note the irregular selection in the frequency domain, and the photoshop-like tool buttons in the bottom right.

iZotope has many frequency domain editing possibilities, as does Adobe Audition since the days of Cool Edit Pro, more than 10 years ago.

With an old tool called Coagula, you can convert image files into sound, which means that you can actually create something in MSPaint or Photoshop and have it played back as sound (there's even a guide on lifehacker: http://lifehacker.com/5807289/how-to-hide-secret-messages-and-codes-in-audio-files). Aphex Twin put his face in his track "Windowlicker" in this way:

Answer 3

Years ago I remember playing with Spear:

http://www.klingbeil.com/spear/

A nifty program that did just about what you seem to be looking for. Unfortunately, it's development was discontinued in 2009, and the latest version available is PowerPC only.

Searching around though, I found that Sony is offering something that looks similar, only much more powerful (and much more expensive), called SpectraLayers:

http://www.sonycreativesoftware.com/spectralayerspro

Answer 4

Additionally https://www.izotope.com/iris

seems weighted towards discovering the creative possibilities of mangling sound visually, which might be what you're interested in. While RX can be used creatively it's not what it's designed for (cleaning).