I'm a 3D artist and don't know anything much about audio, so please forgive my ignorance but, why is it easy to hook up the volume data from audio files to trigger an animation, but seemingly very difficult to use the pitch? Is there an audio format that would not require "spectral analysis of an FFT track"(?) to extract the pitch data, but a more simple way? So that the animation software can just match the movement of an object to the notes being played? Thanks
1 Answer
Short answer to that is that it takes a lot more processing to split the frequency spectrum up into sections and then use triggers(or envelope followers) on the level of each section, than to simply use a single trigger on the output level. I would have thought there'd be some out there though.
I've done a bit of 3D design on a personal level and I managed to get frequency-based reactions working great(I built a speaker with 3-way crossover) in Cinema 4D I think. Also in After Effects with third-party plugins, I created a video for a song I made and I had a full frequency spectrum reacting to the music. Lotta, lotta processing power for that one. I did have quite a few effects on it too though. LOOONG render time. It looked pretty darn cool if I do say so myself! ;)
As for the bit about audio formats, no. Any format would need to be analyzed in the same way.
If you are talking about a specific piece of music, then you could try to find the MIDI data for the music and use the MIDI notes to trigger movement.
With pitch, the fundamental has to be found for each specific note, then followed, frame by frame as the notes change. As I said before though, I'm sure it can be done, so I'm sure there should be something out there.
