Apologies if this is outside the purview of this forum, but I thought that if people here are doing fun things with all kinds of sound, why not?

I'm curious to know if anyone here has used a video signal AS an audio signal. I'm doubly curious to know if anyone has ever recorded video to an audio format (tape, digital, vinyl, etc.), and has then been able to transcode back to video.

Am I correct in thinking the obstacles have to do with bandwidth? My understanding is that a black and white video signal can be up to 5 Mhz, obviously problematic for audio media.

Pixelvision and Vinyl Video seem to have overcome these obstacles. In the case of the former, the solution was to speed up the tape, effectively increasing bandwidth. I'm not sure about the latter - looks like they've employed some kind of mystery decoder box.

But what if you wanted to be able to encode and decode video without extra hardware, using the medium's standard speeds/sample rates? Are there any suggestions on creating a "low bandwidth" video signal, which could maintain useful information in the audio range? Or would you suspect that the only way to make this happen would be to use some kind of encoder/decoder? Or the more likely answer, which is that there's something I haven't even thought of!



I never did anything like that, but at the place i worked at for a while in new york, Adam Rokhsar did a lot of cool stuff like that. My favorite was when he played a picture as audio and captured that from the other side of the room and translated the result back to picture. Really cool and unpredictable results. That was done with Max/MSP You can check some of his stuff at: http://makeyourselftransparent.tumblr.com/

I can put you in touch with him if you want.

  • @Filipe that would be great, thanks. It sounds like he was dealing with digital data (Audition used to be good at this, I remember) so I'd be curious to ask him his thoughts on doing the same with analog video. I'm pretty new to SSD. Is there a way to PM? – David Herman May 11 '11 at 17:48
  • @David there isn't, but shoot me an email: filipe.chagas@gmail.com cheers – Filipe Chagas May 11 '11 at 17:51

I got intrigued and just had a go at taking the video out of my DVD player into my recorder's line in. Got a constant buzz with some harmonics in the higher frequencies.

I reckon you're on the right track with bandwidth... you could obviously use a lower bandwidth cable, or one with high impedance or resistance to try to get a usable signal. Perhaps add resistors along the chain? I's guessing you'll end up doing a fair bit of circuit bending if you want to be getting something more interesting.

On another note, I'm still wanting to try out my new toy. An induction coil pick up that will pick up electrical signals. http://hydrophones.blogspot.com/


I've actually been looking into something similar lately. My thoughts on this have been to use something like Max/MSP to do real-time conversions from the MHz range to the Hz and kHz range.

The problem is that from what I can tell about the light-spectrum distribution of colours, it appears that the exponential pitch/octave progression of sound is not matched by light/video. In fact, rather than any kind of direct progression (whether linear or exponential) the light spectrum works in frequency band "blobs", green and violet being the biggest. This poses an obvious problem for direct (1,000 : 1 or 10,000 : 1 conversion). I've been looking around for a while to see if anyone has done the math and come up with the necessary equations. So far I haven't found anything easy enough for me to understand.

I don't know how "musical" you want your resulting sound to be, but if your aim (like mine) is for anything even approaching consonance, I fear we may have a fair bit of complicated math in our future.

  • @g.a.harry, that's interesting. I had thought a solution would come in some kind of transposition down into the audio range. But it seems problematic to do the conversion inside Max/MSP. If the video signal is in fact on the order of Mhz, an AD converter would have no way to capture all of the information (unless it was able to sample at 10 Mhz!). It seems like, if this were to work, it would have to happen in the analog domain. My end goal is to use audio as a storage medium for the video data, and to see how that influences the resultant video once it's decoded. – David Herman May 12 '11 at 12:48
  • Ah, I see. I misread what you wrote. | The reason I was looking at Max/MSP for this kind of stuff is that it's offshoot "Jitter" is an application for manipulating video in real-time. I've seen various projects and installations that make use of full colour (if not resolution) video feeds. I admit that it never occurred to me just how we'd get the video in. | I have no idea how you'd to it in real life. You could try running an RGB cable into three separate inputs on an audio interface that does 192k. You might get some interesting quantization artifacts. – g.a.harry May 12 '11 at 20:35
  • Oh, here's a wacko idea. Try running your analog video feed through an octaver guitar pedal. If it does a second octave down, it might get you close enough to the audio spectrum to be useable. Lord knows what kind of distortion you'd get though... and you'd have to run it through a pitch-shift on output too... – g.a.harry May 12 '11 at 20:37

Look at this link text


David, just saw this this morning and thought of your question.

It's not a direct video signal->audio conversion, but with the right amount of parameters and routing set up, you could probably still get some very dynamic results out of a video signal:





MAybe you want to have a look at THIS

I havent followed the progress of this application lately, but it looks really cool. basically it reads gestures from a video input which are then mapped to MIDI or OSC controls ad the rest is magic, as they say. But he adds in the second paragraph that he was looking at patching the code to allow a pre-recorded video to control the synthesiser...

Not exactly what you were looking for, but interesting nonetheless.




I do a lot of stuff with transforming still images into sound with Quartz Composer, Max/MSP, Metasynth, Photosounder etc. - but am only now starting to explore doing it with moving picture. The new pre-built add-ons for Max/MSP make this a lot more accessible.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.