Hi folks,

apart from my occupation as sound designer, my main focus (since I'm working at a university) lies in media (technology) research, software development, and (sonic) interaction design.

As such I'm excited about the quasi-revolution that is taking place in the user interface realm through the development of tablet devices and the like. Also, software companies such as Native Instruments or Ableton are currently pursuing new ways of human-computer-interaction.

Contrary to this, at the latest audio convention I attended, to me it seemed like the only innovation were the obligatory 16 channel strips added to the large mixing consoles, etc.

Since I'm somewhat rooted in both worlds, I find these developments very exciting, and am constantly thinking of how a new kind of user interface which sort of abandons the old "build a mixing console as a software package" way of doing things, could revolutionize sound-design and mixing. Without any more allusions from my side, I would really be interested in what kind of interaction you envision for your future (computer) work?

I'm also thankful if you have some thrilling examples of nice interaction design for me... I try to follow developments as tightly as possible, but there's so much going on at the moment...

All the best, Julian

5 Answers 5


As a professional interaction designer (and that's for context, not to set myself up as any kind of expert), I think the "future" is hard to nail down without sounding either kooky or like Ray Kurzweil (which some would argue is the same thing). However, I can offer what I've seen and heard recently that might offer some suggestions and ideas for your exciting investigation.

  • For all the talk we do of placing sounds in the stereo (or any multi-channel) field, there's been no serious attempts to visualize it in a professional manner, covering spread, gain, and frequency.
  • I agree with Tim that haptic feedback is sorely lacking. While one always wants to remove cognitive friction, physical friction is what makes things truly tactile. This is why I tape printmaking paper on my drawing tablet: Plastic on plastic doesn't feel anything like the real drawing or writing I do on a daily basis, and the lack of physical friction creates cognitive friction as I try to be visually inspired while managing the oddness of the experience. (Many haptic feedback actuators are driven by low-frequency sound files playing back, let that simmer in your brain for the possibilities...)
  • I strongly believe that the future of interactions is a range of input methods, not just one. Not just gestural (massive fatigue and huge accuracy issues), not just keyboard (indirect, proxy control), not just speech recognition (failures in loud environments and in translation). We're in an era where we don't need to assume someone will input commands in just one way - remember that Kinect's most overlooked feature is that it's not just a gestural controller, but it does speech recog., too. That's a trend I think we'll see more of: Contextually-appropriate controllers.
  • People think of touch and multi-touch in terms of touch*screens*, but that's limiting. The touch-sensitive encoders and faders on mixing boards really is a huge productivity changer. Heck, for scrubbing, I'd probably rather have a ribbon controller on my keyboard than a jog wheel.
  • XY input devices like the Wacom Intuos and Bamboo, or even the XY touchpads on Novation controllers, allow for two axes of input rather than one, which is a huge think. Could the new Apple USB touchpad input device - heck, two of them at once - be an interesting model for mixing? (remember, though, that some things like faders probably still want strictly-controlled single-axis movement)
  • In my opinion, all interface design should reduce the apparent distance between input gesture and the system's response, yielding at least a perception of more direct manipulation of a value, setting, or a piece of content.
  • Investigate peoples' mental models of things like mixing and editing. Have them draw how they thing of sound data spatially, away from hardware and out of the studio environment. You might gain interesting insights...
  • The radically new is almost never adopted by the market. Look at the QWERTY keyboard as the ultimate example. The key is to retain enough affordances from the old way of doing things that people are drawn in and somewhat comfortable, and then sneak in some innovation around the way the familiar things work, or a new thing in addition. Dreyfuss (who largely created what we now know as human factors) calls such things "survival forms:" If you design a pencil sharpener to be even vaguely pencil-sharpener-esque, people will know what to do with it, but if it looks like a featureless egg, no one will know where to begin.
  • 2
    that, Sir, was more than I could hope for. thanks!
    – Julian
    Feb 7, 2011 at 17:46
  • Well, just one guy's opinion, so take it FWIW. Glad it might be helpful! :-) Feb 7, 2011 at 23:05

I had a mixing lecturer who told us that our generation would be the one to make huge changes in the physical aspects of mixing (eg. faders, pots, etc.). I'm constantly trying to think about what would improve things.

One thing that comes to mind is some kind of X/Y touchscreen/tablet automation interface. You could set the X and Y axes to whatever parameter you choose (eg. volume/pan, volume/send vol, etc.). Maybe even add a third axis that's related to finger pressure. This might require greater finger dexterity, but i've been messing around with GRM plug ins lately (which have a similar X/Y axis with a dot on it that controls 2 parameters) and i find that having access to 2 parameters, with the one control, makes things more intuitive.

Also, my feet don't do much while i'm mixing. It'd be pretty cool if i could have pedals to skip fader banks or stop/play or something.

  • yes, the 2D interface keeps spinning around in my head, too. i was also thinking of maybe arranging channels in a circle and achieving the best possible mix by positioning the "listening point" in it... I have to admit I have never thought about using the feet... Thanks!
    – Julian
    Feb 7, 2011 at 8:53

Force feedback is one aspect totally lacking from a lot of computer based ui design. Whether its knobs on an analogue synth, controls on a real mixing desk or playing an acoustic instrument, part of what allows people to develop skill & dexterity is friction - there is art in that friction! These empty weightless gestures on many of the new devices really do not feel like the future to me. A simple example: I'd like a surround panner on my iPad but I suspect if I get one I'll end up sticking things like plastic disks on it to provide physical friction and/or to restrict movement.

  • @tim I actually really love the touch screen pan module on the D-Control. The joysticks on it just don't do it for me. You can get some really great pan moves with it. It's nice (but not mandatory) to graphically see where in the sound field I am and just glide my finger across it. It's a very intuitive process that feels more natural than grabbing the joystick or a knob. Oddly enough, the touch screen on it does provide a decent amount of friction and I can't see that it would be any different on an ipad. Feb 16, 2011 at 2:57


there is much truth in what you say. it reminds me of a short section in Theo van Leeuwen's "Speech Music Sound" (p. 138f):

"In this area, too, electronic sound presents a challenge, partly because it often becomes very much a 'mind event' rather than a body event, difficult to relate to physical, bodily experiences and physical, material objects, and partly because the kinds of actions we have to perform to produce them often bear little connection to the sounds themselves, as when a vibrato is produced by touching a button rather than by vibrating one's finger on the string, or the sound of a Spanish guitar by tapping the keys of a piano rather than by plucking strings. Sounds lose their materiality, their link with the actions that produce them, and as a result their meaning might increasingly come to be seen as conventional rather than motivated. If we are to keep mind and body together, and not reduce all semiotic action to moving a mouse on a mouse pad or tapping keys, we urgently need to develop new, subjectively meaningful 'interfaces' between physical actions and the sounds they produce."

I believe, though, that this gap can be overcome. Maybe I'll make a dissertation out of it :)

(sorry, this post was too long for a comment :) )

  • maybe its all just darwinian ie it simply takes time - once artists & musicians have been using the new generation of ui for a decade or six we will see art as genuinely astounding as that achieved with a rosined bow and a stradivarius, or a drummer & their battered kit...
    – user49
    Feb 8, 2011 at 9:54

@Tim @NoiseJockey - Oddly enough I'm a huge fan of the surround panner install on the Digi/Avid ICON consoles. Touch screen integration (+ the joystick) as well as variable glide functions ( via variable speed/velocity glide ratio via the jog wheel + Hot Key/Keystroke control variables) and it also doubles as a Pong interface ala: some of the SSL J and G series consoles. Slick stuff. Mixing without one is actually kind of shitty and overly complex without one, especially once you get spoiled by access to one an you have to go back to multiple automation pencil drawings or multiple fader combinations.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.