It's easy to move a sound from left to right with panning. But is it possible to give the illusion the sound is moving away / toward / up / down the listener ?
Yes! The effect is referred to as a binaural recording. The idea is to capture and playback the audio stream in a manner similar to how your ears would have heard the audio stream to begin with, were you sitting in the room when the capture was made.
Take, for example, someone walking from left to right across a floor that is over your hear. If you're looking straight ahead you know right away that: the sound is above you, it starts on the left, and it moves to the right. You can figure this out because your brain takes the two sources it has (your left and right ears) and analyzes the delays between each source to produce a location for the sound in three dimensional space. It's pretty darn cool when you think about it.
You can make a binaural recording by using a binaural microphone setup: it's two mics mounted on either side of a dummy human head. An object that has roughly the same size, shape and density as a typical human head. What the mics capture in this case is audio data that's encoded very much as your ears would have encoded it if they were in the room listening to the sound.
Here's an image of a Neumann KU-100 binaural microphone:
I grabbed that image from this site which has some awesome, high quality binaural recordings you can listen to. For best results headphones are recommended.
But, since we live in The Age of Wonder and Amazing, you can now turn any audio recording in to binaural recording through the use of plugins like Wave Arts Panorama. These handy plugins will take a source signal and let you "place it" in three dimensional space. They take care of all the left and right channel delays as well as any filtering and phase shifting that needs to happen to make your brain think the sound is in some particular place in space. So anyone can make a binaural recording now, even if you don't have the microphone setup.
Not exactly 3D movement, but you can 'displace' sounds, or make them sound like coming from interesting places, with short delays -- I like FabFilter Timeless presets for this.
For the theory behind it, http://www.moultonlabs.com/more/principles_of_multitrack_mixing_the_phantom_image/.