I am working on this project where we are doing a live performance with about 6 musicians placed away from each other in a big space. The audience will be wearing their headphones and as they move around we want them to hear different kinds of effects in different areas of the place. For calculating the position of users we are using bluetooth beacons. We're expecting around a 100 users and we can't have a latency of more than 2 seconds.
Is such kind of a setup possible?
The current way we're thinking of implementing this is that we'll divide the place into about 30 different sections.
For the server we'll take the input from all the musicians and mix a different stream for every section and stream it on a local WLAN using the RTP protocol.
We'll have Android and iOS apps that will locate the users using Bluetooth beacons and switch the live streams accordingly.
Presonus Studio One music mixer - Can have multiple channels that can be output to devices. 30 channels.
Virtual Audio Cable - Used to create virtual devices that will get the output from the channels. 30 devices.
FFMpeg streaming - Used to create an RTP stream for each of the devices. 30 streams.
Is this a good idea? Are there other ways of doing this?
Any help will be appreciated.
-
What is the hardware like that the users of the space will be using? Can you simply do a multi-channel stream and have the devices change channels? This seems like more of a multi-channel multiplexed broadcast is what you want.– AJ Henderson ♦Nov 17, 2015 at 15:31
-
@AJ Henderson. The users will be using their smartphones. We'll be creating Android and iOS Apps. You are right we want a multi channel multiplexed broadcast. This way all the mixing is done in one place and the phones just switch between streams.– SaneetNov 18, 2015 at 17:19
-
As @AJHenderson mentions, you might have a simpler setup by sending a multichannel stream and using either webaudio api or native platforms api to dynamically adjust the rendering (mixing, effects) on each device.– audionumaDec 19, 2015 at 7:42
2 Answers
I'd look for any encoder that can support MPEG-TS encoding for 30 channels. The problem you will run in to with doing 30 independent ffmpeg streams is that they will not properly interleave and you will likely get jumps when switching channels.
You need an encoder that can take in to mind all 30 streams and use a fixed clock to sync the packets of audio so that the latency is the same from one channel to the next. The MPEG-TS protocol will easily handle 30 audio streams and the ability to change between programs while maintaining synchronization as long as it is properly encoded, but you need an encoder capable of doing that.
I'm unfortunately not aware of any free or cheap systems that will handle this type of situation well. Either you need something that will handle 30 channels at once (not cheap) or you need something that can genlock between devices for clock sync (also not cheap).
Your best bet is likely still going to be to use a home grown approach with a 32 channel input, either from a 32 channel audio interface or from a 32 channel digital mixer with ASIO capability. The other nice thing about having ASIO capability is that you will be able to wire up the audio by channel without needing the virtual audio cable. Note that if they are stereos streams, you will actually need 60 channels of audio input, though you can use 30 stereo audio streams in the MPEG-TS file.
-
Hey thanks. This seems like it may work. Can you give me a few links about how I can get the encoder and about syncing audio on FFMpeg?– SaneetNov 21, 2015 at 22:25
Personnally, I would do the mixes using an actual mixing desk that offers modular scalabilty, like Digico desks or Yahama CL consoles. You put a stage rack at each musician location and like everything with network cables or optical fibre (if the distance is too great). A digico SD8 can easily manage 30 stereo mixes that you can then send to your broadcast thing. Doing the mixing in a console will keep the latency low, won't use computer ressources for the actual mixing and will actually free those ressources to do your encoding and your broadcast. It will also let you forget about timing issues. Relying solely on software that is not especially made for the purpose of live/realtime dsp is taking a risk in my opinion.