Are there any open-source alternatives to VST/AU/RTAS that work across multiple OSs and can be used in host-programs such as Ableton's Live that are either in development or fully functional?
Unfortunately, no. Programs like Ableton Live only host certain kinds of plugins. In the case of Live, these are VST and AU. So even if someone had come up with another format for plugins, your host program wouldn't be able to load them.
Audio Units is an API which is compatible with Open Source Software licenses. There can, in theory, be open source plugins released in AU format. In practice, there are not a lot of high quality ones available (that I know of), but here are some that I found:
Of course, AU is not cross platform, so maybe those won't work for you. VST and RTAS, on the other hand, seem to be incompatible with OSS licenses. :(
There are at least 2 open-source standards for audio plugins in the Linux world, LADSPA and its erstwhile successor LV2. They were designed for Linux, but at least LADSPA has been ported to Windows and Mac OS, and there are cross-platform hosts such as Audacity that support LADSPA plugins on all 3 platforms.
If you aren't stuck on a particular DAW platform (I don't think any of the commercial ones support LADSPA plugins), you might look into it. There are a lot of good free plugins for dynamics and EQ; the reverb options are limited but there are several good ones; and if you prefer "bang" over "bling" in your plugins you will find a very high bang/bling ratio in the open source world.
There are a many open source VST plugins. Many of these can be compiled cross-platform. Here's a few of them:
Don't know if that is what you want, but it may be pertinent: Juce http://www.rawmaterialsoftware.com/juce.php which allows parallel development of plugins for AU, VST, RTAS... (if you already have VST, RTAS [...] SDKs).