I'm making a DJ type program (it's geared more towards theatre productions) where I need to fade sound. I know that when you fade, doing a direct linear line doesn't sound natural and that a bit of a curve is better. Does anyone know/have an idea of what the exact curve should be. Is there a formula or equation or anything you can use to describe it?
There's no particular standard - there are lots of ways to fade audio. A straight line is one of them, but in many cases some kind of curve is preferable and it's down to your preference which one is best.
I found this image from this page about potentiometers and their fading properties, which I felt illustrated this well:
Some kind of logarithmic curve, like depicted here, is common, but some audio packages like Reaper let you customize it a bit further, using something like a Bezier curve. I don't know if that's the specific kind of curve that is used, but it or something like it should work just fine.
Although these represent cross-fades, you can get a feel for the wide range of possibilities (or perhaps read it as the lack of a standard).
doing a direct linear line doesn't sound natural
Hm. First off, fading isn't a "natural" thing to start with, so if anything sounds natural it's mostly what we're used to, rather than some "law" based on mathematics, physics or even psychoacoustics. Traditionally, fades were in fact done by hand, and the engineers usually would have a whole lot of stuff to their mind, so the exact curve was often something of a random process. Indeed many pre-70s records have fades that sound hasty, undecisive or otherwise odd.
There are sure enough some aspects of a fade where you can find objective criteria. Since a fade is something artificial, generally a means rather than an end, it should normally be as subtle as the time permits. This means, at no point change too fast, and also not too fast-changing derivative. Particularly, it is seldom good for a fade to start abruptly, rather it should smoothly get going, i.e. the "upper edge" should smoothly approach the constant "on/loud" level, in a round curve. At the other end, this applies in principle as well: if the curve is too steep there, it will sound almost like a discrete switch at this end. The most natural thing would be an exponential decay, like a guitar string. However, particularly for fades of just one track in a multitrack setting, the faded one will usually be inaudible already at about -30 dB, so the very low-end is hardly relevant. You shouldn't waste too much time here then.
For the middle part, there doesn't remain much choice, just smoothly keep the track.
Going after this, we get pretty much the "S-curve". That is IMO a good default for fades to 0. For crossfades, something constant-power-ish is often better.