I was listening to my alarm this morning from a lofi telephone. It's actually a step up from lo-fi so the sound quality is considerably improved, can't tell you the actual specs. The alarm itself is this chill, padlike song for like 10 seconds and then it repeats, like any alarm clock. So I was listening this morning and noticed that the piano part kind of blended in at certain parts with a pad in a very gentle way that made the piano's attack go away some and I was wondering if it was masking that I heard or just good mixing. So it got me wondering how useful masking really is, rather than something to just be avoided. Maybe I don't understand it well enough and maybe we can uncover some of the confusion and learn something new.
Masking in a film mix is something to be aware of when preparing & layering sounds... I remember one case in particular where I just thought WTF?! It was in the film Black Sheep and this girl is being hassled by a 7 foot tall mutant sheep in the kitchen (!?) and to distract it her boyfriend grabs a haggis and throws it across the room, the thump of the haggis landing was what motivated the uber-sheep to stop attacking the girl & scurry across to sniff the haggis etc... I edited together a realistic 'haggis hits floor/wall' sound using 4 or 5 layers, put it up in the mix and it just didn't rate. Of course the score is playing the action/jeopardy loudly at this point, as are the uber-sheep vocals & the girl shrieking etc... So over the next half hour I kept adding layers until we had something that made the haggis landing rate in context. If you solo'd that haggis landing you would have heard a ridiculous over the top composite sound - a VERY hard hit combined with a subby short explosion etc etc.... But in context it just sounded quite normal, all due to masking!
Masking can be a useful tool, especially when it comes to dealing with noise issues in a given audio track (i.e. production audio). There's frequently noise components that you can't get rid of with noise reduction tools...well, maybe you can, but it usually hurts the rest of the audio in those extreme cases. If removal detrimentally affects the sound, attenuate the noise as best you can without adversely affecting what you need. After that it's time to look for your masking solution.
If you can incorporate it into your sound design (if it works aesthetically), a sound that has components that will mask the, now attenuated, offending element will finish what you started with the noise removal. You can do this with musical elements, sound effects or dialogue depending on the original sound.
A lot of people also forget that, in addition to frequency masking, there is also temporal masking. A hard transient sound can effectively mask a softer sound that occurs slightly before it. Yes, it will mask a sound that occurs BEFORE it. We're not talking about huge gaps here; more like on the order of 10s of milliseconds. It can, however, be just the thing you need to "hide" that element you can't get rid of.
Along the lines of something that Shaun has already mentioned -- something I learned quickly when first cutting (fast turn-around..) TV shows that have noisy exterior environments, is that nice tidy fade-ups on regions aren't always the best idea. If you don't have time to fill, or if the idea of adding more noise is just ridiculous, then cutting hard on the incoming dialogue can sometimes turn what would be a patchy edit in to a seamless one. I think it's a combination of not giving your ear any clues as to the change about to happen, and also that the incoming dialogue will mask the change in noise considerably, aided by the compressor tapping down the transients and lengthening the period that your ear has to get used to this new noise. It can make for a far more palatable edit.
It seems to me that the ear is far more accepting of an environment shift following dialogue than if it's preceded by it. To a degree of course..
And of course spending the time filling, expanding and/or noise reducing is better, but when the clock is ticking...
I work mostly in Games where no 2 playback systems are exactly alike. My concern about masking is that it is very dependent on playback environment and equipment - very unpredictable. Its a fine line(s) between masked, mushed, and overloaded. Mixing is tricky at best in games, but where possible I err on the side of getting rid of freq range buildups and parts of sounds that are not relevant.