A belated answer that may be helpful to anyone searching in the future... Disclaimer: I have not read through other posts here, so this answer may repeat information discussed in more detail elsewhere.
I have been using the PCM-D50 + Mic2496 combination for over 4 years to make stealth recordings of rock shows in clubs and concert venues. I also bought the DPA-4061 microphones that Core Sound recommends for use with the Mic2496 in high-volume settings. (Core Sound hand-matches the microphone pairs and wires them with an XLR connector.) I put the D50 and the Mic2496 back-to-back and use a custom-made optical link in the shape of a rectangular jumper. The result is the size of a brick but still something you can hide under your coat.
The results that I have gotten with this combo are excellent, IMHO. Many of the recordings I have made compare favorably with other audience recordings made at the same events with higher-cost equipment (e.g., $3K microphones and a $2K pre-amp). It is also more portable and does not require set-up at the venue. IMO, with this combination the electronics are not a limiting factor in the quality of the end result. The main determinants of the outcome are the recorder's location, the venue sound system and the venue acoustics.
I only have used the D50’s built-in microphones infrequently, usually for recording lectures. They seem to work well for that purpose. For music and other situations where higher fidelity is desired, they are good but can never yield results as good as dedicated microphones. There is only so much quality that can be built in for a given price point.
Another limitation of the handheld units is noise. The dirty little secret of these units is that the signal-to-noise ratio (SNR) of their electronics is not as high as it should be. For instance, I believe the SNR of the D50 is 93 dB. Since the rule of thumb is 6 dB per bit, this is merely adequate for 16-bit recordings, and not good at all for 24-bit recordings. In practice, what this means is that the noise inherent in the electronics will make its way into the low bits of the digital samples. Amplifying the recording will also amplify the noise. 24-bit recordings require an SNR of 144 dB (6x24). The Mic2496 has an SNR of 140 dB, which is nearly perfect for 24-bit recording and far beyond what is required for 16-bit recordings. In my experience the noise in 24-bit recording is undetectable. I usually amplify the results by 6-12 dB and I do not hear noise.
I use the same configuration for all recordings. In fact, I have taped over the switches so that I cannot accidentally change them. I have the Mic2496 set to a 96kHz 24-bit conversion rate (48 kHz sample rate and 2x multiplier), a fixed gain of 60 dB, the level LEDs off, and 48V power off. As mentioned, I use an optical link to feed the digital samples to the D50. The D50 is only used as a sample recorder in this configuration and any other sample recorder with an optical link could be substituted. However, the D50 is a very nice little unit with a great selection of features.
Core Sound does not specifically advertise the Mic2496 for use in stealth recording, but many of its features seem to have been chosen for that purpose; e.g., the lights-out setting for the level LEDs. Likewise, the fixed gain setting removes the need to adjust the levels when the performance starts. The recordings usually come out with quite a bit of head room and are nowhere near clipping, even in very loud settings. In 4+ years I have seen clipping only once, and that was only on infrequent isolated peaks.
I also bought the wired remote for the D50. This makes it easy to start and stop the recording without having to find the right buttons on the D50 itself. This is very helpful when making recordings in clubs and other venues where the lights are out during the show.
The net result is a combination that is perfect for stealth recording in high-volume situations. It is only necessary to turn on the Mic2496 and the D50 when the show is close to starting, then press record-pause and pause on the remote to start recording. As mentioned, the fixed-gain setting on the Mic2496 means that no adjustment of levels is required. The recording will have quite a bit of headroom, but due to the high SNR it can be amplified to the desired levels in post-processing without noticeable noise.
The Mic2496 does seem to draw a fair amount of power. A fully-charged pair of the Lithium-Polymer batteries that Core Sound recommends will last about 2 hours. If I am recording two sets, I bring a separate set of batteries and switch them between sets. Again, Core Sound has made this relatively easy to do in a stealth recording situation by putting the batteries in a carrier that plugs into an external connection on the unit. (It is also necessary to change between 4GB memory banks on the D50, which unfortunately cannot be done via the remote.)
FYI, for post-processing, I copy the data from the D50 to a PC using the USB connector and process the results using Adobe Audition. Audition is more than adequate for the editing that I do; there are many other digital editors that would work just as well. The Mic2496 has no built-in bass roll-off, so it is usually necessary to reduce the levels at the bottom end. I also put a V notch in the mid range to separate the bass and drums from the other instruments, and boost the high end a bit. The exact adjustments vary from recording to recording. I then boost the entire recording to -0.1 dB to get the maximum volume without clipping.
I hope people find this information useful. I have been very satisfied with the Mic2496 + DPA 4061 combination. If my equipment were stolen I would buy them again. I might look for a PCM that is smaller than the D50. If you already have a D50, as I did, the Mic2496 is a great front-end that will boost the quality of your recordings immensely.