As Jay wrote, editing out the noise and find alt takes is the way to go.
Sometime, I had acceptable results using the Waves X-crackle (at extreme settings) to smooth out the noise (but you can lose some of the voice character/definition).
By the way, the Waves x-crackle it's the only decrackle that I tried that can help with cloth/lapel noise, why? I think that what help, in those cases, are the "artifacts/distortion" that the plugin adds...
You can also check out the izotope RX spectral repair, it could help.