If you normalize all of your samples to -0.1 dB, then use them in your DAW projects, you'll end up clipping your audio tracks all the time as soon as you use a plug-in.
I use Pro Tools where the channel faders default to unity gain in a new project. The channel fader on an audio track is post-plugin. This means if I put a Kick Drum sample on Channel 1, and it's peaking at -0.1dbFS, then I apply some channel strip plug-ins, it's quite likely it'll be clipping straight away. Now I have to go into the plug-in and adjust the input gain level.
I prefer recording at around -6dbFS, to give myself a little headroom when I use it. This leave me to bring up the gain on the mixing screen, where I want to be when adjusting levels.
If you're using something like Kontakt, to put all your samples in, you can just adjust the overall gain. However if you put individual samples through FX chains in Kontakt, you might get the same clipping problem. You'll end up turning down all the samples you normalized to the max.
This is completely personal preference though, and applies to your work flow method and the tools that you are using. There is no hard and fast rule. If -0.1dbFS works for you then it makes the most sense. It all depends on whether you find yourself turning every sample track up or down all the time, in which case they all need adjusting.
If I had to choose one option I'd go for the larger headroom of -3.0 dB, but personally I don't do this.