A sample rate is the rate at which samples are taking from the source sound. It says nothing about how much information is stored in those samples. Whether an audio codec is considered lossy or lossless is dependant on how much information is carried over from the original recorded medium usually (for the sake of argument) based on the 1,412 kbit/s bitrate at 44.1kHz sampling rate which is considered to be uncompressed (CD quality).
Basically the sample rate it is the rate at which the original sound is sampled. MP3 does support low sample rates of down to 16kHz, but these are not very common, as frequencies around 8kHz will not reproduce well.
That's the sample rate. The bit rate is the amount of information that is conveyed at each sample. If you lower the bit rate you effectively lower the resolution of the sampling itself, which will lead to oddities in the audio even when using sample rates of 44.1kHz.
Strictly the resolution refers more to the amount of bytes required to encode each sample, which is why I used the word "effectively" since it sounds like a course representation of your music; whereas in fact MP3 encoding has access to the same sample resolution; however it removes a whole load of information.
44.1kHz is almost always used, however when you drop the bit rate, you generally hear some quite unpleasant side effects in the music. For example, uncompressed audio as stored on an audio-CD has a bit rate of 1,411.2 kbit/s, whereas 128kbit/s MP3 is quite heavily compressed, and even on cheap headphones you can tell the unpleasant differences on any track with more than voices.