In a typical CD recording (44.1kHz 16 bit), there are 44,100 samples per second and each sample represents a number between 0 and 32,768. This (theoretically) would enable us to playback a square wave sound at a little over 22kHz at full volume if we wanted to.
Any form of compression removes parts of these scales - for example, if we were using a 22kHz square wave, we would only need the top and bottom values. Instead of using all 16 bits to give us 32,768 different values, we could just use 1 bit and translate 0 as 0 and 1 as 32,768 during playback - saving 15 bits of information in each one of those 44,100 samples.
Most recordings don't use 22kHz square waves so we won't always be able to drop down as far as 1 bit but we might be able to take fewer samples per second.
Bad compression can be heard in early MP3 files where both the sample rate and the bit depth were reduced to remove data and therefore give a smaller file size.
Different compression types have different rules guiding which bits and which samples they can lose to find a compromise between sound quality and file size - I'm sure you could find some very dry data on exactly how this works for the different filetypes you mentioned somewhere on the web but it's probably PhD level mathematics.
Essentially, compressing sound files means taking away some of the data that the computer thinks you don't need and the different filetypes have different rules for doing that.