Suppose you want to save the following sequence of numbers:
0123456789 0123456789
You can store those numbers individually, which would be equivalent to how uncompressed .wav or .bmp works.
But, if you agree on certain rules and define those in a certain format (like mp3), you can also save these numbers without explicitly writing down the value of each number:
Say we agree that r(0,9,1) means a linear ramp from 0 to 9 in steps of one. suddenly I only need to save 4 values to represent 10: r091 now gets me exactly the ramp. r091 r091 gets me the sequence I had before, using 8 instead of 20 characters.
So far, that's lossless encoding: the exact same data can be retrieved after decoding.
Now, suppose we know it's sufficient to build a ramp up to 7, because the values 8 and 9 hardly have any influence on the way the ramp works. Now we can save the values of the ramp as 3 bit values (2^3=8 spaces, just enough to save the number range from 0 to 7), where we needed 4 bit slots with 16 possible values before, because we had to save the number range 0 to 9. We just saved one bit for each value that is stored.
While this might be OK because the values 8 and 9 are not that important in this case, the original sequence can never be reconstructed again as soon as this information is deleted: lossy encoding.