In my last apartment, I drilled a hole in the wall to a built-in closet in the kitchen and put my computer there. With extension cables for everything (including the on-off button), I had a completely silent room where the ground noise from the speakers was the most apparent noise source.
In my new apartment, I don't have that possibility so I'm struggling with machine noise as well. Piece by piece I am replacing the loudest components in my computer with more silent ones. So far, I've replaced the processor cooler (Scythe Mugen Rev B, about $40), case fans (Nexus 800rpm or Scythe SL 800rpm, or Noctua 800 rpm, $7 each), computer case (Antec P183 which has triple layer walls and silicon mounting thingies for the hard drives, $140) and I replaced an old Maxtor hard drive with a newer, bigger one that happened to be much quieter too (Samsung Spinpoint F3). And an ssd for the operating system is on its way.
I would say the current configuration makes a third of the noise of my old setup. If after a while it's still too noisy for my taste, next steps would be a quiet aftermarket cooler for my graphics card (a passively cooled card would be better, but right now I want to keep this one), and then perhaps one of those "bitumen mat" absorber kits that cost about $20.
My last resort would be to go to ikea, buy a cupboard just big enough for the computer to fit and stuff it with mineral wool - leaving some strategic holes and open space inside for air circulation.