I use a Tascam US-122L. It's a somewhat dated sound card with two microphone/line inputs with 96kHz/24bit specs, there are big dials for the respective two gains, one for setting the ratio of the main output (RCA jacks) between monitoring and computer, and one for headphone/main output volume. In a pinch, this will serve as a USB-powered mixer.
It's built like a tank and has no digital mixing circuitry included. So hanging it off some power supply instead of a computer (though no harm done by using a computer or powered hub) will land you with a minimal analog mixer and/or headphone amp. Frankly, it's probably heavier and bulkier than a more modern component and a power brick together, but then you can just stuff it in your bag without much of a worry that it wouldn't survive.
The gain dials don't go down all the way to zero, though.