A "physical device that is pre-programmed with the sounds of my choice" is called a sampler and what kind you'll need is entirely up to your personal requirements. They can be quite simple things that basically just play back short recordings when you press a button, all the way up to very elaborate devices that you can compose entire albums with. I personally have used the Boss SP-202 in the first link, and can say that while loading sounds onto it can be a bit awkward, it works quite well and simply once they're loaded.
However, to save cost, you may find that software works just as well for you, since you already have a computer running into your mixer. If you need physical buttons you can probably get a simple MIDI controller to trigger sounds with (a la Friend of George's answerFriend of George's answer), although at that point you may as well go with a hardware sampler for simplicity.