The interesting thing about reverb, at least in my own experience, is it tends to be the same situation as the human voice - subconsciously, our mind is very picky and can easily determine what sounds true, fluid, and natural and what sounds fabricated/altered. It's a high pintpoint of our species' hearing sensitivity to nuance.
So this, in general sense, may be why you're facing a lot of frustration with this - it's not easy to replicate believably, it's even a challenge for seasoned pros too. Many times it's finding that happy medium where our ears 'buy' what we're hearing but not spending an astronomical; amount of time and effort beyond what would be reasonable. This is where IR reverbs tend to help, many work quite well in a pinch, but even they need help sometimes. And it also depends on the reverb plugin - some (in their overall algorithms, regardless of what IR or parameters you plug in) sound metallic, some sound lush, others sound smooth and rich. So that's part of the challenge, in knowing which ones you like for which certain occasions - some work well for voice, others sound like they were destined for music treatments. A lot of finding what you like for which occasion just comes with time and experience working with them - it's hard to find a "magic bullet" for something like this. Run a search on here and I know there's a reverb post on here, I believe a post by Tim, which has quite a few responses tossing out a variety of reverb tools, some of which I had never heard of - some reverbs out there which immediately come to mind are TLSpace, TrueVerb, RVerb, Altiverb, Revibe, ReverbOne, Valhalla, but there are many many others.
My guess is that what will head you int the right direction as far as the verb situation you speak of is to set up a slap delay (i.e. short delay with little/no feedback), THEN follow up that slap delay with a reverb (maybe even an IR) to treat the soften the slap and give it more depth - try letting the reverb run on the source dialogue and the slap delay, and also try verbing the slap delay only (this is where Aux sidechains help) and see what you like. The slap delay helps give that perceived feeling of the sound slapping walls (and it even kinda takes into account that believable amount of delay you might get from speaking into a mic and having it go through the signal chain to the speakers). Again, it's really just an experimentation thing, finding what hits your ear right, there is no perfect answer to this - so if this is the method you tried already, trying using a different set of plugins and/or different reverb IRs/parameters).
Also worth noting, if you're doing this treatment with a clean and naked dialogue track it's GOING TO sound very weird regardless of what you do. This is where BGz are critical in giving that bed of existence and 'space' for which the dialogue can interact with. Even dead quiet auditorium or opera/theater environments can contain a lot of sonic texture just with the roomtone layering alone.
Good luck!