What you should likely be looking at is the efficiency of the speakers, the loudness per Watt of input. Speakers with a higher power rating might be less efficient, so for equal desired end loudness they might reach the limits of the amplifier sooner.
Larger specced speakers also tend to project sound better (at the cost of volume in close vicinity) but filling a larger space requires a larger amp, too.
That's for loudness.
What's often overlooked is that it is much more harmful for the speakers when you overdrive a solid-state amp than it would be to overdrive the speakers with an amp that is not itself at its limits. This particular danger is particularly imminent when using multi-way speakers with separate tweeters: the tweeters are designed to take a minor fraction of the total output power, but a clipping amp will give them a significant portion of its maximum power.
A small two-way or even one-way (broadband) speaker will be much less likely to blow its tweeters on a clipping solid-state amp than a somewhat larger three- or four-way speaker would.
So as a rule of thumb and somewhat contraintuitively it makes sense to have your amp at least as powerful as your speakers because the consequences of the amp reaching its limits tend to be more severe than the speaker maxing out on its own, and you get more audible advance warning (distortion) with the latter.
For instrument amps (particularly valve amps usually intended to be overdriven) and matched speakers the rules are different as they and their speaker enclosures are laid out quite differently.
