Bearing in mind taller men are on average more successful, it's probably fair to say that evolution would by now have biased women to prefer taller men.
It's not necessary to examine all the individual reasons why tall men do better, or cite specific successful tall men - the evolutionary record tells us that on average humans have been getting taller. So I must just cite one individual - 'Lucy', a hominid skeleton dated to be more than 3 million years old - who was barely a metre tall. I know it's commonly held that Turkana Boy might have grown to 6' 1" if he'd reached adulthood, but he died at 5' 3", and later studies say he'd only have been likely to grow another 2"-5".
There will obviously have been many points in the history of hominid evolution when it was a positive disadvantage to be large/tall (when food was scarce, for example). So we're not looking at a slow, steady increase here. But the genes for height have been consistently preserved, probably largely through sexual selection by females. To the extent that "fitter" females get more choice of mate, they tend to choose taller men. And thus have taller children - who have more chance of surviving because their parents are better than average providers.
Since we now have money, women can use a man's wealth as an even more accurate indicator of his fitness as a mate, but height remains a powerful influence.