My first thought was that this was certainly a myth, but it seems like I was quite wrong - this appears to be true (or at the very least, it's corroborated/perpetuated by a lot of sources).
From Frigidaire (page five):
Please always wait 3 minutes when turning unit off then on again, and when changing from cool to fan and back to cool. This prevents compressor from overheating & possible circuit breaker tripping.
From Amana (page 17):
If you turn off the air conditioner, wait at least 3 minutes before turning it back on. This prevents the air conditioner from blowing a fuse or tripping a circuit breaker.
And page 18:
If the air conditioner is off and then turned on while set to a COOL setting, or if it is turned from a fan setting to a COOL setting, it may take approximately 3 minutes for the compressor to start and cooling to begin.
If you turn off the air conditioner, wait at least three minutes before turning it back on. This helps to avoid the air conditioner blowing a fuse or tripping a circuit breaker.
From Haier (page nine):
Also when you shut off the air conditioner wait for 3 minutes before turning it on again. This is the compressor cycle time.
And from Topline HVAC, an HVAC company out of Chicago (which has one of the more irritating websites I've come across in a while, do check it out, including the home page):
When turning your air conditioner off, try to wait 3 to 5 minutes to allow the refrigerant pressure to equalize before turning your air conditioner back on. It gets very hard on your compressor, when it tries to start under high pressure, if you do not wait for your system pressures to equalize.
There are also about a million other results from Google. It's entirely possible this is all a super-safety-precaution, like a warning that peanut butter contains peanuts, and you might never experience any problems at all. But it's easy to be better safe than sorry here and let the compressor units stabilize.