Every time I use it, it always seems to give exactly the speed my ISP is advertising +-1%, whereas other services (that actually do something, but happen to report the speed at which they do so) have only half or less the speed. Also it doesn't seem to be affected by running torrents. Speedtest gives some info here about cutting off the top 10% and the bottom 30% of speeds, then averaging. Does that really result in doubling the actual speed?
Besides personal expeirience, I have also heard various stories of how either speedtest.net is paid by the ISP to report the speed you paid for or that the ISP prioritizes speedtest data.