Google published a paper on drive failure based on analysis of their use of over one hundred thousand drives.
Failure Trends in a Large Disk Drive Population, Eduardo Pinheiro, Wolf-Dietrich Weber, and Luiz André Barroso , Proceedings of the "5th USENIX Conference on File and Storage Technologies".
One aspect they looked at was whether disk utilization affected failure rate:
Overall, we expected to notice a very strong and consistent correlation between high utilization and higher failure rates. However our results appear to paint a more complex picture. First, only very young and very old age groups appear to show the expected behavior. After the first year, the AFR of high utilization drives is at most moderately higher than that of low utilization drives. The three-year group in fact appears to have the opposite of the expected behavior, with low utilization drives having slightly higher failure rates than high utilization ones.
One possible explanation for this behavior is the survival of the fittest theory. It is possible that the failure modes that are associated with higher utilization are more prominent early in the drive's lifetime. If that is the case, the drives that survive the infant mortality phase are the least susceptible to that failure mode, and result in a population that is more robust with respect to variations in utilization levels.
Another possible explanation is that previous observations of high correlation between utilization and failures has been based on extrapolations from manufacturers' accelerated life experiments.
[...]
Taken as a whole, our data indicate a much weaker correlation between utilization levels and failures than previous work has suggested.
So, if your fear is that the extra work being down to defrag is likely to kill your disk-drive earlier, it seems that it might cause a borderline drive to fail earlier. However, this isn't a strong effect.
Another effect might be observed: If a drive has a lot of read operations per defrag, then the time saved in seeking the right block will add up, and effectively reduce the overall utilization of the the system. If there is frequent defragging, but no other use, it will obviously increase the overall utilization.