Thanks for the answers, everyone! I think I found a source that takes the cake, though. HERE is a full-text meta-analysis of many, many studies about caffeine, including its effects on sleep. See the section entitled "Effects of caffeine on sleep."
To answer my own question, I'd have to say that the answer is both "yes" and "no."
- Yes: From an analytical/research/measurement perspective, it appears that there have been a considerable number of studies showing that caffeine does, in fact, affect sleep quality (duration, efficiency, latency, etc.).
- No: From what I would consider a practical perspective, two bolded studies below show that despite being able to measure caffeine's effects on sleep... these measurements do not seem to correlate with mood or performance the next day. Thus, I'm left to ask if I should/need to care about such measured effects in daily life.
To dig into the linked paper above, here are the main points:
- Caffeine increases sleep latency (the time required to fall asleep from being awake ([LINK}(http://en.wikipedia.org/wiki/Sleep_onset_latency)). 
- Caffeine reduces sleep duration. 
- Caffeine administered in the early morning can affect sleep. 
- Individuals in studies have reported that they can consume caﬀeine bedtime with no adverse effects.  
- High caffeine consumers appear less report sleep disturbance infrequent consumers. 
- Tolerance develops to the effects of caffeine sleep. 
- There are no withdrawal effects on sleep once caffeine consumption is ceased. 
- It is unclear whether there is any correlation between measured sleep disturbance and changes in mood and performance on the following day. 
- It is easy to demonstrate that caffeine consumed late at night produces effects on sleep; it is difficult to show the same for high consumption levels of caffeine in general.
- An inverse relationship has been found between daily caffeine consumption and sleep duration, but no significant relationship has been found between caffeine consumption and sleep satisfaction. 
- Several studies are listed which found little evidence to correlate caffeine consumption and and sleep.   
The three conclusions drawn from the meta-analysis are:
- Over 3 mg/kg of caffeine in a single beverage consumed in the late evening will both increase sleep latency and reduce duration. Smaller amounts vary considerably between individuals, and there is evidence to support that high consuming individuals are more resistant to sleep side effects of caffeine.
- Impacts on next-day behavior after caffeine-induced sleep effects is not known. Neither are effects on long term health (assuming long term health effects related to sleep alterations, but it's not specified).
- High levels of caffeine have not been strongly related to sleep parameters, but this may be due to the scheduling of consumption as to not adversely affect sleep (i.e. not drinking caffeine later in the day.
I bolded a couple of studies, as their methods indicate that patients were asked to self-report. I see potential sources for error and bias in methods like these. I happen to drink a lot of coffee, and could see having a vested interest in making myself look better, and thus skewing responses toward the "little/no adverse effects" end of the spectrum.
I'll close by saying that further studies would be helpful to 1) eliminate self-reporting bias (use objective measurements rather than subjective questions) and 2) examine more handles than just "mood" and "performance" to establish/dis-establish the connection between sleep disturbance and any practical effects on the life/experience of the individual.
 Zwyghuizen-Doorenbos, A., Roehrs, T.A., Lipschutz, L., Timms, V., Roth, T., 1990. Eﬀects of caﬀeine on alertness. Psychopharmacology 100, 36–39.
 Hicks, R.A., Hicks, G.J., Reyes, J.R., Cheers, Y., 1983. Daily caﬀeine use and the sleep of college students. Bulletin of the Psychonomic Society 21, 24–25.
 Landolt, H.P., Werth, E., Borbely, A.A., Dijk, D.J., 1995. Caﬀeine intake (200mg) in the morning aﬀects human sleep and EEG power spectra at night. Brain Research 675, 67–74.
 Colton, T., Gosselin, R.E., Smith, R.P., 1967. The tolerance of coﬀee drinkers to caﬀeine. Clinical Pharmacology and Therapeutics 9, 31–39.
 Levy, M., Zylber-Katz, E., 1983. Caﬀeine metabolism and coﬀee attributed sleep disturbances. Clinical Pharmacology and Therapeutics 33, 770–775.
 Snyder, S.H., Sklar, P., 1984. Behavioral and molecular actions of caﬀeine: focus on adenosine. Journal of Psychiatric Research 18, 91–106.
 Searle, G.F., 1994. The eﬀect of dietary caﬀeine manipulation on blood caﬀeine, sleep and disturbed behaviour. Journal of Intellectual Disability Research 38, 383–391.
 Smith, A.P., Maben, A., Brockman, P., 1993b. The eﬀects of caﬀeine and evening meals on sleep and performance, mood and cardiovascular functioning the following day. Journal of Psychopharmacology 7, 203–206.
 Lee, K.A., 1992. Self-reported sleep disturbances in employed women. Sleep15, 493–498.
 Greenwood, K.M., Rich, W.J., James, J.E., 1995. Sleep hygiene practices and sleep duration in rotating-shift shiftworkers. Work and Stress 9, 262–271.
 Bliwise, N.G., 1992. Factors related to sleepquality in healthy elderly women. Psychology and Aging 7, 83–88.