15

We obtain a great deal of data about trends in sensitive and personal topics through anonymous surveys.

For example, in a science writing class I'm taking, the professor showed us results from an anonymous survey taken by undergraduate students. The survey showed the prevalence of various forms of academic dishonesty. (I didn't copy down the reference, but I can get it if it's important to be specific here.)

The assumption here is that anonymity makes participants more likely to respond honestly. This sounds plausible - it's the old "strangers on a train sharing secrets" story. But is it actually a valid assumption?

  • How honest are people's responses to anonymous surveys?
  • What sorts of studies are done to examine the honesty of these surveys?
  • It really depends on the survey: in some cases it can be more accurate than a non-anonymous, and vice-versa. Can you be more specific about the survey? – Sklivvz May 6 '11 at 10:49
  • I updated the question to change "accuracy" to "honesty" since I think that might have caused some confusion about my question. Specifically, I wasn't trying to compare anonymous and non-anonymous surveys. – Mark Eichenlaub May 6 '11 at 10:56
  • 2
    How would you find that out? With a (anonymous) survey? ... Survey Honesty – Oliver_C May 6 '11 at 11:18
  • @Oliver_C One example: In some cases, it might be possible to infer the percentage of people who ought to respond "yes" to a given question, then compare that to the percentage who actually respond "yes". For example, a survey could have a question asking if the responder has ever had a certain STD. Then, by examining medical records, it might be inferred that a certain percentage of people (within a range) have actually had the STD. Then the percentage inferred from medical records and the percentage from the survey could be compared. – Mark Eichenlaub May 6 '11 at 11:29
  • My question is about whether that or other techniques to assess honesty have been implemented, and what the conclusion are. – Mark Eichenlaub May 6 '11 at 11:30
6
+75

A majority of the literature on the net seems to support that non-anonymous and anonymous-surveys give very similar results.

The paper "A comparison of confidential versus anonymous survey procedures: Effects on reporting of drug use and related attitudes and beliefs in a national study of students." studies anonymous vs. confidential surveys in a specific case (drug usage reported in 8th and 10th graders), and concluded that there was a very modest or possibly no effect on the data.

From the abstract:

This study presents a comparison of reporting of drug use and related attitudes and beliefs by national samples of 8th and 10th grade students under two different modes of administration conditions: confidential and anonymous. Data come from the Monitoring the Future project for the year 1998. The results show that there were clearly no differences between the conditions in 10th graders' reports of drug use and related attitudes and beliefs. With 8th graders, the results show, at most, only a very modest mode of administration effect and quite possibly no effect at all.

The same conclusion is drawn in more countries in the paper "Adolescent Substance Abuse in Mexico, Puerto Rico and the United States: Effect of Anonymous versus Confidential Survey Formats".

Similarly another study "The Effect of Anonymous Vs. Nonanonymous Rating Conditions on Patient Satisfaction and Motivation Ratings in a Population of Substance Abuse Patients" concluded that

Anonymity had either no effect on ratings or accounted for <1% of the variance.

In a study about collecting data regarding bullying, "The Efficacy of Non-Anonymous Measures of Bullying" concluded that

The findings supported the hypotheses that the respondents did not differ in their report of the incidence of either bullying or victimization, regardless of whether they were required to identify themselves by writing down their names on the questionnaire forms.

Another study, "Differences between ‘talking about' and ‘admitting' sensitive behaviour in anonymous and non-anonymous web-based interviews" which (among other things) compared difference between two questionnaire modes- anonymous web-based forms or dektop-based video interviews concluded that:

Nevertheless, the expected differences between the interview modes were not observed.

On the other hand:

A study called "High risk behaviour and fertility desires among heterosexual HIV-positive patients with a serodiscordant partner – two challenging issues"(PDF) says that

Non-anonymous data collection on condom use may underestimate high risk behaviour

However, there are several caveats to these papers, some issues which may need exploring:

  1. The surveys sometimes seems to be carried out through a trusted authority.
  2. The surveys seem to target some specific portion of the population. The validity for general polls like opinion polls etc. may be different.
  3. Non-anonymous may mean several things, perhaps each with it's own effect on results.
  • Interesting link with a surprising result, thank you. – Mark Eichenlaub May 12 '11 at 11:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .