Do children of gay and lesbian couples have a lower high-school graduation rate?

Question

I recently came across this 2013 study: High school graduation rates among children of same-sex households which looks at a large sample of Canadian families, and concludes:

Children living with gay and lesbian families in 2006 were about 65 % as likely to graduate compared to children living in opposite sex marriage families. Daughters of same-sex parents do considerably worse than sons.

This, the abstract admits, is in contradiction to almost all previous studies of same-sex parenting.

I read claims that the difference found in this study was not significant.

Is this study reliable?

Answer 1

Is this study reliable?

We often get questions on Skeptics.SE about one particular study, asking if it "proves" some controversial claim.

The short answer, without even looking at the study, is "No. One single study is not reliable. Studies are often wrong. Unfortunately, sometimes it is all the evidence we have."

If we have the luxury of more than one study of a phenomena, and they disagree, we can use a systematic review (or even better, a meta-analysis) to try to resolve the dilemma. More on that in a moment.

---

This particular study has received some criticism.

For example, Philip N. Cohen, [now] Professor of Sociology, wrote a blog article in 2013 criticising several aspects.

He claims the sample size is actually about 85 gay-father kids and 194 lesbian-mother kids. That should be enough to get statistically significant results for large effect sizes, but it is considerably smaller than the headline figure of "20% of the Canadian Census" might suggest.

He claims the sampling has selection bias by including 17-22 year olds that still live with their parents, while excluding those who moved out.

[Note: This means even if the results are "statistically significant", they are still biased.]

He claims the odds ratio presented is misleading:

Anyway, the paper does provide the marginal effects, which show that the children living with gay parents have graduated from high school at an adjusted predicted percentage 6 points lower than those living with married different-sex parents

---

Meanwhile, Mark Regnerus, [now] Professor of Sociology, wrote a 2020 article, Understanding how the social scientific study of same-sex parenting works defending his own 2012 paper that found differences in children of same-sex relationships. That paper caused some controversy in the social science community, with open letters attacking the research and defending it being signed by dozens or hundreds of scholars.

Regnerus argues that population studies are hard, and subject to coding errors, that the differences in outcomes are hidden in the choice of factors to control for and sampling strategies, and accuses fellow scientists of not being neutral in their choice of papers to criticise:

As a result, it is difficult to conduct solid social science research on such topics, when the world of scholarship on sexuality has tacitly ruled some conclusions more worthy of publication than others. Scientific neutrality is out. Political expedience is in.

So, we can see that individual papers on this subject have triggered a lot of criticism and defence, reinforcing that we should treat each paper with care.

---

But back to systematic reviews...

In 2022, a team published Family outcome disparities between sexual minority and heterosexual families: a systematic review and meta-analysis.

They looked at 34 studies (16 for the meta-analysis) for a large number of outcomes.

In general:

Most of the family outcomes are similar between sexual minority and heterosexual families, and sexual minority families have even better outcomes in some domains. Relevant social risk factors of poor family outcomes included stigma and discrimination, poor social support and marital status, etc.

However, the question here is about academic achievement, so let's look at what they found there.

We conducted a narrative synthesis of six studies on children’s educational outcomes. Four studies indicated that children from same-sex couples appear to have the higher rate of grade retention, lower graduation rate or worse educational attainment than children from different-sex couples. On the contrary, two studies reported that children in sexual minority parent families outperform children in heterosexual parent families on standardised test scores, high school graduation rates, college enrolment, and school/academic competence.

[Note that the original study in the question is one of the four.]

They did not conduct a meta-analysis on this data, so it isn't clear how strong each study is, but as you can see: The result is unclear:

Four studies found children of same-sex parents do worse at school. Two studies found they do better.

Answer 2

Unknown

Usually, an analysis of this type would be independently verifiable. The authors would summarize how the data was generated, how it was analyzed, and a skilled scientist could then reproduce the analysis (or could perform a different analysis to test a different hypothesis).

This is a core concept for science - Reproducibility.

In this case, the author provides very little methodology and omits virtually the entire analysis:

This file is not a public use data set. To use the data, a proposal is screened by the Social Sciences Research Council of Canada, an RCMP criminal check is conducted, and the researcher becomes a deemed employee of Statistics Canada subject to the penalties of the Statistics Act. Empirical work was conducted at the SFU Research Data Center, and all results were screened by Statistics Canada before release. Statistics Canada does not allow any unweighted observations or descriptives to be released, nor any maximums or minimums of weighted estimates, nor sample sizes for the weighted regressions.

It is possible that this analysis is correct. It is possible that this analysis is partially correct, but uses inappropriate controlling (for example, by regressing out or controlling for a causal factor which then confounds the analysis). It is possible that this analysis is completely invalid, the result of a trivial coding or statistical mistake. It is possible that this analysis is completely fraudulent.

With the information provided, it is unknown and unknowable if this study accurately analyzes the underlying data or if the analysis is appropriate.

Answer 3

What is there to be skeptical about with respect to statistical reliability?

The results are based on a 20% sample of the census of the entire country. That is by far a much greater sample size than is available for almost any other statistical study on almost any other topic.

Whatever the results are, they should be considered very statistically significant (i.e. the sample reliably represents the actual population).

What is there to be skeptical about with respect to 65% being a significant difference?

The results themselves show a 35% reduction in high school graduation. That children are more than 50% likelier to graduate if they aren't living with same-sex parents is a significant difference.

One could argue that say a ¼% reduction has statistical significance, but have little meaning when applied to reality. But this isn't ¼%, it is 65%, which is highly significant in all contexts.

Here are the relevant abstract and conclusion from the paper:

Abstract

Almost all studies of same-sex parenting have concluded there is “no difference” in a range of outcome measures for children who live in a household with same-sex parents compared to children living with married opposite-sex parents. Recently, some work based on the US census has suggested otherwise, but those studies have considerable drawbacks. Here, a 20 % sample of the 2006 Canada census is used to identify self-reported children living with same-sex parents, and to examine the association of household type with children’s high school graduation rates. This large random sample allows for control of parental marital status, distinguishes between gay and lesbian families, and is large enough to evaluate differences in gender between parents and children. Children living with gay and lesbian families in 2006 were about 65 % as likely to graduate compared to children living in opposite sex marriage families. Daughters of same-sex parents do considerably worse than sons.

…

5 Conclusion

A casual reading of the literature on child performance would suggest that no-difference in child outcomes exists between children in same-sex or opposite-sex households. Indeed, the unanimous opinion of so many studies would appear conclusive—as noted by Justice Walker. However, a closer inspection reveals that there are really fifty-plus “preliminary” studies, and no general conclusion about child performance differences is warranted based on the literature. The samples used in these studies are often biased in some way, and the sample sizes are often very small. The one study that did use a large random sample and address a reliable performance measure (Rosenfeld 2010), turned out to draw the wrong conclusion, did not compare gay versus lesbian homes, did not examine the gender mix of the household, and did not control for parental marital status. As a result, there is little hard evidence to support the general popular consensus of “no difference.”

I have argued that the 2006 Canada census—though not perfect—is able to address most of these issues, and the results on high school graduation rates suggest that children living in both gay and lesbian households struggle compared to children from opposite sex married households. In general, it appears that these children are only about 65 % as likely to graduate from high school compared to the control group—a difference that holds whether conditioned on controls or not. When the households are broken down by child gender it appears that daughters are struggling more than sons, and that daughters of gay parents have strikingly low graduation rates.

This paper confirms the findings of Allen et al. (2013), and taken together they cast doubt on the ubiquitous claim that no difference exists in child outcomes for children raised by same-sex parents compared to married opposite sex parents. That is, both the US census and Canada census show that children living with same-sex parents perform poorer in school when compared to children from married opposite sex families.

The question is: why? This study suggests further work is necessary to narrow down the source of this difference. This will require an exceptional data set that not only identifies sexual orientation of parents, but also has a retrospective or panel design to completely control for marital history. A better data set would also be able to test for the reasons behind any difference. An economist may be inclined to think that fathers and mothers are not perfect substitutes and that there must be some gains from a sexual division of labor in parenting. Others may suspect that children of same-sex parents are more likely to be harassed at school, and therefore, less likely to graduate. In any event, it is time to investigate the difference and reject the conventional wisdom of “no difference.”