Skeptics Stack Exchange is a question and answer site for scientific skepticism. It only takes a minute to sign up.
Sign up to join this community
This article claims there is a software, that can say if a person is homosexual, based only on their facial images. The other article calls it "pseudoscience" though:
critics say we’re revisiting pseudoscience
Is it true or fake? How did they measure 91%? If 7-11% of people are homosexual, simply by saying "straight" all the time the software would have 89%-93% rate of success.
Is it true or fake?
The research paper in question has passed peer review, suggesting that the methodology was deemed to be sound by other experts in the field. It is conceivable that the results were faked (as occasionally happens in research) but this is very unlikely given how easy it would be to reproduce and verify the findings. In fact, there's nothing stopping you from reproducing their work yourself.
Could the results be wrong, though? Well, the authors of the study themselves wrote the following:
Our findings could be wrong. In fact, despite evidence to the contrary, we hope that we are wrong. However, scientific findings can only be debunked by scientific data and replication, not by well-meaning lawyers and communication officers lacking scientific training.
in a response to critiques from HRC and GLAAS (source).
How did they measure 91%? If 7-11% of people are homosexual, simply by saying "straight" all the time the software would have 89%-93% rate of success.
As explained in the paper, the algorithm was presented with an equal number of photos of heterosexual and homosexual people. In other words, the a priori probability of the person in each image being homosexual was 50%, not the 7-11% of people you pass on the street.
Summary: This paper is a real scientific paper that has passed peer review. The real judge of whether their findings are true or not is whether they can be replicated. There simply has not been enough time since the study came out for a replication to be published. Additionally there are questions about what exactly their results mean. Artifical Neural Networks do not explain to the user why they made their predictions; they are a black box.
Artificial neural networks (ANNs) are a machine learning tool that can be trained to recognize patterns in data and make predictions based on those patterns. Their structure was inspired by the structure of natural neural networks, brains. Like a natural neural network, their decision making processes are a black box. ANNs make a prediction (whether the individual is gay or straight) based on input data (a set of photographs), but they do not explain their reasoning. The researchers speculate on what the ANN might have been picking up on, but they don't really know.
What facial features were employed by the algorithm to detect sexual orientation? The average faces most likely to belong to gay men (see Figure 1) were more feminine, while the faces most likely to belong to lesbians were more masculine. Typically, men have larger jaws, shorter noses, and smaller foreheads. Gay men, however, tended to have narrower jaws, longer noses, larger foreheads, and less facial hair. Conversely, lesbians tended to have more masculine faces (larger jaws and smaller foreheads) than heterosexual women.
The gender atypicality of gay faces extended beyond morphology. Lesbians tended to use less eye makeup, had darker hair, and wore less revealing clothes (note the higher neckline)—indicating less feminine grooming and style. Furthermore, although women tend to smile more in general, lesbians smiled less than their heterosexual counterparts.
Additionally, consistent with the association between baseball caps and masculinity in American culture, heterosexual men and lesbians tended to wear baseball caps (see the shadow on their foreheads in Figure 1; this was also confirmed by a manual inspection of individual images).
Maybe the ANN was mostly guided by facial features, and maybe by other cues like smiling or wearing baseball caps. The training set was taken from a dating website. I expect gay men and lesbians to choose photos that would be attractive to other gay men and lesbians. The ANN could be picking up on those differences. In response to these questions the authors argue that:
First, we tested our classifier on an external sample of Facebook photos. It achieved comparable accuracy as on the dating website sample, suggesting that the images from the dating website were not more revealing than Facebook profile pictures. ...
Finally, the deep neural network used here was specifically trained to focus on fixed facial features that cannot be easily altered, such as the shape of facial elements. This helped in reducing the risk of the classifier discovering some superficial and not face-related differences between facial images of gay and straight people used in this study.
Note: Although they try to eliminate any dependence on things other than fixed facial features, I am not sure if they succeeded. I am not deeply familiar with facial recognition algorithms or neural networks.
The ANN in the paper used a training set with half gay and half straight faces. The ANN would be presented with a single gay face and a single straight face, and asked which was which. When the faces were made using 5 photographs, it was able to correctly choose which one was gay 91% of the time. This is quite different from any real life application, where roughly 10% of the population is gay.
Speculation: The highest accuracy prediction, 91% accuracy, was achieved with 5 photos of the same person. However, there were only 219 gay men and 223 lesbians with 5 photos in their data set, compared to ~3500 each for 1 photo. This could be a weakness in their methods, but 200 subjects is still quite a few. It is possible that their methods overstate their accuracy. The predictions based on 4 photos had significantly more subjects and only slightly lower accuracy. If I arbitrarily decided to throw out the multiple photo accuracy, we are still left with an 81% accuracy for gay men.
If you are reading this in the future, this google scholar link shows all of the articles that cite the article in question. Hopefully in a couple of years, there will be quite a list of articles at that link. The articles that will be published will hopefully be a stronger form of review than peer review. I would expect that many of them repeat the conclusions of this article briefly and uncritically. The authors of these papers probably did not read this paper deeply enough to give a real critique of its methods. If one of them discusses it at length, those authors probably understood the methods and conclusions well enough to be properly skeptical. If that paper also is on a very similar topic, or builds on this work, I would be very interested in that authors opinion of the paper in the question.
