6

The EPA and other regulatory bodies have long concluded there is a casual relationship between poor (ambient) air quality and death.

This news article challenges that claim:

researchers have finally published a study claiming to have debunked science the Environmental Protection Agency (EPA) has used to justify imposing costly regulations on U.S. industries.

The study led by veteran statistician Stan Young found “little evidence for association between air quality and acute deaths” in California between 2000 and 2012.

Is it true that this study 'Air quality environmental epidemiology studies are unreliable' by Stan Young debunks the EPA's claim that there a causal relationship between poor air quality and death?

8

This is effectively a partial answer. It does not seek to respond to the overarching question of whether or not there is a causal relationship between poor air quality and death. I'm not currently aware of any notable sources who have claimed that it does not. It only seeks to address the direct claim that Stan Young's paper "debunked" the existing scientific work the EPA had used to determine it. The answer to that is... not really. He's at best pointed out an indicator of potential concern that should be checked further.

This paper is probably technically correct in its assertions. It assumes bad faith on the part of fellow researchers (effectively claiming that they first acquire their data sets and then poke through them in order to find a particularly juicy conclusion, without including that in their calculations of statistical significance). It was not able to find publication for over a year. When it was published, it was published in a journal with a probable pro-corporate bias. This paper may well be making a valid point about statistical rigor, but "debunked" is a rather severe overstatement for the strength of the conclusions that can actually be reasonably drawn.

  • The Daily Caller is a Conservative News and Opinion site, reporting on politically charged science news. "Science News" of all sorts is chronically bad at actually knowing what papers are talking about, and "News and Opinion" sites tend to be a little more fast and loose than most, when reporting things that support their ideological position. This finding supports the conservative position. This does not inherently undermine the findings of the paper in and of itself, but it does mean that the article itself doesn't add much (if any) credibility. If you include the tendency towards both sensationalism and inaccuracy of science news in general, it means that the claims that initiate in the article may require particular scrutiny.

  • Regulatory Toxicology and Pharmacology is a monthly peer-reviewed scientific journal. The editor is Gio B. Gori. Wikipedia suggests that he's spend the past few decades working more or less directly for cigarette companies, and googling his name seems to support that. The journal is published by the International Society of Regulatory Toxicology and Pharmacology (http://www.isrtp.org/) which openly acknowledges that they have as one of their major objectives "identifying and engaging the policy makers and scientists who would be most concerned with developing, modifying, or applying those regulations, or affected by their development, modification, or application." - or, in short, lobbying. The Journal clearly describes itself as being intended to support policy decisions. Wikipedia suggests that the institute is financed in part by the tobacco, pharmaceutical, and chemical industries. Their own webpage merely states "Sponsors to be announced soon", and looks like it has done so for multiple years. None of this is proof of anything, but the most likely state of affairs given the available evidence is that the institute is largely a pro-corporate lobbying group that does not wish to make that obvious on their website. That in turn would imply rather heavily that the journal would be at least somewhat biased. It is peer-reviewed, and these conclusions are not nearly enough to suggest that it will knowingly publish things that are false or lack academic rigor, but there is a good chance that the choice of which papers to consider may be influenced by political slant.

  • The paper, on direct observation of the abstract https://www.sciencedirect.com/science/article/pii/S0273230017300673, appears to be selecting eight papers that were used in meta-analysis papers, looking at the available information, determining how many things could have been tested for with the given data, and then asserting that random chance would have produced a reasonable number of apparently statistically significant results, and thus that the meta-analysis results aren't as compelling as they might be. It was also mentioned (in the Daily Caller) that he had to shop it around for a few years before he got anyone to publish the thing - suggesting that no one was willing to publish it until he got to the industry-friendly ears at Regulatory Toxicology and Pharmacology.

So, short answer is that it looks like the paper is probably technically correct, and it may even have a valid point (saying that people are acting with more certainty than they really ought to have without a few more tests), but it's not anything like as strong as "debunked".

To clarify further, at least by my understanding of Young's abstract, he's essentially assuming bad faith on the part of the researchers of the papers he is studying. His thesis appears to be, in essence, there are enough possible questions that could be asked of the datasets in question that someone who was willing to take their time to pick and choose could easily find questions that would appear to be statistically significant, even if the input data was noise. He concludes from that that the papers in question, and all meta-analysis papers based off of them, are unreliable. It is not clear from the abstract how how chose his eight example papers, given that there are apparently over 100 papers on air quality available.

  • 4
    I'm fairly certain that all US courts would admit the sources of financial support of Dr Gori as evidence of potential bias. I'm not sure Skeptics should have a higher threshold. – Andrew Lazarus Aug 31 '18 at 23:49
  • 2
    @AndrewLazarus: Not higher. Different, because science has different goals to courts. – Oddthinking Sep 1 '18 at 5:58
  • 4
    (+1) Also heartland.org/about-us/who-we-are/s-stanley-young seems to suggest the author of that paper (S. StanleyYoung) is getting some support or has some kind of affiliation with the Heartland Institute. They are skeptical of evertything from the effects of smoking to the existence of global warming. – Fizz Sep 1 '18 at 9:35
  • 5
    @daniel It's true that I'm including analysis of the bias of the news site and the journal, but I think it's pertinent. Specifically, under normal circumstances, journals and news sites could be expected to add credence by filtering out unreliable/biased/non-pertinent papers. In this particular case, though, the probable biases of the publications undermine that benefit to a degree (though not entirely). I've tried to make it clear that my bias analysis is intended in that light. – Ben Barden Sep 4 '18 at 16:07
  • 4
    @daniel information can be true or false regardless of where its found, but the source of information says a great deal about its credibility, and credibility is one of the few things we have to go on when figuring out how true something is. That's why we don't just blindly reference Wikipedia and assume that it's correct. That's why scientific journals are given more credence than random websites. Provenance matters. – Ben Barden Sep 9 '18 at 19:20
1

The larger (heading) question: 'Is there a causal relationship between poor air quality and death?'
It is undebatable that some amount of pollution in the air will cause death, we all need air to breathe and this has happened in history, Most agree there should be regulation on air pollution to prevent these deaths. An example from the comments is from 1952 when thousands died from the great smog of London.

The smaller (body) question: 'Is there evidence for association between air quality and acute deaths in California between 2000 and 2012?'
No, there is no good solid evidence of deaths caused by this current level of pollution, there is no smoking gun, like with Mesothelioma for asbestos. Its possible someone will find solid evidence for this time period in the future, possibly right after I post this answer.

What about the Epidemiological studies? I will borrow from the previous answer here to show why they are not solid evidence. Those types of studies are strong enough to show the link of smoking and lung cancer (again for most people, some die hard smokers still think the studies are inconclusive), but comparing the smoking studies to the air pollution studies has a huge difference of the relative risk.

There is this study on PM pollutants in outside air with a relative risk factor of 1.1.
Compared to this study on smoking, which shows a relative risk at least 10 times larger than the previous study.
I was looking into particulate matter in particular with the last answer so if anyone else finds comparisons of epidemiological studies for smoking cigarettes vs air pollution in California between 2000 and 2012 with comparable relative risk factors please feel free to add them.

  • 3
    You seem to be saying "The evidence doesn't suggest that it's as bad as smoking, so it's not meaningful." I'm perplexed why the standard should be "as bad as smoking". There are a lot of things that aren't as bad as smoking, but are still bad. Also, you make the assertion "No, there is no good solid evidence of deaths..." and you don't say what evidence you base this assertion on. Was it a three-day dig through available research journals? A five minute google search? Off the top of your head? It's a pretty core assertion. It deserves at least some indication of where you got it. – Ben Barden Sep 11 '18 at 14:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .