121

This claim is being made in some parts of the media:

The 9th Circuit Court has been overturned 80% of the time.

Similar versions abound. Here is another one

Why? Because the “reversal rate” of the 9th Circuit is hovering at a solid 80 percent.

What does that mean? It means that this court is filled to the brim with individuals who have no regard for the constitution and who look to further an agenda that suits their personal politics — also known as “activist judges.”

This is in the context of their ruling against the Trump administration in relation to his executive order banning immigrants from 7 countries.

Is this true?

  • 116
    It might be worth noting that the Supreme Court doesn't tend to review cases that have no chance of being overturned. I.e., there's a selection bias in effect on the cases they review. – Ben Hocking Feb 10 '17 at 12:59
  • 2
    @BenHocking That's not right. They tend to decide cases where there is jurisdictional conflict, i.e. the 9th versus any other district has a different legal interpretation – user37696 Feb 10 '17 at 15:23
  • 4
    @user37696 here is a quantitative study of that: scholarship.shu.edu/cgi/… – DavePhD Feb 10 '17 at 17:31
  • 3
    @user37696, that sounds to me like potato/potahto. Jurisdictional conflict implies more likely to be overturned, no? Here is a more complete discussion of factors influencing whether the Supreme Court will hear a case: litigation.findlaw.com/legal-system/… – Ben Hocking Feb 11 '17 at 15:24
  • 2
    Facts have a liberal bias. – Jesse C. Slicer Feb 16 '17 at 16:32
248

Summary

It depends on your interpretation of the claim.

  • Were 80% of all 9th Circuit Court decisions overturned? NO, not even close.

  • Were 80% of the small minority of 9th Circuit Court decisions that made it to review by the Supreme Court overturned? Yes.


Only 0.12% of cases ruled on by the 9th circuit were overturned. 99.85% of cases were not heard by the Supreme Court. 0.03% of cases were confirmed by the Supreme Court.

In the remaining 99.88% of cases, the Supreme Court either wasn't brought in, chose not to take the case or approved of the 9th circuit's rulings.

This is from the data found by DavePhD. The total number of cases decided by the 9th circuit court of appeals was 114,199. Of that number, 107 were reversed, 33 were vacated, and 35 affirmed by the Supreme Court.

A total of 0.15% of 9th circuit cases were heard by the Supreme Court

Of those cases, 80% were overturned (reversed or vacated). This is the "Reversal Rate" and is the number they are using. The quote omits the fact that the "Reversal Rate", as understood by lawyers, does not include cases that weren't reviewed by the Supreme Court.

Cases referred to the Supreme Court are not a random sample of cases decided. They are selected by one of the parties to the case (usually the loser) because that party believes that there is an error of law in the decision and are willing to spend a great deal of money to have that perceived error verified. As such, cases referred to the Supreme Court are preferentially the ones most likely to contain a reviewable error of law and therefore be overturned.

Nevertheless, with data from Brythan (see their excellent answer), the Supreme Court sees approximately 7,000-8,000 petitions for writs of certiorari each term, only 80 of which they agree to take. Assuming the larger number for simplicity, that means 1% of cases requested for review are reviewed. Given that court reviewed 175 cases in the given time frame for the given court, we can extrapolate an approximate 17,500 cases were requested for review within those parameters.

This would give an approximate breakdown of 84.7% of cases weren't even considered by the Supreme Court, 15.1% of cases were declined by the Supreme Court, 0.12% of cases were overturned, and 0.03% of cases were confirmed.

Therefore, the biggest deciding factor from a purely statistical point of view, is "Will the Supreme Court hear the case"? 99% odds they won't, even with a petition. If you hit that 1% odds, then you have 80% odds of the case being overturned. Of course, there's actually an 84.7% chance you won't even seek the Supreme Court. But these numbers assume a randomly selected case, or that the decisions in a future case will be randomly made according to the past-performance statistics.

The case in question isn't going to be decided by statistics, but by judges making non-random decisions. The chances of the Supreme Court taking the case, and in fact whether they would overturn the decision of the lower court, cannot be determined by the past performance of all cases. The particulars of the case have a significant impact on the outcome, and neither the original articles, nor the statistics given here, take the particulars into account.

In short, they've taken a number that applies to a very small sliver of data and claimed it applies to all the data.

They've built an argument around that small sliver in order to make a claim that applies to the entirety of the 9th circuit. Sometimes they acknowledge that the data only applies to a subset, but they fail to clarify just how minuscule that subset is.

This is a common fallacy called the Statistics of Small Numbers fallacy or the Hasty Generalization fallacy, and also shows Selection Bias.

  • Comments are not for extended discussion; this conversation has been moved to chat. – Sklivvz Feb 11 '17 at 11:52
  • 5
    Just deleted a large debate that should have happened in the chat room. Most were bickering about the correct interpretation of words. In an attempt to stop the debate, I added a summary. Still got a problem? Take it to chat. – Oddthinking Feb 15 '17 at 12:46
74

According to the American Bar Association article Supreme Court Reversal Rates: Evaluating the Federal Courts of Appeals:

The reversal rates in Figure 2 range between 55% and 84%. Interestingly, this comparison of reversal rates reveals that the Federal Circuit has the highest reversal rate at about 83.33%, and the Ninth Circuit has the second highest reversal rate at 80%. The Seventh Circuit has the lowest reversal rate at 55.26%. The median reversal rate is 68.29%.

The American Bar Association is defining "reversal rate" as (decisions reversed or vacated by the Supreme Court)/(decisions reversed, vacated, or affirmed by the Supreme Court).

In the time period of the bar association study (1999-2008), 114,199 cases were decided by the 9th circuit and 107 were reversed 33 vacated and 35 affirmed by the Supreme Court.

80% reversal rate does not mean that 80% of all the 9th circuit's decisions are reversed. For only about 1% of decisions is review by the Supreme Court requested by one of the parties, and of that 1% only a small fraction are heard by the Supreme Court.

For more recent information:

Percent reversed for the 9th circuit:

2015 80%

2014 63%

2013 92%

2012 86%

2011 71%

2010 79%

2009 73%

  • 6
    This doesn't answer the question title "Are 80% of Ninth Circuit judgements overturned by the Supreme Court?", nor the first quote "The 9th Circuit Court has been overturned 80% of the time." – EnergyNumbers Feb 10 '17 at 15:16
  • 24
    @DavePhD well, quite - so please can you address the first cited claim "the 9th Circuit Court has been overturned 80% of the time". All you've addressed is how often they've been overturned when an appeal has been heard. The original claim is practising lying through half-truths; and one of the things we try to do here, is expose it when we find it; unfortunately, as it stands, your answer instead reinforces it. – EnergyNumbers Feb 10 '17 at 15:23
  • 6
    @Oddthinking The term "reversal rate" is a standard term and the American Bar Association is a neutral source. I think quoting the ABA plus giving the raw numbers is the most neutral, correct answer possible. – DavePhD Feb 10 '17 at 15:53
  • 18
    I think a perfect answer would be "No. The 9th circuit has been overturned in 0.12% of cases. It has been affirmed in 0.03% of cases. The remaining 99.85% of cases the court's findings were left untouched." – felipa Feb 10 '17 at 16:08
  • 9
    With the changes that have been made, this is clearly the most helpful answer that has been posted. It both states the context of the claim (that it's for cases heard by the Supreme Court), and it links to an article putting the 80% into context with other court circuits. After all, while it's true that some people will misinterpret the 80% number, thoughtful people want a way to compare one circuit with another, when the cases end up before the Supreme Court. Good answer! – TheGerm Feb 13 '17 at 20:06
30

The original question title asked:

Are 80% of Ninth Circuit judgements overturned by the Supreme Court?

No. Asked like that, the rate is between .1% and .2%.

The quoted claim is

The 9th Circuit Court has been overturned 80% of the time.

Which would be better stated as: of the cases from the 9th Circuit that have been reviewed by the Supreme Court, 80% are overturned. This is misleading in that many people will hear the first claim. The wording is overly ambiguous.

As @DavePhD posted the "overturn" rate statistic of 80% is correct. The problem is that not all cases are reviewed. So there might be a 114,199 cases, 175 are reviewed, and 140 are overturned. 140 is 80% of 175, but it is far less than 1% of the overall cases. That rate is between .1% and .2%.

USCourts.gov says:

In fact, the Court accepts 100-150 of the more than 7,000 cases that it is asked to review each year.

Obviously 150/7500 is only 2%. 80% of that would only be 1.6%. And of course, not every case is brought to the Supreme Court for Review. Numbers just for illustration -- the actual rate may differ somewhat.

The Supreme Court says:

The Court receives approximately 7,000-8,000 petitions for a writ of certiorari each Term. The Court grants and hears oral argument in about 80 cases.

It's not clear why the difference in numbers between the two sources. Regardless, the point is that the court does not hear most of the cases where review is requested, much less most of the cases issued by the circuits.

Note that in the context of the travel pause stay, it's not as misleading as it first seems. While only a small percentage of the overall judgments are overturned, a high percentage of those requiring review are overturned. This will almost certainly require review, as there is a conflict in the district courts of different circuits. A court in Seattle issued the stay while a court in Boston ruled the actions constitutional. The only way to resolve conflicts in different circuits is for the national court to rule. The Supreme Court is the national court. The term is circuit split.

The locution used in the quotes is skipping a step. In a random case, it's a big step. In this particular case, not quite as big. But 80% is still a high estimate at this point in the process.

As a side note, one of the reasons why the 9th Circuit has a high overturn rate is that it is too large for en banc review to proceed easily. So more litigants try to get a Supreme Court review without the step of en banc review. The 9th might have better statistics if it overturned some of its three judge panels itself, like the other circuits do. There have been some recent reforms that may help with that going forward, but that's unlikely to affect this particular case.

  • The Supreme Court itself says 80 out of 7000-8000 supremecourt.gov/faq.aspx – DavePhD Feb 10 '17 at 18:26
  • 1
    Perhaps the difference between "about 80" and "100-150" are cases where the court grants certiorari but in the same breath (that is, without argument or opinion) remands for reconsideration "in light of" a recently decided case. – Henning Makholm Feb 10 '17 at 18:54
  • 3
    "Note that in the context of the travel pause stay, it's not as misleading as it first seems." - Except it's not really talking about how likely this decision is to be overturned, it's passing moral judgement on the judges of the ninth circuit. – Random832 Feb 11 '17 at 20:38
  • 2
    "80% of reviewed cases" is a bit misleading. In addition to upholding or overturning a ruling by a circuit court, there are some other things the Supreme Court can do. They can decline to rule on the matter (essentially, saying "we shouldn't have accepted this for review"). Or, they can return the case to the circuit court for reconsideration in light of specific facts (usually functionally equivalent to overturning it, but not counted as such; occasionally a court will say "no, we got it right the first time"). I think the "80%" is 80% of cases where the court issued a conclusive ruling. – Mark Feb 11 '17 at 21:57
  • Is 80% a high estimate? Given that we know which circuits are involved in the split, I think a rather good estimate could be found using the conditional probability, specifically "The probability that a law is found to be constitutional on review, after the 9th Circuit previously held it was unconstitutional and the District of Massachusetts held it was constitutional", or even better, a slight variation dealing with probabilities conditioned on temporary injunctions. – Ben Voigt Feb 14 '17 at 1:11
6

Yes. The cited claims are correct but the simplified/paraphrased post title is incorrect.

Both cited articles do qualify their respective "80% overturned" statements.

The first notes(with my emphasis):

If Trump brings makes his plea to the Supreme Court — with Justice Neil Gorsuch onboard — the “rogue” 9th Circuit decision will be overturned easily.

The second, quoting National Review, states:

The Ninth Circuit’s best showing in recent years was October Term 2009, with a 60 percent reversal rate in the 15 cases on which certiorari was granted.

I agree that someone not reading beyond the headlines might come to the mistaken conclusion that 80% of all of the 9th Circuit's decisions have been overturned, but we do not do that here, right?

Some of the information contained in this post requires additional references. Please edit to add citations to reliable sources that support the assertions made here. Unsourced material may be disputed or deleted.

  • Welcome to Skeptics! Please provide some references to support your claims. – Larian LeQuella Feb 10 '17 at 23:43
  • 1
    Yes, explaining how something is misleading is something we do here. – Oddthinking Feb 11 '17 at 5:31
  • The only sentence on this entire page of any value is this one: "Both cited articles do qualify their respective "80% overturned" statements." – Fattie Feb 11 '17 at 14:36
  • Both linked articles exactly misrepresent the data as is presented in the question. What you quote as qualification doesn't make the article(s) an ounce less misleading. I certainly couldn't have interpreted the numbers if not for the explanation in the good answers here. – Martin Feb 11 '17 at 21:47
  • If Trump brings makes his plea to the Supreme Court - this implies, to my reading, that it is Trump's decision whether the court hears the case. It is the court's. Otherwise he is one of those 7-8000. – Someone Somewhere Feb 12 '17 at 9:31
0

The number is completely misleading.

In many cases, the loser just goes away without complaining. If the loser tries to ask the supreme court, most cases are just rejected without anyone looking at them seriously. The rest, one judge looks at the case, and must of the time that judge says "the 9th Circuit Court was fine". There is a tiny minority, 0.15%, where that one judge says "I think this should be overturned, I present this to all the nine (or eight) judges".

Of those 0.15% of cases where one judge has already said it should be overturned, the full set of judges agrees for 0.12% of all cases, which is 80% of the cases that he presented. That is in no way negative for the 9th Circuit. It is very positive for the judge who reviewed the cases and managed to pick cases where the complete Supreme Court agreed with him in 80% of the cases (0.12% of all cases).

And it's not 0.12% of all decisions that are overturned, it's 0.12% of all decisions where the loser tried to involve the Supreme court. So the percentage of all decisions is a lot lower.

If Trump had the power to force the Supreme Court to review a case (which he hasn't), based on statistics the chance would be 0.12% that the 9th Circuit will be overturned, the same percentage as for all cases with a sore loser.

Some of the information contained in this post requires additional references. Please edit to add citations to reliable sources that support the assertions made here. Unsourced material may be disputed or deleted.

You must log in to answer this question.

protected by Sklivvz Feb 11 '17 at 11:54

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged .