Did Stack Overflow cut the number of negative comments nearly in half between the beginning of the fourth quarter of 2019 and January 21, 2020?

Question

I read on https://stackoverflow.blog/2020/01/21/scripting-the-future-of-stack-2020-plans-vision (mirror) (emphasis is mine, to match the question):

In the fourth quarter of 2019, we created a large task force made up of passionate community advocates from across the company to propose solutions to this core problem. The team’s mission is to improve our feedback loop and working relationship with our community. We added a member of our Community team to our leadership team and restructured the organization to invest in Product leadership to build Community-centric features. We are also forming a moderator council, which will include a group of users with diverse experience levels and backgrounds who can help guide our processes. We’re making hard choices and treating no assumptions as sacred in considering ways to evolve the community.

[...]

Community engagement and inclusion is a top priority for Stack Overflow in 2020. Already, the team has established and released a plan for improving communication and empowering our users internally. We are experimenting to improve in areas we know need work: encouraging more question asking, reduce user to user friction, expanding inclusion, and creating a more integrated experience between Stack Overflow and other technical sites on Stack Exchange. The results of all of this work is being shared publicly through our new Loop series on the Stack Overflow blog and by subscribing to our Stack Overflow newsletter and podcast. We have seen some encouraging results: more people are asking questions, we cut the number of negative comments nearly in half, and December was our best month ever for new user signups!

Did Stack Overflow cut the number of negative comments nearly in half between the beginning of the fourth quarter of 2019 and January 21, 2020 (day when the blog post was published)? I don't see any source in the linked webpage supporting that claim.

Answer 1

I think it's a little ambiguous if this is on topic for Skeptics, since it relies on knowledge only Stack Overflow employees have.

So I posted a long answer on Meta Stack Overflow (we're only measuring on Stack Overflow) that goes into some detail about how we measure negative comments and the recent trends we have seen that lay behind Prashanth's comment in the blog post.

The TL;DR is:

• There is an algorithm that we trained on user flags of negative comments, and then refined until it had accuracy comparable (actually slightly better, as of the latest revision) to actual people based on moderator feedback.
• We plotted that algorithm's classifications of new comments negativity over time, and have used it to investigates trends. It is that plot that suggests a 50% reduction in negative comments.

The graph from the MSO Post gives a 30,000 ft view:

Because it is not possible construct a split test for this, what we strictly have is a lot of suggestive correlations. In particular, I wouldn't pin reductions on any single action we have taken nor would I say every action we have taken was successful. However, in aggregate and for practical purposes I'm professionally comfortable with "negative comments, as a proportion of all new comments, have been about cut in 1/2 over the end of 2019".