TL;DR - we don't know if the study is corroborated.
I will continue to look into this topic to edit and improve the answer. It may be too early to conclude if the results of the 2018 paper are corroborated by other sources.
RE: Methodology, the paper covers it well (emphasis added).
We constructed an original database of all confirmed uses of deadly force by police officers in the United States in 2014 and 2015.
We began by drawing on data gathered by Killed By Police, a
nongovernmental entity that tracks police killings reported in the
news and updates its data set each day. We chose this source as a
base because the site links each killing with a news story that we
could locate online. In order to ensure that the accuracy of the data,
we cross-checked it with two other websites that collect data on
police killings (lethaldb.silk.co
and
FatalEncounters.org).
All three
data sets have been used by other scholars studying police killings
(Lott and Moody 2016; Nicholson-Crotty, Nicholson-Crotty, and
Fernandez 2017).
The Killed By Police data contained the victim’s name, race, age,
date of birth, gender, date and time of killing, city, state, and a
news account of the killing. We supplemented these data with
other variables available in news accounts and other police killing
data sets, including local population demographics, cause of death,
geographic location of the killing, type of offense, presence of a
nonpolice witness, and whether there was a warrant for the suspect.
...We also coded for a range of
variables about the officers whenever possible, such as officer race
and gender, years of police service, and type of officer. Because of
missing data, we had to thoroughly analyze every news story that
we could locate on each killing.
First, let's look at the credibility of the websites.
Looking at Killed By Police, each entry in the database indeed includes a news article (and usually, 1+ news articles). Thus, it appears credible. Something interesting to note is that it has been fact-checked by FiveThirtyEight. From FiveThirtyEight:
We randomly sampled 146 incidents (10 percent) from the news links posted to Killed By Police. All the posts linked to established outlets, although in some cases a new url for the article had to be found because the news site had restructured its links.
Looking at lethaldb.silk.co, we find the message "It’s time to say goodbye" and a notice that it has been shut down. The Internet Archive didn't have screenshots of older versions of the webpage, so I cannot verify its authenticity.
FatalEncounters.org also appears credible, as it includes a news story with each entry. Note that for all sites I checked some entries, but not all.
Next, let's see if other scholars have actually used the same data set.
The pdf for Lott and Moody 2016 can be accessed here and is published in the Social Science Research Network. I couldn't find the impact factor for this journal. From what I can tell, it is similar to ArXiv, so I would take documents here with a grain of salt. I could see from the appendix that data was collected from killedbypolice.net and fatalencounters.org, but not from the other source.
Nicholson-Crotty, Nicholson-Crotty, and
Fernandez 2017 can be accessed here. It is published in the Public Administration Review, the same journal as is the paper in question. In 2017, the impact factor was 4.591. Reading the paper confirms it also draws data from KilledByPolice and FatalEncounters (but, again, not from the third source).
Thus, some scholars did actually use data from 2 of the sources as the paper in question.
It may be too early to conclude if the conclusion is consistent with other research on this topic. The paper was published in 2018, and as the authors say
However, to our knowledge, no study has directly assessed the
racial composition of officer killings of suspects.
To address your questions:
Is this paper "bullet proof"?
No paper is truly bullet proof.
Has it been criticized (e.g. for methodology)?
I have not yet found criticisms of this paper. However, it may be too early to tell. By checking the three sources the paper used for its data set, two (KilledByPolice and FatalEncounters) appear to be credible. The third (lethaldb.silk.co) could not be fact-checked due to lack of an online presence (both current and historical). The paper mentions the three websites as having been used by other scholars. This is a true statement for KilledByPolice and FatalEncounters. This is an unverified (possibly false) statement for lethaldb.silk.co.
Is the conclusion of this paper consistent with other research on this topic?
It may be too early to answer this question as the paper was published in June 2018 and has been cited 2 times. A 2004 paper found undergraduate students were more likely "to shoot Black targets but not Whites."
