We still don’t know how many people the police kill in the U.S. annually. But we’re getting closer.
In the past two months, the president and his attorney general have said they need better data on the number of people killed by police — a number that nobody knows and that no government agency can agree on. On Tuesday, the Bureau of Justice Statistics released a report that at first glance looks like it fills the gap. It found that more than a quarter of killings by police are not included in either of two federal databases.1 The true number of annual police killings, according to the report, is likely around 930 — about twice that of each of the two other U.S. government counts. It’s even higher — about 1,240 — if you assume that local law enforcement agencies that don’t report any killings have killed people at the same rate as agencies that do.2
It’d be tidy if that were the final word — but it isn’t. On Wednesday, researchers who specialize in estimating unreported violent deaths issued acritique of the report. Based on their team’s experience in Syria, Guatemalaand other conflict zones, they argued that the true number of killings is probably even higher than the new, higher range of estimates by BJS.
This is a math puzzle with real consequences. Solving it would get researchers closer to understanding how many lives have been lost — and how many victims we’re not yet counting.
Patrick Ball, co-author of the critique and executive director of the Human Rights Data Analysis Group, said it’s not the BJS’s fault that it underestimated the number. That it estimated it at all was an important step, he said. Government agencies don’t all audit their own data or undertake the difficult task of matching records that often have incomplete or missing information. So they should get some credit for that.
But to make their estimates, the report’s authors had to make some simplifying assumptions, ones that Ball said likely led to an undercount. “For sure, the estimates are too low,” Ball said in a video chat Wednesday.
Lance Couzens, a co-author of the report and a research statistician at RTI, acknowledged this and other limitations, as did the report. Couzens said in an email that the number of people the report estimates were killed by police “is probably artificially deflated.” The results, he wrote, “should not be interpreted as providing an accurate estimate.”
Here’s what the BJS tried to do and why it might not have worked well enough. It merged two databases of killings by police — its own Arrest-Related Deaths database and the Supplementary Homicide Reportsmaintained by the Federal Bureau of Investigation. It estimated the number of cases that were repeated in both databases. This step was important for two reasons. First, it wanted to make sure it didn’t count any cases twice.3Second, the number of duplicate cases can be used to estimate their mirror image: the cases that were in neither database — the gaps in the data.
The trouble was that they had to make some assumptions to estimate the gaps. A big assumption was that the two data sets were independent — that a victim being in one database doesn’t make him or her any more or less likely to be in the other. (We’ll get into more detail on why that matters later.) But the researchers had no way to know whether that was true. Ball said it probably wasn’t. That’s been his experience when merging databases of killings. It’s a problem that can be minimized with three or more databases. But the BJS report’s authors only used two. That, Ball said, led them to arrive at too small a number. And there’s no good way to know by how much they missed.
To understand what might have gone wrong, let’s take a close look at the method the researchers used to estimate the gaps in the police killings data. The method is called capture-recapture, and it is used to count groups where traditional methods fail or are too difficult.
Here’s an example of how that would work to count wildlife, a scenario in which capture-recapture was originally used: Researchers capture a random sample of animals from a closed population — say, fish in a lake. They tag them in some way that doesn’t affect their behavior or health. Then they release the animals into the population. At some later date, the researchers collect a different random sample of animals from the same population and calculate what percentage of them are tagged. Researchers can then use two numbers — the number of animals they initially tagged and the percentage of recaptured animals that were tagged — to estimate the overall population.
The human world is messier. Researchers generally don’t tag people, and even if they did, people don’t distribute themselves randomly and don’t stay in a confined area. So researchers use messier techniques, and more data, to apply the principles of the capture-recapture method to humanity.
One way to do that is to compare more than one database. Suppose that the name, age and hometown of each person killed by police in the U.S. last year were written on separate pieces of paper and then each piece of paper was folded up and placed in the middle of a small ball. Then all the balls, representing all the people killed by police, were placed in a large barrel. Suppose we didn’t have the time and resources to count all the balls. So we try another way. First, we send an FBI analyst to the barrel. She collects 10 of the balls at random, removes the pieces of paper and makes a list with each of the victim’s information. Then she puts the balls back in the barrel and stirs them in. Now a BJS analyst approaches. He removes 10 balls randomly and takes down the information of all those victims before returning the balls.
No comments:
Post a Comment