One federal report says violent crime went up in 2015. Another says it didn’t. What’s going on?

Last month, FBI data came out suggesting that America’s violent crime rate rose by 3 percent from 2014 to 2015, with the murder rate going up by more than 11 percent. It was an alarming finding, causing a lot of talk about whether the one-year increase is a reversal to the decades-long fall in crime and violence in the US.

This week, the Bureau of Justice Statistics released the results of the National Crime Victimization Survey (NCVS), also detailing crime rates in 2015 — only it found that violent crime victimization had remained relatively flat and potentially decreased:

The survey generally found statistically insignificant changes — mostly decreases — in violent crimes such as robbery, assault, and domestic violence. It did find a significant uptick in reports of rape and sexual assault, but the sample size was so small that it’s hard to say how valid the finding is.

So, uh, what’s going on here? How are these two reports so different?

The first thing you should know is that these reports are measuring two different things. The FBI is reporting crime detected by and reported to local and state police, which is then logged and reported to the FBI. The NCVS is reporting the results of a survey of more than 160,000 Americans 12 and older in which people report if they’ve been victimized by a crime and what crimes they’ve been victimized by.

One notable difference caused by the varying methodologies: The FBI report includes murder, while NCVS does not, since dead people can’t report victimization.

Obviously, this means NCVS is missing some crime in America — and it’s a particularly important part of the 2015 story, since the FBI report suggested that the murder rate rose by nearly four times as much as the violent crime rate overall.

But the FBI report always misses a lot of crimes, too, since it only picks up crimes that are detected by and reported to the police. There’s a lot of crime out there that cops never hear about. Case in point: The NCVS typically finds higher crime rates than the FBI’s report.

Which one of these reports is right? We just don’t know. As criminal justice expert John Pfaff explained, NCVS is widely considered more reliable. But both reports have enough gaps to prevent any hard conclusions in either direction. “This does not refute the contradictory findings in the 2015 [FBI report] that all index violent crimes had risen, but it complicates the story,” Pfaff wrote.

So we don’t really know if violent crime went up in 2015. That leaves us with a very unsatisfying conclusion: To see if the decades-long crime drop has started to reverse, we’ll probably have to wait for a few more years of data from both the FBI and NCVS to see the long-term crime trends instead of focusing on just one year of data.

As you might have guessed from this discussion, the US does a pretty pathetic job of collecting crime data. Not only are we just now getting data from 2015, but we also don’t even know if that data is reliable.

This is a big problem. As Richard Rosenfeld, a criminologist at the University of Missouri St. Louis, previously told me:

Our crime monitoring system — if that’s what one wants to call it — is based on the Uniform Crime Reports from the FBI. It is woefully inadequate, and it need not be. I’ve been calling for a long, long time for a Uniform Crime Reports program to release monthly crime statistics in a very timely way.

So we should be seeing crime data, at least for the large cities in the United States, one or two months after the month the data was collected — just as we do for unemployment; just as we do for inflation; just as we do when influenza breaks out, and the CDC is able to give us weekly case counts.

This has policy implications: If the murder rate is going up, it would be helpful for lawmakers and police to know about an increase much sooner — preferably as it’s happening — and be able to judge whether the trends are local, statewide, or national to find the right solutions. Yet it took until late 2016 to conclude that we still don’t really know what’s going on.

Crime data isn’t the sexiest or most exciting issue in the world. And collecting more of it more quickly would cost more money — conducting surveys and gathering reports from police departments isn’t exactly cheap. But when it could help save lives, it needs to be taken much more seriously.