Jukin’ the Stats

There’s an old saying, “Liars figure and figures lie.” When I think of that old saying, I think of our Uniform Crime Report (UCR). I think that way because this report  is a compilation of only the crimes that are reported to police and police, in turn, report to the FBI.  I hope you immediately see the potential problems inherent in such a system. Unfortunately, it is a system of counting we have come to accept. Changing it will be difficult. And that’s why this is discussed in my new book.

Therefore, a decrease in “crime” could mean that victims were too scared to report the crime (or, today, worried such a contact with police would get them deported). Or that crime victims resigned  themselves to the fact that reporting wouldn’t do any good anyway.

But there is a second, almost unknown, crime reporting system in play, the National Crime Victimization Survey (NCVS), it is a census used by academics in tandem with the FBI’s Uniform Crime Reports to get a sharper focus on crime trends – in this survey, more than 135,000 Americans self-report their own victimizations. There are problems with this methodology too; primarily it is not compiled by city by city. To highlight the problem we have with the standard UCR system is that the  NCVS survey tends to find that less than 50% of violent crimes are reported to police.

Overall, there is a big problem with using the UCR to tell us whether crime is actually going up or down. In my book, I talk about the problem in the numbers and the things we count. Here’s a piece from my chapter on evaluation (one of the seven necessary steps police need to take to improve):

 “In a recent interview [April, 2009], Bill Moyers talked with David Simon, creator of the popular HBO series ‘The Wire.’ Simon, a former journalist and police reporter, mentioned the propensity of corporations, governments, and their agencies to ‘juke the stats’—that is, to alter data so that it appeared that you were doing good, even when you weren’t!

“The following dialogue is from an episode in which ‘Prez’ Pryzbylewski, a cop turned teacher, has an interchange with the school principal about the ‘numbers:’

 Principal: So for the time being, all teachers will devote class time to teaching language arts sample questions. Now, if you turn to page eleven,  please, I have some things I want to go over with you.

 Prez: I don’t get it—all this so we score higher on the state tests? If we’re teaching the kids the test questions, what is it assessing in them?

 Teacher: Nothing—it assesses us. The test scores go up, they can say the schools are improving. The scores stay down, they can’t.

 Prez: Juking the stats.

 Teacher: Excuse me?

 Prez: Making robberies into larcenies, making rapes disappear. You juke the stats, and majors become colonels. I’ve been here before.

 Teacher: Wherever you go, there you are.

In the interview with Moyers, Simon further reflects:

You show me anything that depicts institutional progress in America, school test scores, crime stats, arrest reports, arrest stats, anything that a politician can run on, anything that somebody can get a promotion on. And as soon as you invent that statistical category, fifty people in that institution will be at work trying to figure out a way to make it look as if progress is actually occurring when actually no progress is. And this comes down to Wall Street. I mean, our entire economic structure fell behind the idea that these mortgage-based securities were actually valuable. And they had absolutely no value. They were toxic. And yet, they were being traded and being hurled about, because somebody could make some short-term profit. In the same way that a police commissioner or a deputy commissioner can get promoted, and a major can become a colonel, and an assistant school superintendent can become a school superintendent, if they make it look like the kids are learning, and that they’re solving crime. And that was a front-row seat for me as a reporter. Getting to figure out how the crime stats actually didn’t represent anything, once they got done with them.

“I spent a good deal of my time as police chief trying to teach members of the news media, as well as elected officials and community members, what exactly crime was and what it meant when FBI crime statistics were released. And that a numerical increase or decrease did not necessarily mean that crime was actually up or down and the difference between reported and actual crime. I tried not to ‘juke stats’…

“I have found that a good way to visualize and understand the difference between actual and reported crime and the difference between the UCR reported crime data and the actual victimization survey conducted by the NCVS [National Crime Victimization Study] system is through the following chart:

 

“It is true that citizens who live in a community where the police are respected and responsive are more willing to report crimes than those who live in communities where police are not respected. The result can be that a police department that is viewed as ineffective by the community will have a low incidence of crime reporting and therefore a ‘low’ crime rate, while a police department thought to be highly effective and responsive would experience a ‘high’ reporting rate—and therefore, an apparently high rate of crime.”

What the triangle above demonstrates is a simple fact about crime – counting it is difficult – yet even in the best system there will be the lower layer of unknown crimes. In terms of gross numbers, arrests are a poor determination of criminality in a community, police reported crime is one way to count crime but it, too, has its problems. So far, the best way to count crime is for independent academic researchers is do a “victimization study.” So next time you hear the police are reporting a decrease in crime (or the press saying that crime is increasing), simply cock an eye and say, “oh yeah?” and remember what David Simon had Prez Pryzbylewski say in the HBO series:  “Jukin’ the stats! Making robberies into larcenies, making rapes disappear. You juke the stats, and major become colonels. I’ve been here before.”