Dr. Cynthia Lum is an Associate Professor at George Mason University and the director of the Center for Evidence-Based Crime Policy. She was formerly a Baltimore City police officer and detective, and an instructional professor at the U.S. State Department International Law Enforcement Academy. She and Rutgers University professor Leslie Kennedy recently published an edited volume, Evidence-based Counter-terrorism Policy.
Mike German, a fellow at the Brennan Center for Justice at NYU Law School, interviewed Cynthia Lum on October 15, 2014.
Q: Today I’m with Cynthia Lum, who is an Associate Professor at George Mason University and one of the founders of the Center for Evidence-Based Crime Policy. Why don’t we start with the basics: what is evidence-based policing?
LUM: Evidence-based policing is the use of science in policing; so it’s advocating for police to pay more attention to the scientific research about all areas of policing so crime control, prevention but also internal affairs, police relationships with the community and other topic areas. So evidence-based policing is about not only using that research, but it’s also about applying scientific processes such as evaluation, crime analysis, and survey – XL survey methodology to look at relationships with the community across different areas of policing. It’s not a panacea to policing- it’s meant to be an additional source of information that police can use to improve their decision-making about crime and relationships with the public.
Q: Okay. And the police are used to handling evidence in pursuing the criminal prosecution so you would think that they would be open to this sort of a scientific approach.
LUM: I think evidence-based policing, it’s often confused about crime evidence in policing. Evidence-based policing specifically refers to research evidence and knowledge as opposed to evidence that’s collected at a crime scene. Police are very familiar with collecting evidence, with developing cases using evidence. They’re less familiar with the use of research evidence and science in their daily applications. So yes, police are very aware of the use of evidence in criminal investigations and the evidence collected at a crime scene.
Q: And I should say you were a police officer yourself.
LUM: I was…
Q: And a detective?
LUM: Yes. So we’re very familiar with collecting evidence and handling it properly and using it to make decisions about investigations. That’s different than what evidence-based policing is talking about. It’s talking about the use of research evidence, both for tactical and strategic decision-making. And this is something that police are less familiar with. They’re becoming much more familiar with the research evidence now about policing. But many of the practices that many police agencies in the United States – and around the world – use would not necessarily be considered evidence-based or strongly evidence-based. They’re much more procedures-based: traditional, reactive-oriented, given that that is the nature of policing as it’s developed in the United States…
Q: And how receptive have police departments been to this approach?
LUM: I think it varies. It depends on the police leaders and commanders that are in charge of a police organization. I’ve encountered many who are extremely receptive, who have thought very hard about research evidence and have been involved in research projects. For example, we have a number of individuals in the Evidence-Based Policing Hall of Fame now where we reward police chiefs and other officers- you don’t have to be at a chief level- for their consistency and commitment to these researching practices.
Q: And I imagine as budgets tighten at state and local police departments that it becomes almost a requirement to start basing your activities on actual scientific evidence, that it’s effective?
LUM: I think in theory that’s what we would hope, but it’s not how it works in practice. To go back to your previous question, there are some agencies that still are resistant to innovation more generally. Evidence-based approaches require you to re-think entirely the way you’re doing deployment of your patrol and investigations. Right now, police are very individual-oriented, they’re very reactive and procedures-based and they’re very general in their tactics and strategies. The evidence tends to point to place-based strategies, proactive approaches to policing, and also tailored strategies. This is a sea change in the way police are doing business and any kind of major innovation is going to result in some resistance. And it can vary; it’s not just the large agencies that are more progressive, for example. Sometimes larger agencies aren’t more progressive. Sometimes the smaller agencies are much more willing to try out something new, or to work with a researcher, or to use an innovation.
Q: Great. Since the 9/11 terrorist attacks, obviously we’ve devoted a tremendous amount of resources to counter-terrorism, at the Federal level, at the State level and local level. We’ve also given law enforcement agents an awful lot of authority- new authority that impacts the privacy and civil liberties of normal Americans. What evidence do we have that the counter-terrorism methods that these agencies are using are actually effective?
LUM: We have very little evidence about counter-terrorism measures- either in the United States or abroad. A few years back, I had done a systematic review for the Campbell Collaboration where we looked at all the research about counter-terrorism interventions and terrorism more generally. There were thousands and thousands of pieces of research. We could only literally find seven studies that had actually evaluated using at least moderately rigorous methods, the impact or the effectiveness of a counter-terrorism intervention. So, as you can see, the amount of research that’s publicly available and that we know about is very, very small in terms of understanding the effectiveness of Homeland Security, counter-terrorism, even military operations overseas.
Q: And the government actually funds a tremendous amount of studies of terrorism.
LUM: It does.
Q: Why aren’t these studies more evidence-based?
LUM: I think it’s always easier – and this is not just in the terrorism and counter-terrorism field – but it’s in policing, education, social science – it’s much easier to fund the study of a phenomenon – like terrorism, or a disease or kids that are working in school or something like that, as opposed to actually evaluating an intervention that might be put in place to change something. Sometimes people are concerned about the interventions themselves – they don’t want to test it. We see this in the medical field often. You know, will somebody get sick if they become a test subject for a medication that’s trying to vaccinate against Ebola, for example? So there’s some fear I think, about evaluating interventions and there’s also risk for organizations about what you might find when you evaluate an intervention. Often we find that interventions don’t work. I think tactics and strategies don’t actually lead to impacts that were thought to occur; DARE is a great example, or these boot camps, where millions of dollars were spent on these programs. And after a long period of time we’ve started to do some evaluations and found out some of these programs are not effective. Sometimes they backfire, in that they cause more crime to occur. The findings of science are always risky for organizations in some ways. And it takes a lot of money and time and expertise to do an evaluation. So that might be one reason why we see more studies on terrorism than we do on counter-terrorism. Let me also add that much of the data that we would need, the access that we would need to carry out a highly rigorous evaluation, is not there. So it’s very hard to gain secret clearance. If you gain secret clearance to do evaluations, then you can’t publish your results often.
Q: Which, for an academic, is very important?
LUM: I think academics understand the balance between national security and publication but there’s a whole area of disagreement, I think, between groups – public interest groups, academics, the community, the news, and the government.
Q: And for a potential reformer in Congress, who wants that report because it shows a policy isn’t working, when the Agency wants it to work…
LUM: That’s right.
Q: …that secrecy can actually harm reform…
LUM: A good example is how police were five decades ago – four decades ago – in the 1970s – I think they were much more suspicious of police scientists coming in and doing an evaluation. Their concerns were wide-ranging, but one concern was, if we release this evaluation of this intervention, maybe people who are committing crime will figure out what we’re doing in terms of stopping it- and that might give away the farm, you know? And I think police have become much more sophisticated about this. They realize that because of monetary concerns, because they want to know what can reduce crime, they’re trying their hardest to figure out what’s going to actually be effective and so they’re much more willing to listen and to think about science as one option, to help them make decisions.
Q: And some counter-terrorism measures themselves are dangerous. We’re now in a period where we’re re-entering a military engagement in Iraq bombing in Syria. Is there any evidence about the effectiveness of that type of military intervention?
LUM: So in terms of counter-terrorism measures, going back to the systematic review, there’s one type of counter-terrorism measure that we’re pretty sure is secure- we think that it works. And that is, metal detectors at airports have been shown to prevent hijackings. But other things, we’re not as certain about. A good example is military operations. The research done on military operations – and there’s very little of this – but the research done seems to indicate a short-term increase in suicide bombings during a military operation to deal with counter-terrorism, or perhaps a high-intensity police operation that seems more militaristic to deal with terrorism issues. An increase in suicide bombings tends to occur, but then that drops off again to the levels that had existed before.
Q: Interesting.
LUM: So there is this immediate impact of military operations.
Q: …and with the caveat that there’s very little…
LUM: …there’s very little research. And it’s mostly research that looks at trends over time, which is basically is the only thing we can do. And all that information – the data related to those terrorism incidents or suicide bombings – much of it is stuff that we have to collect from newspapers, from public knowledge of the suicide bombings. The databases that are collected on terrorism incidents- I believe the START Center at the University of Maryland has an excellent database – but a lot of that is not collected by the Government. It’s collected by private or public entities.
Q: Which is interesting, I mean you would think that the Government would have a system so that everybody could work with the same data rather than relying on private entities that might have motivations or political leanings that weigh, in what they’re counting as terrorism or not counting as terrorism…
LUM: They might have this – I’m not aware. And it might be information that’s not available to the public.
Q: You mean they might… the government might actually have…
LUM: That’s right. The reason why I say they might is, there are some publicly available documents that indicate for example, data that’s being collected by the airports on things that are happening – and these are not terrorism incidents. These are just everyday crime incidents – and so we know these types of things exist, and that they are collected in some way. Whether there’s a terrorism database, I’m unsure.
Q: Right. Why is it important for the Government to do evidence-based research about its counter-terrorism policies?
LUM: Wow, that’s a great question. Many, many different reasons. First, we live in a democratic society. That means the government is held accountable, not only financially but also philosophically and politically to the things that it chooses to do. So one of the most important aspects of evidence-based crime policy more generally is the sense that there’s a scientific, more objective accountability mechanism in place. The law is very important, citizen input is extremely important [to] democracy; but also facts that are generated objectively and that are why science is essential in governance in democracies. So that’s one reason. Another reason connected to it is simply we need to know what is effective; I think governments – especially our government – want to do something that helps.
Q: Right.
LUM: They want crime to go down; they want people to be healthier. There is, I think, a general interest in trying to be reform-minded, and evidence-based approaches are meant to help with that process. Third is just purely a financial reason. One thing I tell police chiefs is that if they’re spending thousands and thousands, if not tens and hundreds of thousands of dollars, on something – a technology, a deployment option, but it’s actually not giving them any returns. This is a very serious matter for the chiefs in terms of their budgets, right? A good example is investigative techniques. Clearance rates in the United States have remained relatively stable for many, many years. In terms of homicide, clearance rates have declined. So, the clearance has gotten worse. For decades now, for a few decades, this despite the fact that we’ve had improvements in investigative techniques; technologies surrounding investigations; forensic techniques, even just case folder management and information technologies to deal with that. And yet we still see this kind of very stable clearance rate.
Q: And that’s as homicide rates have generally gone down…
LUM: Yes exactly.
Q: So the rate’s gone down but the clearance rate has gone down as well?
LUM: Well the clearance rates have been going down even during the increases in homicide. So you’ve had, in the 1980s and early 1990s, you did have still an increase in some crimes and you still have the stability of the clearance rate – or the decline of clearance rates in terms of homicide. I believe Charles Wellford at the University of Maryland is doing some work on this area. But just to make a point…
Q: Sure.
LUM: …that when police or Department of Homeland Security or schools are doing something to try to create a better life or a better education or more safety, they themselves want to know how their money is being spent. At least in theory that’s how it should work.
Q: Right… great! Let me follow up on that question a little bit: with the homicide clearance rates going down and so much money and resources devoted to intelligence, where state and local law enforcement agencies are now opening fusion centers and participating in FBI joint terrorism task force[s] and being involved in counter-terrorism – which is well-funded. Is it perhaps appropriate to look at the violence that’s happening in our American communities and perhaps take some of those resources to help solve those problems?
LUM: Wow. Well, I think that… that is a difficult question, Mike. First of all, crime in the United States is a pervasive problem: violence – we still have people getting shot and killed every single day, robbed on the street, or just getting something stolen from them. Fraud committed on their credit cards or on their bank accounts or retirements; so there’s a wide range of criminal activity that continues to go on in addition to terrorism. Those things should always be a priority, just as things like education and health in the United States should be a priority. The problem with terrorism – and counter-terrorism – is that one, people are afraid so they’re afraid that this is going to happen. I remember September 11th very well; it’s a very vivid part of my life. So I think there’s an interest in stopping it or trying to mitigate it somehow. The problem is that it’s a very rare event and in public policy we have to make decisions about how much to spend on certain types of events compared to others. Homicide is a good example of this. Homicide’s actually a very rare event in the United States – relative to other countries it’s not rare. But generally speaking, you’re more likely to be victimized as with another type of crime than homicide.
Q: Right.
LUM: And yet the amount of resources that go into homicide investigation are very, very high, precisely because people – it’s something that people are afraid of.
Q: Sure. And it’s a serious crime.
LUM: It’s a very serious crime. And so, when you think about it, there are other kinds of types of crime that can also lead to homicide: shootings, robberies with weapons, things like that. Domestic violence is a great example of something that can lead to homicide. So I guess it’s a balance between spending funds on those things that are just everyday problems that we really need to deal with and those that generate a great deal of fear among the American public.
Q: And it would seem that’s where scientific research helps, both to understand the threat and the nature of it and the rate of it, so that you can reduce the fear where possible, and also make sure that both your crime policy and your counter-terrorism policy are actually effective.
LUM: Absolutely! And research is only one part of this conversation. But the point is that we need to be part of the conversation. Research hasn’t always been a part of the conversation. We haven’t always been at the table in discussions of police deployment options or counter-terrorism measures. And that in and of itself is an important voice that kind of mitigates some of these concerns that can just pop up without any facts behind them whatsoever. And not just that, but the spending of money on interventions that don’t work.
Q: And that would seem to be an easy choice to make. If it doesn’t work, let’s do something else.
LUM: That’s right; or at least… sometimes it’s hard to change a police agency. They’re used to doing something that may not be as effective. But the question then becomes, “How can we adjust the deployment so that we can move towards something that has a better chance of being effective in reducing and preventing crime?” And also, “How can we do tactics and strategies that are not going to negatively impact our citizenry?” This is a serious issue in the United States. We’re not just concerned about reducing crime, we’re also concerned about protecting people’s rights and dignity while we’re trying to create safety, so this is important.
Q: And, I think one area that gets a lot of attention because many Americans go through it is the Transportation Security Agency – the TSA.
LUM: Oh, yes.
Q: And I know that you did a study of TSA procedures. What can you tell us about that?
LUM: Well, at George Mason – a group of us just completed actually, a big project that looked at assessing TSA’s – one of its programs called the playbook. This program, much of it is SSI – Secure Sensitive Information. The actual name of the program is not, which is why I can say it now. But just broadly speaking, it’s a security program in the airports where they’re trying to create an extra layer of security through prevention and situational crime prevention and deterrence measures to try to reduce crime, and also the potential for terrorism and hijacking at airports. And we weren’t asked to evaluate the program. We were only asked to assess it in terms of how it’s been carried out and what impact it has had on people who work at airports and those TSA officers who are implementing the program. So what we found was that much of their choices in tactics and strategies have been shown by other types of criminal justice research to possibly have good effects in terms of preventing crime and disorder. Of course, whether or not they can prevent terrorism is a whole other question. But the way they’re implementing some of these interventions, that is the question, you know? Does this approach work? And that’s an open question that is subject to evaluation, which would be the next step, with regards to the TSA. What I appreciate about the TSA and DHS in this regard is that they were open enough to have a third party – an academic third party come in and take a look at what they were doing and give them feedback about the program, whether it’s close to things that we know are evidence-based.
Q: And in your book, you actually had to redact an awful lot of information.
LUM: Oh, yes!
Q: But even though it’s redacted, do you think that the Agency was responsive to the findings?
LUM: I think that’s to be seen. It’s hard to know – and this goes for the police agencies we work with, too. It’s hard to know what the impact of research is on practice. This is actually a whole area of study called Translational Criminology that I’m also involved in, where we’re trying to look at how research is used, and whether it’s used, how does it impact practice over time, et cetera. And this is a really hard one to determine with TSA primarily because we’re not privy to a lot of decision-making that’s going on in TSA or DHS. So we don’t know. We can make some educated guesses when we see things that are happening at TSA but the connection between that and our research is really never known. It’s often hard for an academic, it’s hard to reconcile because you want to see some result from something that you’ve done. But with social science research more generally, it’s really about, you know, consistently keeping up with doing evaluations and keeping that conversation going with policymakers at the Federal and state level and local levels about the importance of specific types of research that could help them in their practices.
Q: And that’s an interesting thing because when I think of the intelligence community and the intelligence agencies – their job is to provide policymakers with the best available information so that they can implement successful policies. And you would think that they would be doing this kind of research. You know, they have $70 billion to work with! And even at the state and local [levels] now, we have these intelligence fusion centers where they could be understanding what the research is and pushing that out to the police departments. What do we know about the effectiveness of fusion centers, for example?
LUM: Not much. Fusion centers, joint terrorism task force and also interstate task forces – I want to call them interagency task forces – those types of approaches have not been evaluated to any great extent at all. The vast majority of evaluations in policing are actually within agency patrol evaluations looking at different types of deployment options and their results in terms of crime control. So, for example, we don’t have any knowledge about whether information sharing leads to marginal increases in crime control, for example; we take a best guess and think information sharing’s got to be helpful – we think about 9/11, for example, and one of the key points that were made from the 9/11 Commission is the lack of information sharing. And so we tend to think – or hypothesize – that it might improve crime prevention, terrorism, counter-terrorism, et cetera. How much it improves – or in what way it improves – how well we can create an information sharing system to create that improvement, what are the consequences of that improvement with privacy of citizens…
Q: How do we ensure the information we’re sharing is accurate?
LUM: Right… or safe?
Q: Right…
LUM: …from just hacking and so, all of all of these questions are really important when you’re creating policy around information-sharing but we just don’t know that, so we just don’t have answers to that. So most of the time, agencies are making decisions based on what’s become known as best practices, or consensus-based decision-making processes, where leaders will get together and they’ll come up with some consensus about what they think might be the best way to move forward. This is very common in public policy. The problem with that is that consensus-based approaches are biased towards the individuals that are in that room and they might not have all the knowledge that might give them an alternative approach to dealing with the particular problem. So going back to your question about fusion centers and JTTFs, and inter-agency task forces, these things have been viewed very positively and there’s a lot of money spent in them. However, we’re not clear on how they can operate to improve the outcomes that we’re trying to seek. And that is a serious issue when it comes to the millions and billions of dollars that are being spent on them.
Q: And for all the good reasons you gave about proper governance as well.
LUM: Proper governance is important and often with fusion centers. Since 9/11, fusion centers were very, very relevant at the beginning, right after 9/11, but since then, they’ve had to adapt to, not only the fact that – and I say this knocking on wood – but we haven’t really had a major problem with terrorism since 9/11. There have been problems, but because of that, they have a lot of resources available to them to deal with other types of problems. So now we have fusion centers that are trying to use their resources in ways to facilitate local policing and you know, how they’re doing that and what they’re doing and whether or not those things are useful, I think there are important questions to ask…
Q: Great! If somebody wanted to learn more about these issues, what would you suggest they read?
LUM: I think there are a number of different things; for policing, we have a resource on our website that gives a number of different resources on evidence-based policing – what is it? What is the evidence? We’ve developed a matrix where we’ve pulled together all the evidence on policing and security into a single place where you can see the hundred and twenty-five studies that we know about that are at least moderately rigorous, that evaluate policing interventions; so that’s a good place to start. There’s a number of centers for evidence-based X… you know? Corrections, sentencing, evidence-based policy that I would suggest people go to for those types of resources – just to get a sense of what is evidence-based crime policy? I would also suggest if you’re interested in counter-terrorism in particular that you look at the evidence-based counter-terrorism book that Les Kennedy and I edited. There are a number of scholars in that book who contribute on different areas of counter-terrorism policy that might be of interest to policymakers. And I would also suggest looking at the START Center’s website as well; they are at the forefront of generating research on terrorism and also touch upon some counter-terrorism issues so they’d be a good resource.
Q: Terrific! Is there anything I forgot to ask you or that you’d want to make sure we get?
LUM: I guess in closing I would say that evidence-based crime policy is a difficult venture for any organization to undertake; why? It often challenges traditional approaches; science does that more generally. It challenges people’s hunches and opinions and their personal…
Q: And traditions…
LUM: And traditions! And personal beliefs about things, and that’s very uncomfortable for, not only for large organizations and their chiefs, but also for people – citizens – who are at the receiving end of some of these interventions who might not understand why a policy is being changed – why is a police officer being taken from my community and being put over in that community, you know? And a police chief should take the time – just as somebody in DHS or TSA – should take the time to explain to the community why are we doing this? What is the reason behind changing our approach? What does science say? I think public safety agencies can play an important role in educating the public about the research that’s available. And that all of those things – in addition to just paying more attention to science – are really important in this evidence-based crime policy arena. It’s important for financial reasons and it’s important for governance reasons.
Q: Great! Well, I appreciate your work. Thanks very much for being with me today.
LUM: Sure … anytime.