Episode 89: Graeme Blair
graeme blair
Graeme Blair is an Associate Professor of Political Science at UCLA.
Date: March 14, 2023
A transcript of this episode is available here.
Episode Details:
In this episode, we discuss Prof. Blair's work on community policing:
“Community Policing Does Not Build Citizen Trust in Police or Reduce Crime in the Global South” by Graeme Blair, Jeremy M. Weinstein, Fotini Christia, Eric Arias, Emile Badran, Robert A. Blair, Ali Cheema, Thiemo Fetzer, Guy Grossman, Dotan Haim, Rebecca Hanson, Ali Hasanain, Ben Kachero, Dorothy Kronick, Benjamin Morse, Robert Muggah, Matthew Nanes, Tara Slough, Nico Ravanilla, Jacob N. Shapiro, Barbara Silva, Pedro C. L. Souza, Lily Tsai, and Anna Wilke.
OTHER RESEARCH WE DISCUSS IN THIS EPISODE:
“Community Policing, Chicago Style” by Wesley G. Skogan and Susan M. Hartnett.
“Impact Evaluation of the LAPD Community Safety Partnership” by Sydney Kahmann, Erin Hartman, Jorja Leap, and P. Jeffrey Brantingham.
“Crime, Insecurity, and Community Policing: Experiments on Building Trust” by Graeme Blair, Fotini Christia, Jeremy M. Weinstein, Eric Arias, Emile Badran, Robert A. Blair, Ali Cheema, Thiemo Fetzer, Guy Grossman, Dotan Haim, Rebecca Hanson, Ali Hasanain, Ben Kachero, Dorothy Kronick, Benjamin Morse, Robert Muggah, Matthew Nanes, Tara Slough, Nico Ravanilla, Jacob N. Shapiro, Barbara Silva, Pedro C. L. Souza, Lily Tsai, and Anna Wilke. [Forthcoming book.]
TRANSCRIPT OF THIS EPISODE:
Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac, at Texas A&M University, where I'm an economics professor and the director of the Justice Tech Lab. My guest this week is Graeme Blair. Graeme is an associate professor of political science at UCLA. Graeme, welcome to the show.
Graeme [00:00:25] Thanks so much for the opportunity to come on. I'm a big fan.
Jennifer [00:00:27] Awesome. Glad you're here. Today, we're going to talk about your research on community policing, but before we get into that, could you tell us about your research expertise and how you became interested in this topic?
Graeme [00:00:38] Yeah, so I mostly worked in kind of high violence contexts, mostly in southern and northeastern Nigeria since graduate school on violence reduction of various kinds mostly before the study on armed conflict. So in some ways, this is a bit of a left turn to focus on crime and policing, but it felt to me like it was very much in line with my sort of general interests and expertise in understanding violence and what we can do to prevent it or mitigate its effects. And I've also worked on methods for improving research design. So, for example, I've done some work on how to ask sensitive questions in surveys. So where you want to protect individuals from harm when they're telling you they, for example, shared information with an armed group but still kind of learn about the prevalence of those behaviors and I've been working on some methods for planning research like that we use in this community policing study using simulation methods as well.
Jennifer [00:01:29] So your paper is titled "Community Policing Does Not Build Citizen Trust in Police or Reduce Crime in the Global South." Nice preview of the results there. It was published in Science in 2021 and you're part of a very large coauthor team for reasons that will become clear to listeners shortly. I hope you all forgive me for not reading all 25 names. So let's start with what should be a simple definition, but probably isn't. What is community policing?
Graeme [00:01:53] Yes, it's a term that's used in a lot of different ways and different settings, but broadly to mean changes in police practices that increase the involvement of citizens in the policing process. And even though it varies a lot, they're often kind of four common elements. One is frequent local beat patrols, where cops are assigned a particular area to patrol. They go out, talk with citizens, talk with business owners, and they learn about some of the concerns that they have that may be leading to crime. And then the second set of them are kind of designed to take up those concerns, so they have decentralize decision making often where these beat cops are empowered to make decisions about how to allocate their own effort to respond to these concerns from citizens and problem oriented policing, which is a set of strategies for officers to problem solve those issues that come up.
Graeme [00:02:44] And it often also includes a community engagement component which builds on the patrols with formal community meetings where both those beat cops and also sometimes their supervisors come and meet with community groups and citizens to talk in a more structured setting about the issues that are coming up in the community. Beyond these police practices, I think there's there's a set of practitioners who think of community policing not just as a set of practices, but really a social movement. They think of it as where this has been successful. It's a systematic change in how police operate and how police interact with citizens. And so it's a change in government in how citizens interact with police and how the police organize themselves.
Jennifer [00:03:27] Yeah, and I guess that's the piece that makes the definition less straightforward then than we might like in some settings. It's sort of a feel, sometimes more like a movement than a particular technique. So what makes people so excited about community policing? What are the mechanisms that people have in mind when they say that community policing is going to change the policing outcomes we care about?
Graeme [00:03:51] I think it really comes from this idea that effective policing requires the cooperation of citizens. They provide tips about future crimes and evidence about past offenses. And where community policing comes in is that often citizens don't trust the police for a variety of reasons, and when they don't trust the police, they often don't cooperate with them. And so they don't provide this cooperation that many people see as a key input to how police allocate their effort, how they decide what crimes to investigate and how to decide where to patrol. And so community policing is meant to jumpstart a virtuous cycle that increases trust in the police, puts people in the same room, changes both citizen attitudes about the police, also police attitudes about citizens, and then generates cooperation with the police and then also potentially makes the police more effective at their job, which might have a feedback effect into trust. And so it's this virtuous cycle of trust, cooperation with the police and reduction in crime that people are hoping for.
Graeme [00:04:50] Just like the simple definition, that isn't another set of community policing practitioners, I think, have a more narrow idea of of why this is important. They really want to increase trust in government and trust in the police, and they think of community policing as a tool to do that even if it doesn't have these kind of knock on effects on police effectiveness and crime. So we designed our study to be able to pick up multiple theories of change.
Jennifer [00:05:17] Interesting. Okay. So what had we previously known before you all started working on this study about whether this works, about the effects of community policing in the real world?
Graeme [00:05:29] So I think community policing gained prominence from on the basis of two success stories. So in Boston, they developed in the early nineties what was called neighborhood policing, which was a kind of responding to the crime wave of the eighties and early nineties. And one of the core ideas there was a same cop, same neighborhood philosophy where officers were assigned to local beats and became familiar with the neighborhood and with community leaders that were living there and they attribute this major drop in crime that they saw in Boston at this time to neighborhood policing.
Graeme [00:06:03] The second is in Chicago, which initiated this program called the Alternative Policing Strategy or CAPS, which was tracked from the beginning by a team of researchers at Northwestern. And so it included community meetings that, like the beat patrols, were designed to surface concerns from the community and to kind of jointly develop problem solving strategies. And they also saw a reduction in crime during that period and also they document a rise in citizen trust. So it starts really with those with those two success stories that led to this really broad interest in community policing, first in the U.S. and then outside.
Graeme [00:06:42] So we also did a systematic review of the randomized experiments on community policing. There are none on community policing as a whole, but we looked for experiments on kind of four elements of this set of practices. So first on community forums which tap into that community engagement piece, community presence, which are things like foot patrols, citizen feedback mechanisms like hotlines and also on problem oriented policing and so what we were looking for was experiments that tried to assess the effectiveness of one of those four components. What we found was that there were a number of past trials, but they were almost all in the U.S., the U.K. and Australia, there were just a handful from from the Global South. And the evidence pointed to the effectiveness of patrols or this increase in community presence and also problem oriented policing in terms of reducing crime, but there really wasn't very much evidence about whether these had impacts on this kind of core outcome of trust.
Graeme [00:07:41] This wasn't measured in most of the trials, and the evidence from those that did was kind of mixed. We found almost no evidence from the Global South and maybe most crucially, the set of practices often implemented in tandem and we didn't find any experiments that addressed how they might work together and whether they're more effective together or whether these even do work together.
Jennifer [00:08:04] So this has been a popular technique for a while, but it sounds like the evidence is still much later than we might like, though, you know, it's a space where they're a bunch of cities, which is definitely not necessarily difficult. So, you know, what makes this difficult to study? What are the main hurdles, as you all were thinking about this study and what you're planning to do? What did you see as the main hurdles to figuring out whether community policing works? Is it mostly a data challenge or mostly an identification challenge or both of those things?
Graeme [00:08:30] I would say for the past studies there, there are a number of of quasi experimental studies that try to compare places that implemented community policing to places that didn't, but I think the challenge there is that the kinds of police agencies that are going to decide to implement community policing may be really different from the others. And in particular, these may be places where citizens trust in the police is declining or crime is rising. And so if we're trying to compare places that implemented places that didn't, then they may just be really different at the front end on the same outcomes that we care about, but it's still sort of puzzling why there haven't been any major RCTs on community policing in the US, given how prevalent these practices are.
Graeme [00:09:10] You know, like 100% of large police agencies in the US have implemented community policing, whereas 90% of all agencies and so why is that? I think one is that partnering with the police is is challenging and there aren't outside of the U.S. in the U.K., there just aren't very many examples of these kinds of successful partnerships. They're starting now, particularly in a couple of places in Colombia and Liberia, where researchers have kind of figured out how to convince police agencies of the value of doing this kind of research, but there just aren't that many examples of of doing this. And so you can imagine some of the some of the practical challenges of doing that, that that police are operating on a much shorter time scales often than researchers, and that some of the kinds of data that you need on crime and on police behaviors and where police patrols are going is pretty confidential. and so in order to convince these police partners to work with us on a study we had to convince them that there was something of a value for them and that we would be able to to protect the data that they were going to be willing to share with us.
Jennifer [00:10:16] Yeah, I'm very curious about the data here, and we will get into that shortly. So you and your large coauthor team ran six coordinated field experiments across six different countries in the Global South. So what were these six places and how did you choose them?
Graeme [00:10:31] So this all kind of came about while I was a postdoc at EGAP, Evidence in Governance and Politics, which is a network of researchers and practitioners, economists and political scientists, as well as development practitioners and folks interested in crime and violence and they were just starting this Metaketa study design where the idea is that you, instead of just running one experiment to learn about the causal effect of some policy, you run a set of them in a coordinated fashion so you select a common intervention or policy that you're interested in, and you select a set of common outcomes that you want to measure, and you run them together to see whether not just does this policy work in this context, but does this policy seem to work across the range of kinds of contexts or policymakers might want to do them.
Graeme [00:11:16] And so as someone interested in crime and violence, I was assigned to try to cook up one of these studies for in the area of violence reduction and so working with Jeremy Weinstein at Stanford and Fotini Christia at MIT. We looked around to see what might be feasible, and I think we were we were very cognizant of the fact that there had been very few RCTs done in partnership with either police agencies or armed forces or other other kind of course of groups in the state, and that this might be an opportunity to try to kind of jumpstart research outside the global north on those topics.
Graeme [00:11:53] And so we wanted to have a range of kinds of places so that our evidence could speak to the question of does community policing work across low and middle income countries? And we succeeded on that. We have our six sites are in Medellin, in Colombia and Santa Catarina State in Brazil, in Monrovia, the capital of Liberia, in Pakistan, in the Shikarpur region, which is near the capital, Lahore, in Sorsogon, which is a province in the Philippines and across rural areas of Uganda. And so across those places, we have places that had higher and lower levels of baseline trust, you know, higher and lower levels of police capacity, we have democracies, autocracies, upper middle income countries, and also some of the poorest places in the world and places with no experience with community policing in Liberia and places with a bit more experience such as Colombia. Like the ultimately the answer to your question was it was a practical choice. This was where we could identify or build partnerships with police agencies who were willing to randomly assign an increase in community policing practices in their jurisdictions.
Jennifer [00:12:57] And just thinking about like the logistics of like how you get started here so were these places that some of you all had contacts in the past and you just start asking around or what did this look like in those early months?
Graeme [00:13:10] So we were looking for a set of policies that where there would be kind of a critical mass of partnerships that either existed or people thought it would be feasible to exist. And we found that there were three teams of researchers that were having some very initial conversations about community policing with police in Medellin, in Pakistan and in Uganda and so we thought we would build on that.
Graeme [00:13:33] And so we set we sent out a request for proposals much more broadly to economists and political scientists working in this area and said help us identify opportunities for running experiments on on community policing and so that led to the six contacts that we have. And we then worked to harmonize what kinds of community policing practices these different agencies were interested in and also what they thought would be most effective. And then we started to develop a common research design and questionnaire for the surveys that that would let us study community policing on a common scale.
Jennifer [00:14:08] Very cool. Okay. So let's talk about each of these sub-experiments in turn. Let's start with Brazil. So what was the context there and what was the intervention that you ran?
Graeme [00:14:20] So we're working in Brazil in Santa Catarina State, which is a southern state that's pretty wealthy. We're mostly working in urban, urban parts of the state and this is a place that has lower crime than some of the other contexts, but still relatively high crime rates and also victimization by the the police. It's a place with with organized crime that that is present. And they had started a program in 2016 called Rede de Vizinhos, which was an effort to develop easy ways to communicate between citizens and the police, in particular through WhatsApp groups and so our intervention had two components. The first was a town hall meeting to encourage the formation of ongoing communication between citizens and the police through these WhatsApp groups that in which citizens in a neighborhood would be able to communicate directly with the beat cops in that area, identify problems, ask for help, and the police would be able to solicit information when something happened. And so they implemented these town halls and then these WhatsApp groups that that lasted throughout the intervention.
Jennifer [00:15:32] Okay. Next is Colombia. So the context there and the intervention.
Graeme [00:15:36] So again, in Colombia, we're working in a in a slightly higher income context. It's a police agency at the Medellin Metropolitan Police that are a little bit more capacitated than the other the other places, but it's a place that had had really high crime rates in the eighties and nineties, in an era of narco trafficking and police impunity, and there were a number of instances of of really kind of shocking police abuse that led to a demand for reform. That reform has happened and so crime has really come down, but surveys have revealed that there's still persistent distrust of the police, that even though the situation has really changed, citizens haven't updated their beliefs. And so the police were motivated to try to figure out how can we bring up citizens trust in the police and increase citizen trust in this kind of environment where there were there still is violence, where local gangs, known as 'combos,' provide public safety in some places are competing with the police.
Graeme [00:16:36] And so we implemented in partnership with them community meetings. There were three per beat, one every three months during the intervention and so two patrol officers and sometimes a high ranking officer or a government official would come to these meetings and they shared some information about the police. They shared the reporting lines where citizens could share tips about future crime or about perpetrators of past crimes. And then they talked to citizens to try to develop a cooperation agreement by the end of the meeting where citizens would commit to some actions and police would commit to some actions, and then they would come back to the next meeting and take stock of what had happened.
Jennifer [00:17:16] Okay. Next is Liberia.
Graeme [00:17:18] So in Liberia, we're working with the Liberian National police, and we were working in densely populated neighborhoods in Monrovia, Liberia's capital. So the police selected a set of places with with pretty high crime. We also added a set of places that that were randomly selected from around the city. And the intervention involved holding town hall meetings, as well as increasing foot patrols and encouraging people to form a community watch forum. So both in Liberia and Uganda, we implemented community watch forums as well as town hall meetings. And the idea of this community watch forums was that there are, in these settings, informal security groups that are already providing public safety. And these forums were meant to kind of formalize the communication between those groups and the police to try to reduce the incidence of a vigilante violence where people kind of take matters into their own hands instead of reporting crimes to the police. And so it included town hall meetings, foot patrols and these community watch forums.
Jennifer [00:18:24] Next is Pakistan.
Graeme [00:18:26] So in Pakistan, we're working in two urban and rural districts in this Shikarpur region, in Punjab province outside of Lahore. This has some slightly lower crime rates than in other contexts, but like in Colombia, despite the lower crime, police are among the least trusted institutions in Pakistan. I think there are there are few reasons for that. Pakistani law constrains what kinds of crimes the police can investigate. Many of them require approval from a magistrate judge and some require eyewitness testimony in order to for successful prosecution. People also perceive the police to be to be quite corrupt. And so this all adds up to a setting in which police in which citizens are afraid of and and distrustful of the police. And so our intervention built on some reforms to try to better link police with citizens. And so we implemented town halls, selected increases in foot patrols. We encourage the use of an existing police hotline where citizens could call anonymously to report crimes and a problem oriented policing program. That was a set of strategies for police to kind of problem solve the issues that were coming up in each community.
Jennifer [00:19:44] The Philippines.
Graeme [00:19:46] So the Philippines is an interesting one. We are working with the Philippines National Police, which are known in parts of the Philippines, to be the kind of nasty arm of former President Duterte's war on drugs. They're the ones who are implementing this campaign of extrajudicial violence. And so we wanted to work with them, but we needed to do so in a way that made sure that citizens were safe, that we weren't by increasing contact between citizens and the police, putting citizens in danger. So we selected Sorsogon Province, which is far from Duterte's war on drugs. And so this is a place that has very little drug or anti-drug related violence, but the reputation of the Philippines police is tied up in this extrajudicial violence campaign. And so the commanders and in Sorsogon province wanted to improve the reputation of the police and increase citizen trust. So we worked with them both in urban and rural parts of the province to construct foot patrols in a first phase and then probably more into policing in the second phase. And so in this study, we were able to separate the effects of that community engagement piece from the problem oriented policing, which are these strategies to address the problems that come up during the community engagement phase. And so officers engaged citizens on these foot patrols stopped at businesses and schools, they get home visits and then in some cases held kind of like small, informal gatherings to try to understand how they could improve the violence situation in that community.
Jennifer [00:21:25] And finally, Uganda.
Graeme [00:21:27] Uganda is also a bit of a unique context because the Ugandan police force has dual roles in the country. It's an authoritarian context with a dictator, Museveni and so Uganda police force both is set up to do the traditional tasks of the police preventing and responding to crime, but it also during elections is the tool by which the national resistance movement, Museveni's party, stays in power. And so because of these two roles, the levels of trust are pretty low. So we worked in rural areas of the country, which were places where the Ugandan police thought that community policing might be a tool that could improve trust and reduce crime. It had been tried in in the capital, in Kampala, and they felt that it wasn't successful. And so we implemented a set of community policing practices at these rural police stations that included town hall meetings, door to door visits or patrols, night patrols, and also the formation of these neighborhood watch teams like in Liberia, to try to formalize the informal security arrangements in these communities.
Jennifer [00:22:36] Okay. So you've got basically six papers in one here. And it just it really is just mind boggling that you all pulled this off. And as you mentioned before, you implemented everything as a randomized trial. So at what levels are you randomizing here? Just the officer level, the neighborhood level, the day level? A mix of them.
Graeme [00:22:55] Yeah. So they're a little bit different in each place. So in there we wanted to implement them at a geographic level, the smallest one that we that we could get the police were operating at. So in Brazil, these were at the neighborhood level. So the police identified a set of eligible points in the city and we drew neighborhood boundaries around them. And those were randomly assigned to receive the Rede de Vizinhos program. In Colombia and Pakistan they were police beats that were randomly assigned. In Liberia, it was communities, so the Monrovia was divided up into a set of communities, and we selected from those. In the Philippines similarly, it was barangays, which are communities so in urban areas, those neighborhoods in rural areas, those are kind of villages. And Uganda's the one slight outlier there there were working were randomizing at the police station level because of work in rural areas. That was the lowest level that they felt that they could randomize that. So a little bit different in each place, but all geographic areas that were assigned to receive these increased policing practices or not.
Jennifer [00:24:05] And I imagine that getting data from these places was not always straightforward. It's, you know, not always straightforward in contexts like the U.S. and Western Europe. I imagine it's much harder in some of these places. So what data were you able to use for your analysis?
Graeme [00:24:21] Yeah, So we have kind of three sets of data. The first two, we conducted ourselves first as a citizen survey. So in most settings we did a baseline end line and we also have a midline to separate these two components of the treatment in the Philippines. And these were citizen surveys that measured experiences with the police trust and the police attitudes about the police capacity, police responsiveness to citizen concerns. And then also a detailed kind of gold standard crime victimization survey. And so those were our primary measure of crime, victimization and also of cooperation, whether people chose to report those crimes to the police.
Graeme [00:24:58] We then ran an officer survey at N-line, and each of the six contexts that was designed to understand whether community policing changed officer attitudes towards citizens and also some norms of officers about the kinds of behaviors that were appropriate and what they would do if they if they saw inappropriate behavior. And then finally, we had administrative data from police agencies on crime, and we collected a common set of both violent and nonviolent crimes, basically from police blotters. And so in some of the contexts, this meant that police were aggregating this data for us, but in most cases, they provided the very raw crime crime blotters that we then transformed and merged into the our data on police beats or neighborhoods the level that we were randomizing out.
Graeme [00:25:48] And we treat the citizen survey as our primary measure of crime victimization because the crime data, first of all, measures kind of two things. One is the incidence of crime, how often crime is taking place, but also how often people report it. So the only crimes that come into the police blotters are crimes that people report, but also because of in some of these contexts that in order to get into these police blotters, police have to register these crimes. And so there's a bunch of different steps you have to go through in order for a crime to end up in this administrative data whereas in our citizen survey on victimization, we could measure all of the crimes that these individuals or their families were the victim of.
Jennifer [00:26:31] And so what outcome measures are you most interested in here?
Graeme [00:26:34] There's really four. So we looked at citizen trust in the police and then a range of citizen attitudes towards the police. So trust, cooperation with the police, which we measured as tips and information sharing about crime victimization. Third, we measured these police attitudes towards citizens and these community meetings are meant to not only improve how citizens think of the police, but how police think of citizens. And finally, this ultimate outcome of crime, did this reduce violent or nonviolent crime in these communities and in addition to crime we also looked at police abuse because I think I think the hope of a lot of advocates of community policing is that not only is this going to improve the public safety in terms of crime, but improve public safety in terms of police victimization of citizens.
Jennifer [00:27:19] Okay. So let's talk about what you find. First, you consider compliance. So that is the extent to which you're treated areas actually saw increases and things like community meetings and police patrolling. What did you find there?
Graeme [00:27:32] So this is maybe surprising. We found that the police complied with these planned community policing practices that we had mutually agreed upon, but unevenly and only in some measures of compliance and somewhat differently across sites. So the police in every setting did hold community meetings, and in most settings people went to them often, many often as many as 800 people in one context. And we also find pretty substantial increases in awareness of community meetings in most settings, but overall, we can't distinguish the difference between treatment and control areas from zero across the six contexts. So a weaker effect on these community engagement practice compliance measures than than than we had expected. In every setting except Brazil in our index of community meeting awareness, foot and vehicle patrol frequency we do see statistically significant increases in our compliance index, and these are large in Liberia, but they're they're relatively modest in magnitude in these other settings.
Graeme [00:28:33] And in Brazil, which is the one standout in terms of compliance we found really weak compliance with intervention overall. One reason for this may be that in the time in which we conducted this survey, the program Rede de Vizinhos had had actually already been had had been rolling out at a faster pace than the police had originally anticipated. And so it may have rolled out to some control areas outside of the context of our study. And so in Brazil, we use a slightly different design and encouragement design where we try to encourage people to join these WhatsApp groups and we don't see any first stage effect there and so we aren't able to say a lot about what happened in Brazil because of this really low compliance with the intervention.
Jennifer [00:29:18] Yeah, and I think you mentioned in the paper that at least in robustness checks you drop Brazil or something like that just to see if that seems to be driving your, your other results, is that right?
Graeme [00:29:28] That's right. When you drop Brazil, the results are all the same, basically because we just don't learn much from from the Brazil context.
Jennifer [00:29:34] Got it. Okay. And then the main results next you measure the effects of your community policing treatment on crime and these other policing outcomes. So what did you find there?
Graeme [00:29:44] Yeah. So overall, you know, disappointingly because this intervention is is implemented on every continent in most police agencies in the U.S., in the U.K. and in most countries, we find no change in citizen trust in the police, no change in officer attitudes toward citizens. We don't find a reduction in police abuse. We don't find an increase in citizen cooperation with the police in terms of people reporting crimes or tips.
Graeme [00:30:11] And we don't find any change in crime. And because of the scale of the study of these six contacts, when we put all this data together, this is a relatively well powered null. You can rule out even pretty modest improvements. We can also happily, I guess, rule out modestly sized backlash effects. So I think there there are some folks who worried that by increasing contact between citizens and the police, that there could be some backlash effects if there weren't broader changes to the police, so basically disappointing people. And so we don't we don't see any evidence of that, but also no positive changes overall in the ways that people expected community policing to affect crime and the relationship between citizens and the police.
Jennifer [00:30:52] Were there at least any interesting differences across places in these outcomes? You can cut in all different kinds of ways. Do you find anything when you when you try when you look at different, different sets of places?
Graeme [00:31:05] Yeah. So of course, after you find that there is no effect, you have to dive in to find out like, gosh, like did this work in any way kind of place.
Jennifer [00:31:12] Anywhere.
Graeme [00:31:14] I mean, we just looked at everything. We looked across, we looked at differences across settings. So what are the effects on these different outcomes in the six different contexts, we looked within settings, so we had a set of pre-intervention measures of, for example, trust. So you could imagine that places that already had high enough trust in the police, this might work better if you just had such low levels of trust maybe, maybe this couldn't be effective. We didn't find that. We looked at whether places that had higher crime rates might have more effective community policing because people are just more motivated to try to to help address the situation.
Graeme [00:31:52] We don't find that we thought maybe that this had to be the kind of place where there was broad enough trust in state institutions or in government. We don't find any effect there. And so we basically don't find any differences within settings of different kinds of communities. We do find a couple of differences in a couple of settings in the citizen outcomes, in the citizen attitudes towards the police. So overall, we find no change in the core outcome of trust in the police, but we break that down into a couple of different components. And so in Liberia and Pakistan, we find an improvement in perceived police intentions and in Colombia we find a small improvement in perceived police capacity.
Graeme [00:32:38] And so this kind of question of like, how do you interpret these differences? And I think our our perspective in general is that we don't see the reverse of these effects in Colombia we see an increase in perceived police capacity, but not an increase in perceived police intentions and vice versa with Liberia and Pakistan. And so in a study that has more than 100 outcomes, we think that these effects are relatively small and maybe at best are indications that there's been some small attitudinal change, but not a systemic change in how citizens and the police operate. We do find changes in some non pre-registered outcomes for the mean study that were pre-registered in our Liberia and Philippines teams separately. So in Liberia, like in Uganda, they're worried about vigilante violence and in Liberia they do find a change in how often people seek assistance from the police versus informal justice institutions, so that's a positive.
Graeme [00:33:38] And then in the Philippines, they looked at a couple of crimes that were particularly targeted by their intervention and they do find some reductions in those particularly targeted crimes. But overall, they don't find any change in the crime rate. And so this didn't have a kind of systemic effect on an improvement in the crime setting in the Philippines either.
Jennifer [00:33:58] Okay. So some glimmers of possible good news here and there, but also running a ton of regression. So who knows what to make of that? Mostly a big null effect. Very expensive, very logistically complicated null effect.
Graeme [00:34:13] Yes.
Jennifer [00:34:14] That will be disappointing to many people. So what are the policy implications of all of this? What should the policymakers and practitioners who are listening take away from your results?
Graeme [00:34:24] So I think these it's always hard to interpret a null effect and so we want to be kind of careful in what we make of these results. And so what we tried to do was after after we found that there was a null effect, we went back to our data, we went back to our teams and we went back to our police partners to try to collect some qualitative impressions about kind of what went wrong, what was it that might have undermined the effectiveness of community policing, taking and keeping in mind like the most generous interpretation of how community policing should work. And we came away from that with maybe two, two main ideas. One is that structural reforms may be needed to change how the police operates, how the incentives are set for four officers and how police effort is allocated before these changes to crime or citizen trust can be impacted by these smaller changes to police practices that it may simply not be enough to have regular meetings or foot patrols without changes to, for example, how officers are compensated.
Graeme [00:35:28] And and so it's possible that if there were some of those more structural reforms in tandem with community policing, it's possible that it could be effective, but we don't know. The second is that this is one of a set of experiments where these are policies that are being exported from the U.S. to places that are really pretty different settings that have different levels of police capacity, that have different experiences with civil conflict and with police abuse. And so it's possible that local solutions are needed and that these policies developed in the US just may not be effective. I think on the flip side, though, it's also possible just that community policing isn't effective in the ways that its advocates expect uncovered in our systematic review that there aren't any RCTs on this set of practices as a bundle and that there's really pretty little quasi experimental evidence on community policing as a whole. And so it's simply possible that there even though this set of policies has been implemented all over the world, that we just started out not knowing whether it worked and that that if we had if we had looked and run some of these experiments earlier on, we might have found that that is simply not effective, even in the kinds of contexts in the US where it's been implemented.
Jennifer [00:36:46] It would certainly not be the first time we have done something like that, especially in the criminal justice space.
Graeme [00:36:52] For sure.
Jennifer [00:36:53] Your paper definitely left me wondering that does this work anywhere? So I'll be interested to see more research on that going forward. Have any other papers related to this topic come out since you all first started working on the study?
Graeme [00:37:06] One of them that came out right around when our study came out, which I think is one of the most compelling papers on community policing in the U.S., is by Kahmann, Hartman, Leap and Brantingham, a couple of political scientists, and anthropologist at UCLA who worked with the LAPD to evaluate the rollout of base community policing practices in public housing developments in L.A. and so they're they're assessing in the same way that we were, this kind of bundle of practices that are labeled community policing and it's one of the few really credible studies to do that in the US. And they find really small effects on crime that can't be distinguished from no effect at all, and they find no evidence of spillovers of crime kind of moving from treated areas into places where they weren't implementing community policing. And so I think that reinforces my belief that that we may just not have credible enough evidence about community policing in the U.S.
Jennifer [00:38:02] Fascinating. Okay. And so what is the research frontier here, aside from maybe testing whether community policing works in general? More research on that basic question. What are the next big questions in this area that you and others are going to be thinking about going forward?
Graeme [00:38:18] One thing we're doing is looking back at the evidence from the U.S. with a more skeptical eye. So we're looking at quasi experimental evidence and also at digging into some of the success stories in Boston and Chicago and a couple of other cities to say kind of what do we know? Using modern tools of causal inference, how certain are we about what's the range of possible effects that this could have had and these places where that have been identified us as community policing success stories? And we're collecting data on the U.S. spending programs, on community policing that were responsible for its rise across the US and basically trying to say, what do we know to try to help motivate either policymakers who are making decisions about its use in the U.S., but also researchers who are trying to decide where should we put our effort in in trying to test this.
Graeme [00:39:08] And I think, as you pointed out there, there are a couple of different ways that that we could test community policing, plus what are the kinds of things that might help community policing be effective. There's definitely among the practitioner community, both of police officials, but also international organizations that promote community policing around the world there's an idea that there is there is a well-implemented version of community policing out there that isn't what we implemented. So we implemented what you might call sort of a realistic test of what happens when a partnership between an external agency and a police agency decides to increase the strength of community policing practices. But what if it's the case that when there is this kind of social movement and buy in from those police officers and their leadership, as well as government, who are often a key actor because they have to take non policing actions to address some of the concerns that come up in the community if there's that social movement among government, police and citizens can a well-implemented version of community policing be implemented. And I have some hope that we could we could do some experiments around that because there's so much momentum and interest in this that I think we should try that out and see whether see whether it's this kind of weaker implementation or weaker compliance with intervention. That's the kind of core causal process in the null effect that we found.
Jennifer [00:40:33] You know I have to say, I'm always skeptical when people point to the implementation as the reason that something didn't work, especially, you know, I mean, you all implemented this as it would be implemented in the real world and so so I agree. It's an interesting question. Like, is there something else you could you spur a social movement to make the implementation stronger, but like, that's a really high bar to ask a lot of places to do. So I tend to think that what we're testing is, you know, what would actually be implemented for practical purposes. Do you think you'll do another one of these meta RCTs again?
Graeme [00:41:05] This took over my life from 2015.
Jennifer [00:41:10] I bet.
Graeme [00:41:10] When we started until we're working on a book that's going to come out with Cambridge next year.
Jennifer [00:41:14] Oh great.
Graeme [00:41:14] So right now that kind of has a longer discussion of of each of the cases and kind of lays out the broader set of evidence and so it's been going on for for a long time. And I kind of can't imagine working on something on this scale with with government agencies. That being said, we're I'm working on a project on research ethics. So on what the causal effects of asking about violence and surveys are and we are doing that in a couple of contexts in a coordinated way, but some on a much smaller scale than this, because I think I think the biggest the biggest part of this coordination was was working with police agencies that were on different timelines, with officials who were changing jobs frequency and with lots of institutional constraints on the data and and when we could collect data. So hats off to the next people who do this.
Jennifer [00:42:05] Well, hopefully someone listening is inspired to to run something like it. I'm glad you all pulled it off. It does seem like a miracle. My guest today has been Graeme Blair from UCLA. Graeme, thank you so much for talking with me.
Graeme [00:42:17] Thanks so much, Jen.
Jennifer [00:42:23] You can find links to all the research we discussed today on our website probablecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show and thanks also to our Patreon subscribers and other contributors. Probable Causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Nefertari Elshiekh. Our music is by Werner and our logo was designed by Carrie Throckmorton. Thanks for listening and I'll talk to you in two weeks.