May the Best Study Win: Will More Funding for Evidence-Based Policy Make a Difference?

If you want to change public policy in the United States, you'll eventually find your way to the influential world of Washington think tanks. The Laura and John Arnold Foundation (LJAF), one of the most aggressive foundations seeking to move big ideas, has been investing in Beltway policy shops for a while now. Earlier this year, in its biggest such give yet, the foundation made an $8.4 million grant to the Urban Institute to help develop its Pay for Success work. Now Arnold is taking things a step further: It's setting up its own wonk operation in the nation's capital. 

Kathy Stack and Jon Baron will lead the foundation's new Evidence-Based Policy and Innovation division, which will help develop solutions to a wide range of social problems. Both are well known as leading advocates for evidence-based policy making. Ms. Stack comes from a long career in government, so she likely knows how the game is played. In her new position, Stack will approach government from an outside angle, so she may be able to push for bigger changes than she could from within the systems. The Coalition for Evidence-Based Policy, where Mr. Baron was the former president and founder, will wind down, and the work of that organization will be integrated into the Arnold's new division.

Based on the excitement expressed for these leaders and the level of government enthusiasm for this project, it looks like this team has a strong start for its work of pushing government to more rigorously evaluate existing programs and design new programs with the best credible evidence.

Such work, needless to say, pushes against strong partisan headwinds blowing in Washington these days, where ideology and political expediency routinely trump the facts. But this effort has strong support from the White House, which sees it as a crucial assist to its social innovation agenda. The Arnolds and Bloomberg Philanthopies both recently received props from the Obama administration for being "essential partners" in government's quest to surface the tools, programs and approaches that will help the country adapt to a changing educational and economic landscape. 

"Philanthropy’s role will continue to be critical along a spectrum from helping make data open to the public and enabling local government to make data-driven decisions to using data to support rigorous evaluations of social programs," said David Wilkinson, director of the White House Office of Social Innovation and Civic Participation. Wilkinson argues that "when we know what works best and act on it, we achieve better results" on issues like increasing student reading levels, decreasing homelessness, and helping more families become middle class.

The White House appears to be very glad to have private buddies like the Arnold Foundation and Bloomberg Philanthropies, both of which have been funding tons of research to identify solutions to social problems and then replicate those solutions and bring them to a larger scale.

Obviously, though, there are plenty of reasons to be wary of this kind of work. In a 2013 critique of social impact bonds, Jon Pratt cites Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Also, is anyone else at all worried about the problem of social service research data not having the capacity for reliability that the natural sciences have? There is an important distinction between the quality of knowledge that can be achieved in the biological sciences, for example, as opposed to research in the social sciences that is primarily based on subjective observation of human behavior. While qualitative research practices have improved over the past century, it seems like a good idea to consider the ways in which research funded by people with a particular political agenda might ultimately be more about "policy-based evidence making" than it is about evidence-based policy.

Another critique of worshipping too much at the altar of social science is that it can make you tone deaf to the central role that values play in politics and public policy. In a twist on the old saying "good guys finish last," it is sadly true that the side with the better studies can easily lose the debateunless, that it is, they know how to tap into deep human fears and aspirations. 

The Arnold Foundation's big bet on evidence-based policy reminds us a bit of Hewlett's noble effort to reduce polarization: It's welcome work, to be sure, but we wonder just how much traction it can have absent a reckoning with the powerful ideological actors driving politics today, especially on the far right, which has tapped into primordial anxieties and led the charge against "reality-based" public policy. 

We also wonder about the Arnold Foundation's core values, which aren't so well defined. It's an appealing vision to make "systemic" and "transformative" change based on the best evidence. But that's different than having a normative view of why society should prioritize certain goals. Funders who don't focus enough on defining their core values, and defending those values in the public square, can end up scratching their heads as they keep losing crucial fights.