Basic Science Funders Ask, "How Do We Know if Our Giving is Working?"

Generally speaking, foundations want to know what their grants are accomplishing, or whether they should be putting money somewhere else. This is understandable, to an extent.

But a lot of work—some of the most important and effective in the long term, in fact—doesn’t lend itself to frequent grantee reports packed with discrete deliverables and short-term metrics. That fuzziness can lead funders to either shrink from such work, or require grantees to run a gauntlet of requirements that don’t make much sense.

This is a sticky problem in philanthropy, and one that’s particularly challenging in the funding of basic science—that which is motivated by discovery and not a particular application.

That’s why about a dozen leading funders—including Moore, Sloan, HHMI, Simons and the Wellcome Trust—met a couple times over the past year to work toward some best practices for judging impact in this long-term and often meandering work. They’re not there yet, but a recently released brief on their discussions offers some interesting insights into how funders approach this challenge, and some of the pitfalls in funding research (you can read the full paper here).

Related:

We’ve often been impressed with the appetite for risk in science philanthropy, more so than in a lot of other areas of giving. Some leading funders specialize in giving long-term, unrestricted support for work that may not directly produce tangible or targeted results. This is important in basic research, because even though it may eventually yield world-changing applications, the point of the work is pure discovery, meaning you never quite know where it’s going to go.

But that can be a tough pill for some to swallow, especially as the field’s leaders (represented in the relatively new Science Philanthropy Alliance) are trying to guide more private wealth toward basic research. Meanwhile, established funders want to improve their own abilities to gauge success.

In the recent discussions, hosted by the Moore Foundation, science funders grappled with some of the reasons it can be so challenging—nonlinear paths to discovery in science, the blending of multiple funding sources in any given project, science’s inherently collaborative and incremental nature, and the long stretches of time before impacts manifest.

Participants pointed out limitations in some of the seemingly reliable tracking methods. Analysis of acknowledgments in publications can sell researchers short, and doesn’t account for the importance of each contribution. Self-reporting, and even expert panels, run the risk of being subjective.

They also cited interesting phenomena in science giving, including a susceptibility to the Matthew Effect. Science philanthropists frequently cite funding of risky or unproven subject matter as a strategy to fill unmet needs. But they’re also vulnerable to a tendency in the science community to shower already successful scientists with the most reward. In philanthropy, this translates to successful researchers landing more funding than up-and-comers, partly as a way for funders to mitigate risk. This is also a common dynamic we’ve observed in high-dollar science prizes.

Related:

Another highlight from the meetings is the recognition from participants that some of the more successful tactics science funders employ involve bringing together researchers, through workshops, building networks, and fostering open access. Of course, this adds another wrinkle for funders when evaluating progress.

The funders involved intend to continue meeting to develop better ways to evaluate grantmaking. Some of the principles they’re considering include more clearly defining what “impact” means in any given program, and using a mix of methods to judge accordingly. But they also acknowledge the importance of learning from failure or unintended results, and letting go of the desire to identify a clear cause-and-effect relationship between funding and results.

While more robust and nuanced ways of evaluating impact should be a goal for funders as research philanthropy grows, so it seems should be the willingness to accept some uncertainty in the wild world of scientific discovery.