I have been a CPA for over 40 years, during more than 30 of which I specialized in serving the not-for-profit sector. Going back to the mid-1980s and early 1990s, the charitable world was talking about measuring outcomes and using outcome measurements to rate not-for-profit organizations. This is a noble objective, but over these 25+ years, nothing has come out of these efforts. As a result, donors pondering where to put their money don't have the information they might want.
The latest group to throw in the towel is Charity Navigator, the big charity rating organization. Recently, it announced that it was placing on hold efforts to determine how well charities measure their impact. In January 2013, Charity Navigator announced it was undertaking a process called CN3.0. It noted that the key to making wise funding decisions was in understanding three things: a charity’s financial health, its accounting and transparency and its results reporting. Charity Navigator set out optimistically to tackle that third piece with a new effort to report results.
More than two years later, in August 2015, Charity Navigator acknowledged this effort was too ambitious because the data just wasn't there. The group said that "most organizations are not yet truly working on a results basis and are not equipped to provide the kind of reporting we were seeking." It was simply unable to assess the effectiveness of America’s charities on the basis of what those charities publish about their results.
That outcome was not so surprising, and reflected some stubborn facts about the charitable sector's opacity. For the most part, charities report the amount of resources (inputs) used to conduct an activity. Most charities also report some measure of the outputs from their activities—e.g. the number of classes held, number of students enrolled, number graduated, etc. (Since all of us have been through the education system, education-related examples serve as good examples.)
Outcomes, on the other hand, measure how much better off society and the organization’s clients are as a result of the organization’s activities. A school system reports how many graduated, but not the number of job seekers who are gainfully employed within a reasonable time after graduation; or how many are sufficiently educated to properly function in society.
The charitable industry has evolved into one that measures activities instead of outcomes. Charity managers will point out, however, that there is rarely a sufficient link between a charity's activities and the desired societal or personal outcome. They note that outcomes are frequently abstract and complex and activities are tangible and easily counted. I believe that such statements merely point out the truth that charities see the big picture (like winning a football game) but fail to realize that their activities are all about getting first downs and moving the ball incrementally forward down the field.
While having an educated, employed population is a desirable outcome, the elementary school outcome is merely to prepare students for the demands of high school. The desired outcome of the first grade is merely to prepare students for the second grade! If charities focused on the correct outcome that their agency exists to affect, outcome measurement would become a whole lot easier. But with bumper stickers expressing sentiments like “If you like your doctor, thank a teacher”, what teacher wants to say that their mission was preparing kids for the second grade?
More than measuring charity effectiveness, a concentration on outcomes could also have a deeper impact on the charitable industry. If resource providers (and boards of directors) focused on outcomes instead of activities, charities might do a better job of selecting the programs and activities to which they devote their efforts. For example, a charity requests a grant to purchase a vehicle. The grantmaking foundation might consider the grant outcome to be the actual purchase of the vehicle. However, when the outcome for the new vehicle is understood to be the transportation of the elderly to healthcare appointments, then the board and resource providers are open to consider other options. In order to increase access to healthcare for the elderly, one might also consider train or bus passes or even strategically located clinics.
When viewed in this light, outcome measurement is not just a good idea. It may be crucial to an organization’s ability to efficiently contribute to society. Understanding the outcome(s) an organization is responsible for may be the only way to design activities and programs that have a chance of success.
Charities have to get realistic about the space in which they are operating and the outcomes that will result from their activities. There is nothing wrong with saying you are in business to get kids ready for the rigors of the second grade. And once you realize that that is your mission, you will more easily develop outcome measures that help you monitor how you are doing, e.g., tracking how many are ready for advanced study without remedial work. How many know what they were supposed to have learned in the first grade? These are not sexy outcomes, but they move the ball forward toward the ultimate goal.