“A nonprofit’s financial report reveals virtually nothing about its effectiveness or efficiency in creating social value,” argues Harvard Business School’s Robert S. Kaplan and Allen S. Grossman in their article, “The Emerging Capital Market for Nonprofits,” published in the Harvard Business Review. In a recent interview with the BBC’s Peter Day, Kaplan and Grossman explain why outcome evaluation, not just financial disclosure, is necessary for appropriately evaluating nonprofits and encouraging greater efficiency across the sector.
I’ve long argued that financial measures alone are fundamentally incapable of measuring nonprofit efficiency and effectiveness because they take no account of the outcomes organizations attempt to achieve. How much an organization spends on programs relative to other expenses, for example, tells donors nothing about the impact of that spending in terms of achieving meaningful results. Similarly, Kaplan and Grossman note:
“One financial ratio widely used by nonprofit analysts—administrative expenses divided by funds raised or disbursed—is even misleading. Some organizations may disburse a great deal of money with a small central staff but accomplish little. Others may have higher administrative expenses but attract outstanding staff, have a great model for achieving change, and actually deliver high-impact social services to their constituents. The traditional financial expense ratio makes the first group appear better managed than the second.”
It is useful to make a distinction between two types of nonprofit evaluation. Evaluation based on the principle of ‘outcome accountability’ focuses on whether nonprofits are achieving their promised results and doing so cost-effectively. I use the term cost-effectiveness in place of efficiency to emphasize that efficiency is a ratio of inputs to outcomes, not a ratio simply among inputs. A ratio like the cost per child fed is an efficiency measure because it assigns a financial cost to a specific outcome. A ratio like program expenses to total expenses is not because it fails to take any associated outcomes into account.
Another type of nonprofit evaluation is based on the principle of ‘overhead minimization.’ In this view, organizations are effective and efficient when they attribute a large proportion of their expenses to programs rather than to fundraising or administration. This is appealing because financial measures are easier to come by and simpler to evaluate than outcome measures. Additionally, it feels intuitive that the more a nonprofit spends on program expenses relative to non-program expenses the greater its impact. But why would reducing non-program expenses improve program efficiency? Non-program expenses simply represent a nonprofit’s ‘cost of doing business’ and should be regarded as indirect program expenses.
I’ve also argued that measures of efficiency should take into account both direct and indirect expenses, not solely what are traditionally regarded as program expenses. An efficiency measure such as the cost per child fed, for instance, should include the cost of securing the funding for the program and an appropriate fraction of the nonprofit’s administrative costs. Allocating total costs, rather than just program costs, across a nonprofit’s achievements better represents its true cost-effectiveness.
All this implies that evaluations of nonprofits that rely solely on financial information measure neither efficiency nor effectiveness, but simply arbitrary ratios of costs. I say arbitrary because—beyond anecdotes and speculation—there is no solid empirical connection between a nonprofit’s financial ratios, on the one hand, and whether it is actually accomplishing anything, let along efficiently, on the other. This is a rather severe shortcoming, and a particularly problematic one for anyone trying to measure a nonprofit’s effectiveness or efficiency.
Nonprofits wasting money on ineffective programs shouldn’t be considered ‘efficient,’ ‘effective’ or ‘accountable’ just for having a high program expense ratio. Conversely, nonprofits with unfavorable financial ratios are not necessarily ineffective.
Nonprofits need to be held accountable for achieving results cost-effectively. For this to happen we need a systemic effort to induce nonprofits to disclose specific and measurable goals, to meaningfully evaluate their progress toward them and to reveal the financial costs associated with their achievements. Armed with this information, donors can channel their contributions to the most effective and efficient nonprofits while incentivizing less effective and efficient organizations to improve or perish. This is the kind of oversight and market discipline the nonprofit sector needs.
Kaplan and Grossman propose that emerging social investment intermediaries—acting somewhat analogously to mutual funds in the business sector—could help meet this challenge. The current system, based primarily on the principle of overhead minimization and implemented by a few nonprofit ratings agencies, clearly falls short. Thankfully, one of the most popular nonprofit ratings agencies, Charity Navigator, now recognizes the primacy of outcome accountability and, very laudably, has begun migrating from their old version 1.0, based solely on financial ratios, to their new version 2.0, which promises to assess organizational attributes related to results, accountability and financial sustainability.
Charity Navigator’s progressive new framework is ambitious. Few nonprofits generate, let alone disclose the information necessary for meaningful evaluations. For the foreseeable future Charity Navigator will have to make do with limited data from IRS Forms 990 and organizations’ websites. Historically, the widespread availability of financial information has largely driven nonprofit evaluation practices. I hope that in the future the presence of higher quality ratings and evaluation systems will compel nonprofits to generate and disclose more and better data.
Looking ahead, I’m already thinking about Charity Navigator version 3.0, which I hope will hold nonprofits accountable for truly being effective. Potential donors should be able to compare a nonprofit’s annual promises against its annual achievements to assess its track record. This way donors could feel more confident that their social investments today will have demonstrable impact tomorrow.
But efficient social investment requires not only that nonprofits achieve their goals, but that they do so cost-effectively. In version 4.0 I hope Charity Navigator will hold nonprofits accountable for achieving their results at low cost. Given two or more nonprofits with equivalent programs, donors should support the organization with the greatest impact per dollar. This is the kind of information that the public needs to know to make the kind of informed giving decisions that will force nonprofits to become more efficient. But we still have a long way to go. Currently, nonprofits simply aren’t providing the public with adequate information to support better systemic evaluation practices.
Greater outcome accountability will encourage a more effective and efficient nonprofit sector. The public deserves better information about the tax-exempt organizations it subsidizes. Meeting this challenge may require a mixture of financial intermediaries, independent ratings agencies and government oversight all demanding more meaningful evaluation and disclosure practices from nonprofits. We’re not there yet, but I think we’re finally heading in the right direction.
by George E. Mitchell