During the past InterAction Forum in Washington DC, August 10-12, the TNGO Initiative hosted a panel on "What Really Matters: Perspectives on Organizational Evaluation, Disclosure and Learning". The panel focused on outcome data generated and disclosed by INGOs: to what purpose should this be done? Is it feasible and even desirable to come to standardization of outcome related data, and if so, who should take the lead?
Tosca Bruno-van Vijfeijken moderated the panel, while George Mitchell, Assistant Professor at City College of New York (part of CUNY), and TNGO Initiative Fellow presented on "Outcome Accountability: How Far Are We Willing To Go?", which provided a solid foundation to the discussions, and asserted that most NGOs as yet do not have a consistent policy or practice on outcome evaluation, let alone disclosure ofsuch data.
Ken Berger, CEO of Charity Navigator (CN), the influential US web-based rating intermediary, offered an update on the important ongoing changes in their rating system in the areas of transparency and accountability as well as results reporting. These changes will greatly incentivize NGOs to improve their generation and disclosure of outcome data. Ken agreed with George's assessment regarding the vast majority of NGOs ("there is no there there"), and mentioned that most individual donors give to nonprofits based on loyalty considerations and earlier connections to the organization concerned, and not based on good data about outcomes. He also mentioned that 40% of the charities rated by CN will see changes in their rating as a result of the current re-rating based on the new transparency and accountability standards.
Matthew Forti, Program Measurement Lead at Bridgespan, the well-known US consulting and training group for nonprofits, did a presentation that offered some of the answers to the questions raised above. Matthew mentioned the new website Charting Impact, which offers the beginning of such outcome accountability, although he agreed that it has some limitations in terms of outside verification of information offered, and offers no assurance on the quality of evaluation data offered. He stated that only those standardsetting initiatives that help organizations improve their effectiveness and which contribute to organizational learning will survive and thrive, while those that focus foremost on donor accountability will not. Also, it is crucial that institutional donors financially support the significant investment costs that NGOs face in terms of evaluation capacity. Finally, any standardization in outcome data generation and disclosure should be encouraged, not mandated.
Carlisle Levine, former evaluator at CRS and CARE US and co-chair of InterAction's Working Group on Evaluation, offered detailed comments on the three presentations from her practitioner experience. She highlighted the fact that on the one hand, some of the larger INGOs are investing in outcome accountability ("there is a there there"), while NGOs in general face significant constraints in improving their accountability. What to do with federated and/or highly decentralized NGOs, where coming to consensus on what is the right outcome data to collect and disclose can be difficult? How do we apply measures to diverging forms of NGO activity, such as advocacy, capacity building and service delivery? If we (only) measure what we can easily measure, then we are probably not measuring the right things. And what is the right time frame for such measurement?
Overall, the panel was attended by a good size audience, included spirited discussions and well well reviewed by participants. Tosca has been invited to join Charity Navigator's Advisory Panel to help guide CN's changes in how it rates NGOs towards a greater focus on both transparency and accountability ('CN 2.0') and results reporting ('CN 3.0').