Evaluating the effects of NGO organizational change processes
on human rights advocacy outcomes as well as organizational effectiveness:
Reflections on an External Assessment
of the Global Transition Program at Amnesty International
I am Tosca Bruno-van Vijfeijken, and I direct the Transnational
NGO Initiative at Syracuse University, USA. The Initiative has assisted a
number of major international non-governmental organizations (NGOs) to review
their leadership and management practices related to large-scale organizational
change. My colleagues – Steve Lux, Shreeya Neupane, and Ramesh Singh – and I
recently completed an external assessment of Amnesty International’s Global
Transition Program (GTP). The assessment objectives included among others the
way in which GTP had affected Amnesty’s human rights advocacy outcomes. It also
assessed the efficacy of Amnesty’s change leadership and management. One of the
fundamental difficulties with this assessment was limitations on time and
resources. As such, it was not possible to develop objective measures that
directly represented either assessment objectives. Instead, the assessment
process primarily triangulated staff perceptions at various levels and coming
from across different identity groups within the organization as an approximate
measure of the effect of GTP on human rights advocacy goal achievement.
- The change process was
controversial within Amnesty and generated high emotions – both for and
against. To protect the credibility of the assessment, we gathered multiple
data sources and triangulated staff views through careful sampling for
surveying, interviewing and focus group use. A survey with external peers and
partners added independent perspectives. Workshops to validate draft findings
with audiences that had both legitimacy and diversity of views were critical as
- The evaluators’ credibility in terms
of understanding the nature of the organizational actor (Amnesty’s history,
context, earlier attempts at organizational change, and its culture and
self-understanding) is critical.
- Evaluating human rights advocacy outcomes is complex. Process
and proxy indicators were essential in our assessment.
- It is equally difficult to attribute human
rights advocacy outcomes to Amnesty’s change process, due to the lack of
comparative baseline information or counterfactuals.
- Amnesty is a complex, democratic,
membership-based NGO. Given the controversy around the ‘direction of travel’
under GTP, Amnesty promised accountability towards its members by requesting
this External Assessment barely four years after the change process had been
announced. Statements about the extent of correlation between the GTP and human
rights advocacy outcomes thus had to be all the more qualified.
high profile, high-emotion evaluations like this that are also largely
dependent on staff perspectives, the measurement of number of ‘mentions’, and/or
recurrent staff views was one obvious indicator. However, as evaluators we also need --
in a defensible way -- to judge the strength of points made or issues raised – and
include not just their frequency but also the gravity of their expression.
- Evaluators need to be acutely
aware of where power is situated in organizations if they want to produce
actionable, utilization-focused evaluations.
- In high profile evaluations such
as this, an ability to both understand senior leadership contexts, perspectives
and world views and to speak truth to
power are important.
RAD resources: The frameworks by Bolman and Deal (Reframing Organizations: Artistry, Choice and Leadership, 2017); and
William and Susan Bridges (Managing
Transitions, 2017) to offer consistent value in evaluating organizational
change processes in INGOs.
Continue the conversation with us! Tosca firstname.lastname@example.org