Evaluating the effects of NGO organizational change processes on human rights advocacy outcomes as well as organizational effectiveness:

Reflections on an External Assessment of the Global Transition Program at Amnesty International

I am Tosca Bruno-van Vijfeijken, and I direct the Transnational NGO Initiative at Syracuse University, USA. The Initiative has assisted a number of major international non-governmental organizations (NGOs) to review their leadership and management practices related to large-scale organizational change. My colleagues – Steve Lux, Shreeya Neupane, and Ramesh Singh – and I recently completed an external assessment of Amnesty International’s Global Transition Program (GTP). The assessment objectives included among others the way in which GTP had affected Amnesty’s human rights advocacy outcomes. It also assessed the efficacy of Amnesty’s change leadership and management. One of the fundamental difficulties with this assessment was limitations on time and resources. As such, it was not possible to develop objective measures that directly represented either assessment objectives. Instead, the assessment process primarily triangulated staff perceptions at various levels and coming from across different identity groups within the organization as an approximate measure of the effect of GTP on human rights advocacy goal achievement. 

Lessons learned:

  1. The change process was controversial within Amnesty and generated high emotions – both for and against. To protect the credibility of the assessment, we gathered multiple data sources and triangulated staff views through careful sampling for surveying, interviewing and focus group use. A survey with external peers and partners added independent perspectives. Workshops to validate draft findings with audiences that had both legitimacy and diversity of views were critical as well.
  2. The evaluators’ credibility in terms of understanding the nature of the organizational actor (Amnesty’s history, context, earlier attempts at organizational change, and its culture and self-understanding) is critical.
  3.  Evaluating human rights advocacy outcomes is complex. Process and proxy indicators were essential in our assessment. 
  4. It is equally difficult to attribute human rights advocacy outcomes to Amnesty’s change process, due to the lack of comparative baseline information or counterfactuals. 
  5. Amnesty is a complex, democratic, membership-based NGO. Given the controversy around the ‘direction of travel’ under GTP, Amnesty promised accountability towards its members by requesting this External Assessment barely four years after the change process had been announced. Statements about the extent of correlation between the GTP and human rights advocacy outcomes thus had to be all the more qualified.
    GTP graph screenshot
    With high profile, high-emotion evaluations like this that are also largely dependent on staff perspectives, the measurement of number of ‘mentions’, and/or recurrent staff views was one obvious indicator. However, as evaluators we also need -- in a defensible way -- to judge the strength of points made or issues raised – and include not just their frequency but also the gravity of their expression.
  6. Evaluators need to be acutely aware of where power is situated in organizations if they want to produce actionable, utilization-focused evaluations.
  7. In high profile evaluations such as this, an ability to both understand senior leadership contexts, perspectives and world views and to speak truth to power are important.

Graph, "Overall, do you think that the Global Transition Programme has resulted in greater human rights impact?

RAD resources: The frameworks by Bolman and Deal (Reframing Organizations: Artistry, Choice and Leadership, 2017); and William and Susan Bridges (Managing Transitions, 2017) to offer consistent value in evaluating organizational change processes in INGOs.

Continue the conversation with us!  Tosca tmbruno@maxwell.syr.edu