Under attack about their legitimacy and accountability, CSOs defending citizens’ rights have to maintain credibility among their constituency and the wider public. Tamryn Lee Fourie and Merle Rutz from CIVICUS guide us through the organisation’s journey to ensure strong accountability and transparency to its members, beneficiaries and donors. The present article is part of the magazine Activizenship #3 (launched on 11 February).
-
Civil society accountability in times of declining public trust:
Populist tendencies are on the rise in Europe and beyond. At the same time, there is a harmful lack of trust growing in institutions, including in those of governance and politics, and the ways in which democracy is being practised. As indicated by the Edelman Trust Barometer, this trust deficit (both from the informed public as well as from the general population) also extends, albeit often to a lesser extent, to sectors essential to for holding governments and politicians to account, including civil society organisations (CSOs) and the media. The GlobeScan Foundation’s 2016 results indicate that this mistrust in CSOs includes associating them with self-interest, corruption, unethical behaviour, no real impact, misuse of financial aid, and lack of transparency.
One response to this is that CSO compliance with (voluntary) accountability frameworks such as Accountable Now or the Core Humanitarian Standard is steadily increasing. Unfortunately, as shown by the Edelman Trust Barometer, trust levels are not increasing as a result of this (voluntary) compliance. What does this say about the perception of CSOs’ accountability and transparency efforts? Are these accountability initiatives the right fit for the current global challenges? Are there mechanisms in place, both within individual CSOs and across the sector, to shift the accountability discussion away from a “tick box” compliance exercise into a dynamic values discussion? This should be the time to reflect on unequal power relationships in the contexts of fake news, safeguarding scandals and smear campaigns and one’s actual impact. However, there is “a danger of creating more regulations as a ‘quick fix’ solution to appease donors”.
Over the past year, CIVICUS’ recurring thematic focus on Reimagining Democracy has revealed that some CSOs may consider themselves standing for democratic values, but are liable to be challenged as to how democratic we are internally in practice (and this does not refer to just policies in place). For example, many of us are based in capital cities, and so are not good at reaching and understanding what is happening in rural and isolated areas, from where right-wing populism draws its bedrock support. So, we find it hard to talk to the people that we are supposed to be supporting. Based on their findings, GlobeScan also suggests that championing the public’s “democratic rights space” plays in the civil society sector’s strength in upholding widely-supported principles that will thus increase trust. However, a concern with democratic freedoms and democratic practice is that these are not seen as a mainstream focus for many CSOs.
At CIVICUS, we have taken the opportunity of a new strategic period to relook at what accountability means to us. Accountability does not end with a report or self-assessment, but rather it is an ongoing constructive relationship with stakeholder groups that improves the agency and credibility of CSOs.
-
CIVICUS’ approach to transparency & accountability:
2.1 Designing our accountability framework:
In response to these changing contexts, CIVICUS elevated its accountability agenda, moving away from a technical Monitoring and Evaluating (M&E) emphasis to embedding accountability as a culture and strategic enabler.
As a result, CIVICUS’ new accountability framework was developed alongside its Strategic Plan 2017-2022 to (1) help consistently and systematically track the organisation’s progress and impact against the new strategy, (2) meet accountability commitments and (3) enable organisational learning. It is grounded in two schools of thought – utilisation-focussed evaluation and developmental evaluation theory. The starting point is recognising that in complex environments, where social change is difficult to measure and attribute to one single effort, evaluation needs to be purpose-driven and enhance the likelihood to inform decisions.
We have also shifted away from traditional notions of measurement of success as we acknowledge that the markers of success vary depending on the levels of the implementation of the programme. Neither outputs, nor outcomes, nor impact markers are the most important measurements in our story of success because each depends on the other.
Our understanding is that social change does not occur in a vacuum, and every effort to affect social change requires us to tell the full story of when/how change happened (or did not). Quantifiable indicators are important for more immediate and intermediate changes. However, longer-term changes or measures of success may be, in some cases, less quantifiable because they involve changes in social actors, governments, activists and citizens. These behavioural changes are longer lasting and, ultimately, what we are most interested in capturing, measuring and learning to inform our decisions.
As a break from the norm, we have adopted what we call Critical Learning Questions for measuring these longer-term changes. The below table (Table 1) highlights the difference between indicative impact indicators and our alternative, utilisation-focused Critical Learning Questions, which we propose will replace non-utilitarian “markers of success”.
At CIVICUS, accountability and transparency are more than the frameworks, policies and documents we put in place. More and more, we are seeing accountability becoming a dynamic two-way relationship with stakeholders in order to ensure people’s participation and the systematic use of feedback in decision making at all levels. This type of accountability, called dynamic accountability, is how we are approaching implementing the above framework, to ensure that our well-designed and thought-through intentions are embodied in our programmes’ approaches, communications, reports and decision-making processes.
2.2 Implementing our accountability framework:
In order to breathe life into the accountability framework, CIVICUS has put in place several processes and systems to make sure we not only meet our commitments but also use this framework to embed a culture of values-based accountability supported by robust data-driven decision making. During the first year of implementing the accountability framework, we put in place a central system that houses all our programmatic and organisational performance-based data (2.2.1). This was supported by an organisational-wide reflection process to discuss progress against year 1 of our strategy and how to course-correct (2.2.2). We also launched our online feedback form to enable us to gather feedback from a wider range of stakeholders (2.2.3) while exploring and testing new hypothesis between accountability, data, transparency and resilience. These are expanded upon in the following sections.
2.2.1 Being more systematic in capturing performance data
CIVICUS is fortunate to be in a very data-rich environment with data being generated through online platforms, participant evaluations etc. However, we needed a common platform to capture results-related data to more holistically analyse our activities, outputs and outcomes in relation to our new strategic plan – which is a core requirement of our accountability framework. As such, CIVICUS recently implemented a system called DevResults, which is a secure web-based monitoring, evaluation and portfolio management system that tracks programme, progress and organisational results data.
Our DevResults platform is aligned with our strategic results framework and was pre-populated with planned activities and associated indicators for the first year of our strategy. CIVICUS has now uploaded the 2017-2018 results against the first year of the five-year strategy on DevResults. Analysis of the data gave us the opportunity to generate many learnings, as well as the opportunity to reflect on the process of centrally capturing the data.
Key insight: When we began to capture and visualise initial pilot datasets, we were already able to see, for example, how limited resources for translations hindered our engagement in francophone Africa and Latin America. We have used this to motivate for increased funds in this area and already have been able to launch a Spanish version of our website.
Key insight: As part of our diversity and inclusion agenda, we are also able to capture disaggregated data for gender and youth. And even though our staff compliment shows that women are well represented in CIVICUS at all levels, when we started to track the gender of those writing our opinion pieces (as one form of our external representation) we see that the authors are mostly men. We were also able to course correct here as we began to be more intentional around creating capacity and encouraging women in our organisation to pick up the pen.
This data feeds into a bi-annual impact reflection process (see section 2.2.2) which focuses on reflecting on performance against CIVICUS’ new strategy. In addition, CIVICUS is working on transitioning current Board reporting dashboards onto DevResults. It is envisioned that these dashboards will be used to update the CIVICUS Board on overall performance and organisational health on a bi-annual basis.
2.2.2 Creating a culture of evidence-based reflection and decision making
For us to embed data-driven decision making in CIVICUS, we needed to ensure that there was an opportunity for all staff to engage in the analysis process of the data and inform how this information is presented to management, Board and external stakeholders. CIVICUS piloted this through a new Impact Reflection Process (see Figure 2) which forms the basis of evidencing, analysing and generating learning in relation to our accountability framework and performance story (i.e. our strategy results framework). This process took place for the first time in August 2018 based on the annual results from 1 July 2017 to 30 June 2018 captured on DevResults, with the intention to:
- Encourage data-driven performance discussions based on annual results captured on DevResults (and other sources as well)
- Allow for objective discussions to be held at various levels of the organisation on what has/has not been achieved, why and how we need to change and/or strategically course correct
- Identify content/discuss/analyse our results for inclusion in CIVICUS’ annual reporting to the Senior Leadership Team (SLT), donors, Board, members etc.
In this first round of impact reflection discussions, we held 11 sessions (facilitated by the Impact and Accountability Cluster) across our four Strategic Goals, where teams were able to analyse and reflect on their results in relation to plans to gain a sense of progress in year 1 of our strategy. During this time, we also began our first attempt at answering our Critical Learning Questions and testing whether these are in fact the right questions for us to be answering. The results of these discussions were synthesised, and the cross-cutting themes/trends are currently being presented/further discussed at the programme and operations management forums, SLT, staff meetings and Board committees between September and October.
The outcomes of this process have enabled us to do the following:
- Include deeper analysis of our results in our annual report for donors, Board and Annual General Meeting (in fact the synthesis has served as a critical input into developing the Annual Report 2017-18)
- Refine objectives, activities and indicators for year 2 strategy implementation as required
- Test if our Critical Learning Questions are the “right” questions to help us understand our impact
- Gather initial “baseline” analysis around our Critical Learning Questions
- Strengthen cross-cluster collaboration and identifying synergies across different work streams
Key insight: Each of the discussions took on a life of their own, and we generated large amounts of qualitative data to complement what we gathered on DevResults. However, we were unable in the first series of discussions to unpack our Critical Learning Questions in relation to the impact of our strategy. This is an area we would look to improve in the next round of discussions. We also generated several suggested actions that we need to channel through various decision-making processes to ensure that these discussions do not end as “talk shops” and that we can feedback to our staff on what has changed.
2.2.3 Proactively engaging with and feeding back to stakeholders at all levels
Along with our impact reflection process, we have many other mechanisms to generate feedback and engage with members, donors and other constituents. The CIVICUS Board approved a new Feedback Response Policy in January 2018, which facilitates external complaints, advice and suggestions from members, partners and the general public. Feedback can be filed through an online form (ensuring anonymity if needed) or via feedback@civicus.org. In addition, external feedback and complaints can be filed via the Accountable Now mechanism. Members and partners are furthermore invited to provide regular inputs via the Annual General Meeting, Annual Constituency Survey, Membership Survey, event feedback forms and project evaluations. Exposing ourselves to critical or dissenting voices is important so that we do not risk locking ourselves in echo chambers.
Example: To increase the response rate to our annual member constituency survey we used a shorter Net Promoter Score (NPS) style survey as an indication of member satisfaction in 2018. The survey was shared in English, French and Spanish and we received a total of over 442 responses with an NPS score of 42.[1]
The primary objective of this survey was to boost members feedback (quantity and quality). The Net Promoter Score served the purpose better than traditional survey approaches. When going with the NPS, we made sure to address those additional questions addressed in our previous traditional surveys through the follow-up interview. This includes, for instance, what members value but do not want to get involved in (e.g. because it is not aligned with their expertise), as well as questions of inclusion (e.g. do you feel the voice of youth and gender is represented, as an older or younger person/ men/women/non-binary).
Example: Another way that we engage stakeholders is through our donor coordination group whom we meet with every 6 months. The purpose of these meetings is to actively engage with our core donors on strategy, approach and progress and to allow the opportunities for donors to have robust discussions on our reports. We have also used this opportunity to create an alignment of reporting requirements across our donors, and all our core donors have adopted our accountability and reporting framework which allows us to focus our efforts on impact measurement against our strategy rather than creating too many additional M&E requirements and frameworks at the project level.
CIVICUS is confident that having well-designed and responsive mechanisms for handling external and internal feedback (including suggestions, complaints, or positive feedback) will improve the quality of its work, enhance trust and confidence of stakeholders, identify areas of work which need strengthening, and ensure that CIVICUS learns from feedback provided through such a process — thereby embedding a culture of values-based accountability rather than one-directional reporting.
2.2.4 Testing new hypothesis for accountability
As part of CIVICUS’ contribution to the civil society sector, we not only aim to share our accountability learnings but also push the boundary in terms of testing new tools, approaches and assumptions in relation to accountability. We are currently investing in various innovative approaches to accountability which manifest themselves in different pilot projects. Examples of these projects include:
Resilient Roots: This is a pilot project in collaboration with Keystone Accountability and Accountable Now where we aim to test whether organisations who are more accountable and responsive to their roots – namely, their primary constituents – are more resilient against external threats. We have begun working with 15 pilot organisations, and our initial learnings thus far can be found here.
Innovation for Change: This project aims to create a fully decentralised network of regional hubs to work on social innovation addressing local, national and regional civic space freedoms. Through our developmental approach, we are aiming to understand how such networks are structured to respond to contextual circumstances and develop innovative projects to respond to civic space violations, supported by a network of members.
DataShift: Through this project, CIVICUS is working with local partners to deepen the use of data to increase the accountability of key decision-makers at local, national, regional and international levels to meet SDG targets.
These projects illustrate our commitment to contributing to thought leadership and pushing the bounds in relation to deeper, value-based accountability, both internally and demonstrated through our programme initiatives.
-
Lessons learnt:
In line with good practice, CIVICUS has begun to systematically capture and synthesise lessons learnt, successes and challenges through our impact reflection process which took place for the first time earlier this year (refer to section 2.2.2). Through this process, we were able to analyse, discuss and take action on the information that was gathered through the annual data collection process in relation to progress against the first year of our strategy. The below covers several of the key lessons as they relate to the implementation of our accountability framework:
- We have laid the groundwork by implementing systems such as DevResults to house our programmatic and organisational results data centrally but need to do more to ensure that systematised relationship and knowledge management are further embedded in our culture, e.g. incentivising sharing between projects, teams etc.
- Although we have made progress to meet our accountability commitments, we have not made as much progress in terms of measuring the impact of our work, especially in terms of interrogating our theory of change and why we believe that our approach is impactful;
- We need to continue to improve the quality of our data that is generated across project, programme and organisational levels to inform decisions to understand better the extent to which our accountability framework is evidencing our progress against strategy;
- Our accountability framework has allowed us to strengthen cross-cluster collaboration within CIVICUS as we aim to answer our broader critical learning questions to inform the overall strategy and approach. Further alignment of our planning and reporting processes will be necessary to foster collaboration both within CIVICUS and with our members and partners;
- Finally, we need to work with CIVICUS leadership to identify organisational priorities and set targets against these to help us better understand our progress and effectiveness and demonstrate long-term impact.
With these lessons and many others, we are striving to improve how we implement our accountability framework going forward and will continue to share our experiences with our members and the wider sector in the hopes of contributing to a more effective, innovative and accountable civil society that enjoys greater trust from the public.
[1] A positive NPS (i.e. one that is higher than zero) is generally deemed good, and an NPS of +50 is generally deemed excellent.