Learning Loops

To understand impact we must understand lived experience, not for its own sake, but to improve it. Keystone Accountability has spent the last 10 years working with multiple partners in many different contexts to develop an easy yet continuous and systematic way of understanding and improving stakeholder experience together in small, manageable and collaborative steps. We call the approach Constituent Voice (CV).

CV uses a small number of standard and comparable questions in frequent, low-cost micro-surveys to generate real-time, comparable and actionable data about performance and impact. And then uses this data to foster inclusive problem-solving conversations that deepen insights, strengthen relationships, and inform timely, collaborative action.

CV was pioneered in International Development Assistance (Aid), where mutual-interest relationships are essential for any collaborative effort to improve peoples’ lives – however formal or informal that collaborative purpose might be. 

But CV is effective wherever relationships matter, and where: 1)  the long-term success of a service or business depends on the cooperation and agency of staff, of partners, or of extended communities; and, 2) we are grappling with complex, systemic change processes that require aligning the efforts of multiple actors, fostering local ownership and agency, relationships of mutual accountability and a responsive and adaptive management practice.

Most traditional approaches to social impact measurement do not support agile, responsive, or collaborative action. They are too costly, depend too much on outside experts, are too time- and resource-intensive and the learning they offer often comes too late. They tend to see impact as an ‘end result’ rather than as the constantly changing quality of the relationships and the process that takes us there. They don’t help us learn and improve as we go. 

CV builds on the relatively recent insight of leading consumer facing businesses – that the best way of understanding and improving customer experience is not through long and detailed market research, but through timely and responsive feedback loops based on a few key signals of a customer’s experience and perception.

We have learnt, for example, responses to a single standard question like ‘When you talk to people from {{org}}, do you feel completely free to speak openly and say whatever you want to?’ provide a good indicator of the levels of transparency, confidence, and trust between an organization and its staff, its frontline partners and even the community constituents with which it engages. But more importantly, the anonymously collected survey data creates a secure platform for an open conversation about the relationship, the opportunity to identify problems, re-frame and manage expectations, and to agree on mutual actions to improve the relationship.

The experience of many clients has demonstrated that quick and simple ways of measuring and engaging with the relationships that underpin success is the easiest, most cost-effective, and most powerful way of making social impact interventions more evidence-based, more collaborative, more adaptable, more accountable to those who live the problems being addressed, and ultimately more effective.

1. Simplicity. A key to CV is keeping measurement simple but meaningful. Focusing on fewer questions (usually asking only 4 to 5 at one time), asked repeatedly and at scale allows more frequent engagement without overloading the company or its stakeholders. It slashes data collection costs. Perhaps most importantly, it simplifies the interpreting of data without making it simplistic. It is this simplicity that fosters problem-solving conversations that lead to shared insight and action for both internal and external stakeholders.

2. Inclusivity. Using micro-surveys to collect feedback more or less continuously over a longer time period, means that a company can aim for much higher response rates than those typically achieved in standard ‘research’ survey samples. It is not unusual for organizations to collect feedback from 50-80% of a target population (e.g. workers in several factories or community members in a mining region) at a fraction of the cost of traditional approaches.  Companies can make use of existing opportunities to collect data and to engage with their stakeholders. For example, instead of hiring an independent research firm to survey a small sample of respondents in a single window of time, a company can use local data collectors or ‘always-on’ tablets or kiosks in places where stakeholders gather (such as canteens, community halls or events).

High participation rates in data collection mean that companies are less likely to miss experiences from segments of a population that can offer important, even if outlier, perspectives about company practices. Participation in sense-making sessions feeds a sense of ‘we are figuring this out together’. More people feel engaged and motivated to participate in the interpretation of data and the search for creative solutions – especially if feedback is linked to material improvements and widely communicated. 

3. Timeliness and actionability. Annual survey cycles take too long for collaborative learning and problem-solving. Frequent surveys, quick data analysis and easily deployed communication bring systematic data, insight and action close to real time. Complex issues can be broken up into manageable parts, and progress in solving them can be tracked systematically as well. Through an ‘improvement dashboard’ which combines key data points into a live metric of how well stakeholders feel the company is responding to their concerns, a company can convene internal and external dialogues to find ways to improve their programs.  This provides a good starting point to integrating data, findings and actions taken in response to the feedback into organizational knowledge and culture.

4. Low cost and easy to operate. For those companies that opt to use it, Keystone offers a secure, purpose-designed web platform called the Feedback Commons™ that enables companies and organizations, with remote advice and support, to operate and manage their CV learning loops using in-house staff. This greatly reduces costs and ensures the integration of the learning loops with company operations and culture. Through a very simple and intuitive interface, authorised staff can build and administer micro-surveys, analyze results at any time, and create clear, shareable reports that almost anyone can understand and engage with.

Data collection and dialogue should occur as close to the ground as possible. But using standardized questions enables organizations to aggregate data and observe patterns at a system-level in real time across the organization, its geographies or supply chains. In addition, where other companies collect similar data, Keystone can provide companies with peer, industry or sector benchmarks made up of anonymized comparable data.

Here are some helpful resources:

1. https://shiftproject.org/resource/cultivating-voice-in-relationships/
2. Andre Proctor and David Bonbright, Constituent Voice: Turning feedback into data, voice and solutions in Generation Impact: International perspectives on Impact Accounting, edited by  Adam Richards and Jeremy Nicholls. Emerald Publishing, 2021.

Prepared by Andre Proctor from Keystone Accountability.