Check out a nonprofit’s website or its annual report. You’re likely to find information on how many clients the organization serves, the quantity of programs and services it offers, and how important its programs and services are to the community. What you may not see is the result or outcome of these programs and services.
In its simplest form, impact measurement is evaluating if an organization’s programs and services are working, helping it achieve its goals and objectives. In other words, what happened to clients after they received the organization’s help? How were they better? How did their circumstance improve? What can the organization do to make the program better?
Success Is Not Always Easy to Measure
Many organizations have goals that are inspirational but not easily measured. Words like encourage, uplift, empower, and inspire can be difficult to quantify, yet these are found in the mission and vision statements of many nonprofits. Here are some examples:
- MISSION exists to inspire everyone who has stared down cancer to live a fuller life, with newfound strength and purpose.1
- YWCA USA is dedicated to eliminating racism, empowering women, and promoting peace, justice, freedom, and dignity for all.2
- The Obama Foundation’s mission is to inspire, empower, and connect people to change their world.3
Health and human service nonprofits often focus on change— changing lives or situations. The challenge, however, is that measuring change is rarely clean and simple. Consider the following reasons change is difficult to measure:
- Change is subjective. Different people may have different perceptions, expectations, and experiences of change. What may seem like a positive change for one person may be a negative change for another.
- Change is complex. Change often affects multiple aspects of an organization or a person’s life, such as culture, processes, relationships, behaviors, attitudes, values, etc. Measuring the impact of change on each of these aspects can require multiple methods and indicators. Even then, it may not capture the various interrelationships and synergies included.
- Change is dynamic. Change is rarely a one-time event, but more often a process that evolves over time. The effects of change may not be immediate and may emerge gradually or unpredictably. Measuring change at a single point in time may not reflect the long-term or cumulative change that occurs.
- Change is contextual. Change does not happen in a vacuum, but in response to internal and external factors that influence its direction and pace. The same change may have different results in different situations or environments. Measuring change without considering the context may not account for the variables and constraints that affect change.
Why Measuring Matters
In a world of growing needs and limited resources, funders want more than heart-stirring stories. They want to invest in well-managed organizations that can demonstrate meaningful, lasting impact.
Measuring impact is important to supporters, mission partners, the people your organization serves, and your staff and board.
The pandemic reminded us that we live in a rapidly changing environment. As the world evolves, so do communities and the people who live there. Assessing impact helps your board and staff to be responsive, make real-time adjustments, change tactics, and allocate proper resources where needed.
What to Measure
Resist the urge to measure everything, and don’t just settle for information that’s easy to measure. Clarify what’s important to measure and then evaluate how best to do so using both qualitative and quantitative means.
Involve program staff, board members, and funders to find out which metrics they want to see to know if progress is being made. Also, consider getting input from program participants regarding the effectiveness of your organization’s services.
- Define – Define what you are trying to achieve or change and set indicators of success.
- Develop – Develop instruments and processes to collect information. Gather and input data.
- Analyze – Analyze data, generate reports, and look for patterns or significant findings. Use findings to help identify needed adjustments or changes.
- Modify – Modify existing processes. Develop iterations of the new process or system. Implement in stages and monitor.
- Measure – Measure indicators. Compare to previous findings to evaluate effectiveness of change.
A useful tool in impact measurement is a Theory of Change. This is a visual illustration of your organization’s actions and expected changes. Theories of Change are great for explaining how your programs and services create results and what resources are needed to accomplish the ultimate goal.
The Annie E. Casey Foundation provides a useful guide on how to develop a Theory of Change.4
Another pillar of outcome measurement is a Logic Model. Similar to a Theory of Change, a Logic Model visualizes the change process. This tool also includes information on resources, activities, and outcomes.
| Inputs | Activities | Outputs | Expected Short-Term Outcomes | Expected Long-Term Outcomes | Expected Ultimate Impact |
|---|---|---|---|---|---|
| To accomplish our set of activities, we will need these resources | What we do with resources; activities that deliver outputs | Who we reach and how they participate: services delivered, workshops, trainings, etc | We expect that if completed or ongoing, these activities will lead to the following changes immediately or up to one year | We expect that if completed or ongoing, these activities will lead to the following changes in four to six years | We expect that if completed, these activities will lead to the following changes in seven to 10 years |
Data Collection Methods
Sophisticated data collection techniques are not always needed. To get started, consider using one of these common methods:
- Pre- & Post-Program Evaluations – These evaluations can help assess the condition of a person or the extent of a problem. They are typically given upon entry into a program or service. Using preset indicators and scales, an organization employs these tests to evaluate the extent of a particular condition. The post-program evaluation uses the same questions or indicators to measure how the individual improved as a result of program participation.
- Client Satisfaction Surveys – Organizations can use surveys to gain insights from program participants. The surveys can be distributed in person or electronically. The surveys ask questions about service satisfaction, interaction with staff or volunteers, facilities, and perceptions of benefits and results.
Obtaining feedback from participants can be challenging. In some cases, it can be difficult to track down individuals once they leave your programs. For that reason, it’s recommended to capture feedback before they leave, possibly on the final night of the program.
Caution
Don’t try to measure outcomes beyond your control. For instance, organizations that provide client referrals, but not direct services, shouldn’t try to measure how effective other organizations’ programs were for their referrals. Their role is to provide clients with appropriate referrals. Anything beyond that is out of their control and likely won’t factor into their outcome assessment.
Outcome measures remind current and prospective supporters that your organization is truly changing the world, and you have the proof! These results can be used in marketing, fundraising, and advocacy efforts.
For more information, please reach out to a professional at Forvis Mazars.