What is Outcome Harvesting and why do we often propose it as an approach to project evaluations?
Rather than asking “did the project implement its planned activities?”, Outcome Harvesting asks a different question: “what actually changed, and how did this project contribute to that change?”
In brief: Outcome Harvesting helps to:
– Focus on the changes brought about by the project (outcomes), rather than activities implemented and outputs.
– Understand the process of change and how each outcome contributes to this change, rather than simply to accumulate a list of results.
Outcome Harvesting is an evaluation methodology in which the evaluators, together with the evaluation users define together a set of desired outcomes of the intervention, and then through data collection, work backwards to determine whether and how the intervention has contributed to these outcomes.
Who are the evaluation users? Usually, it’s the organisation commissioning the evaluation, but also their donor, members, grant recipients or partners.
Outcome Harvesting is a highly participatory process in which all of these actors are essential contributors to determining what types of outcomes were achieved and how.
We’ve found Outcome Harvesting to be a particularly useful approach in the evaluation of projects which:
– Operate in a complex programming context, with multiple actors and factors influencing the project’s desired outcomes simultaneously
– In which changes are often incremental and less visible,
– The types of outcomes sought often involve changes in attitudes and behavior of different stakeholders, and
– The relationship between cause and effect are not always clear
Resource: A brief but very informative overview of the Outcome Harvesting methodology was developed by Better Evaluation and can be found at this link: https://www.betterevaluation.org/en/plan/approach/outcome_harvesting#OH_whentouse
Here are a few examples of projects for which we’ve successfuly used Outcome Harvesting:
– A project funded through the European Commision’s DEAR (Development Education and Awareness Raising) programme, in which international development federations of NGOs from four countries were advocating for policies aligned with the SDGs
– A project supporting the resilience of NGOs, human rights advocates and media organisations in the Eastern Partnership countries through small grants and capacity building activities
– A project developing the ecosystem for social entrepreneurship and impact investing and in Ukraine, Armenia, Georgia, Moldova and Azerbaijan
– Several different projects supporting the educational progress of children and youth at-risk of early school leaving in Romania
In practice
The first project mentioned above presented a challenge that is typical with evaluating advocacy and policy-oriented projects. There were four federations of NGOs, across four countries were advocating for policies aligned with the SDGs. There’s no single decision-maker to influence, no straight line between what the project does and what changes as a result.
The project’s goals were structured in a standard logframe, with 3 major objectives and 6 corresponding expected results along the lines of: increased capacity of CSOs, better networking in the sector, more public awareness of SDGs, and regular interactions between organisations and policy makers.
In the first part of the Outcome Harvesting process, we worked with the project partners to develop a set of 15 desired outcomes, all related to the logframe objectives, but defined in terms of observable changes for different categories of actors. Then, through interviews with project stakeholders, we tried to make sense of these outcomes. The result:
🔸 Some outcomes were validated as imagined, and we could document what the change looked like in practice.
🔸 Several outcome formulations were changed during the validation process because we found out that the results were present, but different, more nuanced than planned.
🔸 A few outcomes proved to be too broadly formulated, and were split or combined
🔸 And, not least, we identified from interviews 5 additional project outcomes that were not initially foreseen by the partners.
A few concluding thoughts about the methodology.
Outcome Harvesting is a particularly good fit when projects are complex, with many actors involved at the same time, progress is slow and incremental, when you’re looking for shifts in attitudes, relationships, or behaviours, and cause and effect aren’t easy to untangle.
It is crucial that the initial outcome formulation is done together with the partners. Even then, the formulations are not failproof, but they dramatically increase the quality of evaluation.
In the end, it’s not the easiest methodology to implement, but in complex contexts like this one, it tends to give you something rare: an honest account of what actually happened.