As UK Aid’s £250 million flagship forest governance programme looks to its completion in 2023, its Policy Evaluation and Learning Team is thinking about what the next phase of this work could look like, breaking with traditional M&E to provide timely, on-demand policy studies focusing on core themes of the programme’s theory of change.

Since 2011, the Forest Governance, Markets and Climate (FGMC) programme has worked around the world to strengthen sustainable and legal forest management. The programme has focused largely on support to the EU’s Forest Law Enforcement, Governance and Trade (FLEGT) processes and on promoting legal trade in timber and wood products. In its final phase (it has recently been extended to 2023), greater emphasis is being considered for selected ‘forest-risk’ commodities – forest goods and raw materials whose extraction contributes significantly to tropical deforestation and degradation.

After five years of monitoring and evaluating FGMC through an Independent Review Team, in 2019 the programme embarked on a new approach that focuses more on policy studies, generating evidence and learning and assessing FGMC’s contribution at the outcome level of its theory of change. With the end of the programme in sight, there was a greater need to take stock of overall achievements to guide the development of a new, post-FGMC programme. This would involve assessing FGMC’s results in terms of market and governance transformation, and the validity of the FGMC theory of change and approach; and better understanding under what conditions, and to what extent, changes have led to better impacts.

Rather than a traditional, external and independent approach to evaluation, therefore, focused on reporting and accountability, a Policy Evaluation and Learning Team (PELT) has been appointed to work collaboratively with the Foreign, Commonwealth and Development Office (FCDO) and the Programme Management Support Team to build learning and evidence. The PELT will complete some 20 policy studies and briefings to guide, inform, challenge and influence future programme decisions regarding approaches, outcomes, impacts, coherence and the internal validity of the FGMC programme and its successor.

Triple Line leads the PELT, drawing on our five years of engagement with FGMC, the knowledge and understanding we have of the programme, and the strong and constructive relationships we have established in that time with FCDO and FGMC implementing partners.

With our emphasis on learning, PELT has been different from the outset from most long-term monitoring, evaluation and learning (MEL) assignments in several ways. Its rolling and flexible programme of timely, demand-driven studies and reviews is responsive to the emerging needs of the FGMC programme – so as to be as useful as possible to the needs of programme managers, informing both design and implementation. The research is also forward- rather than backward-looking as our objective is not primarily to provide accountability for performance, but rather to guide future programming choices and design, and to increase effectiveness going forward. In the initial stage we have identified four themes:

• Behaviour change among key actors engaged in the forest governance policy processes,
• Economics and distribution of benefits from the introduction of Voluntary Partnership Agreements (VPAs) and other FLEGT-related measures,
• How FGMC support has influenced public decision making at national and international levels, and
• Governance and rule-of-law outcomes that have been realised in producer countries with the help of FGMC support.

The Qualitative Impact Protocol is a non-experimental, goal free approach that helps address attribution challenges when assessing behaviour change and governance

A particular challenge in assessing behaviour change and governance is to be able to attribute outcomes back to specific interventions. We have been using the Qualitative Impact Protocol (QuIP), a non-experimental, goal free approach which uses rigorous, purposive sampling and aims to sample to ‘saturation’ – at which point each new story adds little to the existing information. It also tackles bias in data collection and adds rigour to the analysis of qualitative data. In Ghana, fieldwork was carried out by local researchers (supervised by the University of Bath) who were ‘blindfolded’ – told nothing about the programme whose impacts were being investigated – in order to remove interviewer bias; and asked to assess how (if at all) forest governance, trade and legality had changed over the past five to 10 years.

Over the course of 2020, we have produced nine study reports and five briefing notes, as well as a summary findings report, as well as a number of workshops to present findings and learning. One key outcome has been the documentation of deliberative national-level public policy and decision-making. By placing emphasis on securing a quality process (with less focus on directing outcomes), FGMC has ensured that changes in law and policy are legitimate, respected and implemented. This approach differs markedly from other forest governance programmes where pre-determined policy goals are advocated by international partners, with less attention to ensuring voice and representation of local interests.

Credit: title photos by Tom Blomley