24 October 2018, Kampala, Uganda – In order to increase resilience in the agriculture sectors, it is important to understand the impact of different climate change adaptation options. With the help of impact evaluation tools, countries can select adaptation interventions in an evidence-based manner, and thus advance their adaptation planning processes. These efforts ultimately contribute to achieving goals outlined in the Nationally Determined Contributions under the Paris Agreement and the 2030 Agenda for Sustainable Development. 

With the aim to build capacity on impact evaluation exercises, Uganda’s Ministry of Agriculture hosted a three-day capacity building workshop in Kampala, 24 - 26 October 2018. The event was organised as part of the Integrating Agriculture in National Adaptation Plans programme (NAP-Ag), co-led by the Food and Agriculture Organization of the United Nations (FAO) and the United Nations Development Programme (UNDP), with funding from the German Federal Ministry for the Environment, Nature Conservation and Nuclear Safety (BMU).

Taking action on climate change is key for building community resilience and achieving Uganda’s Vison 2040 and its NDC, as well as the 2030 Agenda and the 17 Sustainable Development Goals (SDGs), particularly SDG 13 on climate action. In June 2018, Uganda became the first country in Africa to sign the Partnership Plan for NDCs to achieve national climate goals as part of its obligations to the Paris Agreement. 

The value of impact evaluation

Understanding the impacts of adaptation interventions is essential in scaling, mainstreaming and accelerating climate actions worldwide. It is especially useful when countries test innovative climate change adaptation options that seem promising in theory but have little hard evidence to support them.

This is the case for many climate change adaptation projects – the technology being deployed as an adaptation option may be old, and the outcome can vary based on the changing climate, new technologies, new policies or new implementation techniques.

One key component of impact evaluation is that it can inform climate-smart policies. Policy makers who want to use evidence to back their decision-making on adaptation options and inform policies need information on a variety of questions.

The need for impact evaluation tools in Uganda’s agriculture space

During the workshop it was highlighted that Uganda loses approximately 800,000 hectares of crops each year to climate change, and interventions in the agriculture sector would be closely linked to Uganda’s NDC and global SDG targets.  

The purpose of the workshop was to enable key stakeholders to use impact evaluation methods to assess project implementation in real time. The event brought together 20 participants, of which seven were female, from local governments, NGOs, district level governments, and international agencies.

“This is extremely useful in programming future climate actions and creating positive, virtuous feedback loops, where we constantly learn and adjust approaches to ensure more effective climate actions,” said Dr. Babatunde Abidoye, a senior applied environmental economist working with UNDP.

Participants of the workshops used a case study based on the UNDP-supported project “Building Resilient Communities, Wetland Ecosystems and Associated Catchments in Uganda” as an illustration. This recently approved Green Climate Fund (GCF)-financed project will support the Government of Uganda in the management of critical wetlands affected by climate change.

Eight steps of impact evaluation

Dr. Abidoye led the workshop, guiding participants through the eight most crucial steps in conducting any impact evaluation:

1.       What is the primary policy, program or intervention of interest?

2.       What is the causal relationship between this policy and the outcomes of interest (theory of change)?

3.       What ideal experiment could be used to measure the impact of the program?

4.       Do data on the policy intervention exist or do they need to be generated? 

5.       What econometric technique should be used to remove selection bias and identify the causal relationship?

6.       Is the impact statistically and economically significant?

7.       How does the impact compare with the costs?

8.       How should the results be communicated with policymakers and other stakeholders?

Applying the skills to the day-to-day: 

“My main takeaway was how to test innovative climate change adaptation options at smallholder farm level, the ability of the IE [impact evaluation] tool to be dynamic enough to enable on-course corrections for current projects, and the value of establishing counterfactuals in impact evaluations. It provided a very useful perspective on evaluating a project to compare how project beneficiaries would have fared without the intervention...” -  Mr. Andrew Atingi, of FAO-Uganda, Training Participant

Another participant, Kyobutungi R .Winie from the District Natural Resource Office, Gomba District, said, “I realised that many project developers were not considering the aspect of cost- impact analysis, this has resulted into many projects being implemented with no tangible results on ground. My take away lesson is its better not to take up the project whose cost is much higher than the impact.”

The training combined the theory of evaluations, with group discussions. The following key topics were covered in the workshop: understanding the purpose of and how to develop a theory of change; what impact evaluation is and how to design it; the different form of impact evaluation; and, calculation of sampling size and the challenges faced when implementing impact evaluation.

The workshop ended with the participants presenting a theory of change and impact evaluation design for the project they selected. While concluding, it was decided to host another capacity workshop on conducting cost-benefit analysis (CBA) in Uganda.

Learn more about Using impact evaluation to improve policymaking for climate change adaptation in the agriculture sectors