Bowy den Braber is a PhD student in the Department of Animal and Plant Sciences, looking at the social and environmental outcomes of protected areas in the Tropics and Sub-tropics
The Sustainable Development Goals (SDGs) herald a new phase for international development and cooperation. A crucial element of this will be assessing progress towards the target related indicators proposed by the Statistical Commission of the UN’s Economic and Social Council. There are over 200 of these indicators. A plethora of interventions are already in place that aim to enhance sustainable development, and many more are being designed with a specific focus on SDGs. It will be extremely challenging to meet the SDG targets, and some may be impossible to achieve. To have a fighting chance of success it is crucial that progress towards the indicators and the effectiveness of interventions are analysed as robustly as data allow. Achieving this is not an easy task. A team of global experts organised by Drs Johan Oldekop and Karl Evans with funding from University of Sheffield’s Institute for International Development (SIID) recently assembled in Sheffield to discuss this challenge.
Tracking progress in SDGs through monitoring temporal change in indicators can be extremely problematic. Indicators may be poorly designed to monitor goals – e.g. tracking progress in biodiversity conservation via the number of protected areas won’t be very helpful if new protected areas are established in biodiversity deserts (where land is often cheap). When it comes to impact evaluation documenting temporal trends in an indicator before and after an intervention is not sufficient – correlation does not prove causation. Similarly, lack of correlation doesn’t always mean lack of effect, especially when the only monitoring data are collected at a spatial resolution that doesn’t match that of the focal intervention.
With limited resources available, impact evaluation of interventions using observation data is becoming an increasingly popular tool to assess whether interventions work to reduce both poverty and environmental degradation – the core twin themes of the SDGs. Interventions that are designed in an office might fail because they can make wrong assumptions about people’s behavioural response in adopting and using the intervention. A classic example is the distribution of cooking stoves in the Global South that reduce use of wood (protecting natural resources) and air pollution (improving human health). Yet these are often sitting unused in a corner, because they are not fit for local purposes. Another example, a policy to expand the protected area network might lead countries to select areas of low economic value as they can’t be used for agriculture or mining but are of low priority for biodiversity protection. Interventions can also lead to unintended perverse outcomes. Poorly designed protected areas may for example increase poverty of local people because they are no longer allowed to use and access natural resources.
The past few years have seen a significant rise in the availability of environmental and socioeconomic datasets and advanced quasi-experimental statistical impact evaluation methods. These methods aim to mimic randomized controlled trials by selecting groups of treatment and controls that are a similar as possible out of a pool of potential candidate samples and can be very informative. For instance, taking into account for selection bias (the bias that arises when the samples are not representative of the wider population), studies have shown that protected areas have actually led to a reduction, instead of an increase, in poverty in both Thailand and Costa Rica.
Our workshop brought together a set of leading sustainable development and impact evaluation experts and focused on the role of SDG indicators and the potential of analytical techniques that are available to assess joint social and environmental outcomes. When reviewing the SDGs we found significant data and indicator design challenges, which has led to one working group to conduct a detailed assessment of the SDGs, their targets and their indicators. Without appropriate indicators, SDG related policies and interventions run the risk of becoming targets in themselves and provide no information about their effectiveness.
Rigorous impact evaluation does not just require more data (the typical conclusion of many studies); it requires a change in mind-set and an understanding that simply tracking an indicator does not provide any information about the processes driving those changes. Aligning impact evaluation frameworks and the SDGs provides an opportunity for more collaboration between scientists and practitioners. The science of development is crucial to the SDG agenda and bringing impact evaluation into sustainable development might prove to be the most effective and sustainable monitoring approach.