Decision Analysis at World Agroforestry - Download pdf.
Introduction
Our goal is to deliver well-targeted information systems that reduce critical uncertainties in key investment decisions facing stakeholders on improving land health and livelihoods. This will help to make better decisions. Our research therefore:
Focus on decisions
Most research for development is never used in decision-making, and much of it is poorly aligned with the needs of decision-makers. This was the result of a report undertaken by ICRAF’s Land Health Decisions unit, in collaboration with Hubbard Decision Research and commissioned by FCDO.
Since all major pathways, through which research can achieve development impact, require decision-makers at some level to change their behavior, this misalignment between research and decisions is cause for concern. To respond to this situation, ICRAF has embarked on a mission to undertake research that targets specific decisions. Drawing from approaches that are commonly used in business decision support, we construct impact pathway models for specific decisions. These include all major factors of relevance for the specific decision in question, most notably all costs, all benefits and all risks that may affect the decision outcomes. Probabilistic simulation runs are then conducted to assess the likelihood of particular development outcomes, based on the shared understanding of the impact pathway between decision-makers and analysts. The procedure requires close collaboration between decision-makers and researchers to ensure that researchers are able to consider all factors that are important to decision-makers and that study results will be considered in the decision-making process. The analysis does not only provide a projection of likely decision outcomes, it also allows identification of the key uncertainties involved in the decision, which then emerge as priorities for further research. Identifying priorities in this manner ensures that scarce resources for research are expended to close critical knowledge gaps, rather than addressing issues that appear obvious to researchers trained in a particular discipline but that often have little information value.
Considering uncertainty
Considering uncertainty is critical in decision support, because knowledge gaps abound. This is particularly true in many developing country contexts, but it also applies elsewhere: We never know everything we would like to know with perfect certainty. In such situations, it is a common procedure to make ‘best guesses’ for uncertain variables. This strategy can be used to generate precise results, but these can be highly misleading, because the implications of these guesses being wrong are not captured. It is also very difficult to adequately consider risks in models based on best guesses. In our work, we embrace uncertainties and ensure that they are fully represented in the models we build. This is achieved by representing variables not as single numbers, but as probability distributions or confidence intervals. In other words, we determine the likelihoods that a variable takes on a particular value. This information is used in procedures known as Monte Carlo analysis or Bayesian Network analysis to produce similarly probabilistic outcome projections. This means that we obtain probability estimates of the likelihood that certain outcomes are achieved. We can then make statements about the likely impacts of a decision with a reasonable degree of certainty, avoiding excessive precision, which is practically never justified, as well as the temptation to ignore relevant risks in projecting decision outcomes.
The Intervention Decision Model
This process simultaneously addresses three main challenges. Even uncertain and difficult to measure interventions can be assessed and prioritized. And because of the information value calculations, the best agro-ecosystem health metrics can be identified and the value of research itself can be articulated.
In order to get this process started, classes of typical intervention decisions have been modeled in this structured way. To this end, the CGIAR in collaboration with Hubbard Decision Research developed the Intervention Decision Model (IDM). The IDM is a guide to all of the major intervention decisions in sustainable agriculture and ecosystem management. It consists of four major components: a model for all major classes of interventions, the model for forecasting the outcomes of those decisions, a set of “preference curves” that represent risk aversion and other policy choices, and the quantified values engine that combines them all for recommendations.
- Proposed Programmatic Interventions: The IDM covers all major types of intervention decisions including improvements to smallholder irrigation, better floodwater and aquifer management, design of new benefit sharing schemes for water and land in large river basins, improvements to rainfed-productivity including crop and rangeland-productivity, investment plans for waste management, and climate change adaptation or mitigation strategies. These are implemented as direct infrastructure investments, policies, training or incentive programs. Investments in these areas have impacts that improve incomes in both the near and long term as well as improve access and security for water, energy, and food. The IDM includes all of these types of decisions.
- Objective Forecasts: The costs of the intervention may be uncertain and the long term effects of any intervention will be uncertain. The IDM determines the uncertainty of onsite and offsite impacts as well as behavioral factors like the adoption rate of a new practice or how incentives change behavior. Some of the elements of this model are based on known science such as yield improvements from additional irrigation. But some factors, such as long term changes in behavior are much more uncertain. Quantifying the difference in this uncertainty is critical in determining what to measure and what to monitor to track intervention performance.
- Quantified Preferences & Policies: Preferences about what risks are acceptable, how to value long-term effects, or the value of equitable improvements in income, need to be quantified and documented as a matter of policy. These preferences are captured as a set of “utility curves” that make policies – such as the relative value of a near-term certain impact vs. a long term and uncertain impact – unambiguous. Such clarity will mean that various interventions can be evaluated against the same standards of risk aversion and other preferences.
- Quantified Values: Ultimately, the effects of an intervention and the quantified preferences are combined into a single monetized value so that interventions of different types and sizes can be compared. Each intervention creates a set of estimated impacts over a period of time. The timing and uncertainty of these impacts are adjusted so that they can be rolled into a single number. The quantified values can also adjust outcomes for differences in how benefits of a program are distributed equitably. Separately, the IDM assesses the likely impacts on individual System Level Outcomes: poverty, food security, nutrition and health, poverty, and sustainability.
Outline of the Intervention Decision Model
Global Partnership for Sustainable Development Data
The Global Partnership for Sustainable Development Data is an open, independent, multi-stakeholder network harnessing the data revolution for sustainable development. The Global Partnership for Sustainable Development Data convenes, connects and catalyzes action to achieve sustainable development and pave a road to dignity for all using data. It works to galvanize commitments, build capacities and foster collaborations to address data gaps and harness the data revolution to meet the Sustainable Development Goals (SDGs) by 2030. The Global Partnership is a network of more than 150 data champions representing the full range of data producers and users working around the world to harness the data revolution for sustainable development. Members include governments, companies, civil society groups, international organizations, academic institutions, foundations, statistics agencies and data communities.
World Agroforestry (ICRAF) is one of the partnership's data champions committed to supporting the generation, delivery and better use of open data for better, evidence-based decision making on sustainable land use for improved livelihoods. This support has potential to improve livelihoods of millions of resource poor land users, create new jobs around agricultural value chains, and better the environment in support of the SDGs.
Dr. Keith Shepherd (Leader - Land Health Decisions, ICRAF ) is also a member of the Thematic Network on Data for Sustainable Development, one of the 12 Thematic Groups of the Sustainable Development Solutions Network (SDSN). This thematic group serves as an information and education hub on data collection, processing, and dissemination for sustainable development. It seeks to identify solution-orientated approaches to measuring progress on the SDGs, to strengthen the cross-sectoral and multi-scalar analysis of data for SDG monitoring, and encourage greater frequency and quality of data production and monitoring. Read more: -
CGIAR.2012.The Need for an Intervention Decision Model. A concept Paper by the CGIAR
Shepherd K, Hubbard D, Fenton N, Claxton K, Luedeling E, De Leeuw J, 2015. Policy: Development goals should enable decision-making. Nature 523, 152-154.