Risk Analysis
This Investment Learning Platform (ILP) note discusses the various types of risks to be considered in investment planning and project analysis (qualitative risk assessment) and how to measure the impact of some of these risks on the project outcomes and expected returns (quantitative risk analysis).
More particularly for quantitative risk analysis, this note attempts to show how to integrate risk analysis into standard project Financial and Economic Analysis (see FEA). Simple methods are available for modeling risk, requiring minimum expertise in statistics and probabilities, together with user-friendly computer programmes.
In practical terms, quantitative risk analysis complements the scope of qualitative risk assessment, such as that found in the World Bank’s formers' Operational Risk Assessment Framework (ORAF) which is now called the Systematic Operations Risk-Rating Tool (SORT). It also complements classical FEA by providing a more thorough understanding of the project dynamics and uncertainties. The insights gained by quantitative risk analysis may be useful for project design and ex post evaluation.
Investment projects are risky by nature, and risks should be assessed during all steps of the project cycle. Overall, risk analysis should inform decision-making, whether it is a “go ahead” or “don’t go ahead” choice, as well as when it is a matter of choosing between alternative design options.
BOX 1: Key messages HOW? RESULTS |
Considering risks when designing national agricultural investment plans
At the plan level, agricultural risks to be considered (qualitatively) and reflected in plan design and prioritization, are typically related to price volatility, market and institution failures, weather shocks (such as drought and floods), insect pests and crop diseases. Before proposing risk management approaches in Investment Plans (which are not in the scope of this note on risk analysis), the following risks need to be carefully qualified and analyzed:
- Market risks: price volatility and unpredictability; seasonal commodity price variability;
- Environmental and social risks: frequency of hazards (rain failures) and shocks (droughts); recurrence of resource-based social frictions and conflicts;
- Institutional and policy risks: legal framework and policy environment for: (i) early warning and response systems (contingency planning); (ii) price risk management tools (warehouse receipt systems, commodity exchanges, contract farming, grain stock management, etc.); (iii) disaster risk management measures (weather-based insurances, safety nets, etc.)
Considering risks when designing projects
Based on the former World Bank's ORAF, the following categories of risks at project level need to be considered:
- Project stakeholder risks (donor relations, borrower relations, general public perception);
- Operating environment risks, such as: country risks (enabling environment, systemic fraud and corruption, fiduciary management, civil society capacity, politics and governance, security); and institutional risks (ownership and commitment, accountability and oversight, institutional capacity);
- Implementing agency risks, such as: governance risks (decision-making, behaviour and norms, accountability and oversight, ownership); capacity risks (resources, processes); and fraud and corruption risks (prevalence of fraud and corruption, level of transparency and control); and
- Project risks, such as: design risks (technical complexity, scope/coverage complexity, arrangement complexity, design flexibility); safeguards risks (environmental and social); programme and donor risks (programme dependencies, donor collaboration, donor delivery); and delivery quality risks (sustainability, measurability, contract management);
Clearly, quantifying all these intangible risks and assessing their impact on the return on investment is extremely difficult. One suggested practice is to assess the impact of project risks for which it is possible to assume numeric projection values about the future and/or make probability estimations. Typical factors found in agriculture projects subject to risk analysis are summarized in Table 1 below:
Table 1. Typical factors found in agriculture projects subject to risk analysis
Sector | Factors affecting estimates of economic outcomes |
Irrigation and rural development | Cost overruns, implementation delays, untested technologies, cropping intensities, poor water management |
Livestock | Animal mortality rates, parturition rates, off-take rates, price trends |
Rural and agricultural credit | Repayment rates, debt amnesties, rates of interest, government commitment |
Fisheries | Price trends, numbers of vessels, fish stock composition |
Forestry | Benefit estimations and valuations |
Industrial crops | Yields and prices, cost overruns, mill capacity and throughput |
Health and education | Morbidity and mortality levels, cost of services (participation rates and demand) |
Rural roads and transport | Implementation delays, cost overruns, traffic flows, vehicle operating cost (VOC) savings |
These risks are often very significant for agricultural and rural development projects, and simulating them is possible. Risk analysis can help to quantify the impact of alternative project scenarios.
Undertaking quantitative risk analysis at project level
Like the sensitivity analysis, risk analysis is a useful tool to quantify the impact of alternative scenarios. In a nutshell, risk analysis helps respond to the following questions: “How likely is the project to achieve the expected results (or the best possible results/returns) and, if it does not, by how much is it likely to over- or underperform? How does this change if we adapt project design?” Risk analysis (when performed, for example, through a Monte Carlo method) requires only a basic knowledge of statistics and one of the readily available and user-friendly software products (such as @RISK, Crystal Ball or Risk Solver). It is thus a concrete and time-efficient tool that can complement FEA.
Risk analysis considers the following question: “If we had the possibility to run 100, 500, 1 000 or 10 000 times the calculation of the project’s internal rate of return (IRR) with random changes on the value of the key project variables (e.g. yields, prices etc.), what would be the distribution, mean value and standard deviation of the project IRR and NPV?”
The overall methodology for preparing a project FEA with risk analysis is well described in literature such as the Handbook for Integrating Risk Analysis in the Economic Analysis of Projects. Key steps can be summarized as follows:
(i) Prepare an FEA, following a standard “deterministic” approach. This includes: a) calculation of the baseline IRR and NPV; b) identification of the most important project uncertainty variables (see Table 1 above); c) preparation of sensitivity analysis (calculation of the IRR and NPV by varying, either “optimistically” or “pessimistically,” each of the most critical variables of the project); and d) calculation of the switching values that makes the Net Present Value of a project equal to nil;
(ii) Carry out risk analysis (“stochastic risk analysis”) with a risk-modeling programme. Before using a risk analysis programme (e.g. @RISK) set up Excel spreadsheets to record testing of major sensitivity parameters (this step should be performed while preparing the FEA). While using the software, key steps are: a) specify the probability distributions (Normal/Gaussian ("bell curve") Uniform Discrete, Triangular1, etc.) that the software asks the user to define for the important sources of project uncertainty2; b) correlate uncertain inputs (e.g. correlation between input prices and crop price); and c) run the simulation through a Monte Carlo methodology. During the simulation, the programme will randomly generate numbers (over 100, 500, 1 000 or 10 000 iterations, as chosen by the user) following the probability distributions for the uncertain input factors and then recalculate the IRR and NPV.
(iii) Create a distribution graph (e.g. probability density graph, relative frequency histogram) that plots the probabilities of the IRR/NPV (and their average value). An example using @RISK for a hypothetical project is provided below. In the example (Figure 1 : Probability of IRR), the probability of the IRR is around 14.4 percent, but there is also a risk (36.9 percent) that the mean IRR may fall below 12 percent (the discount rate assumed in several FEA).
- Figure 1: Probability of IRR
How to use results of the analysis
Risk analysis strengthens the FEA, but it cannot compensate for the absence of reliable data. The results of the FEA are particularly critical at the design stage. Whereas the FEA should be started at the identification/project concept stage, quantitative risk analysis is most valuable during the design stage, when sufficiently detailed information and data should be available. In other phases of the investment cycle, such as during implementation, it should be possible to re-run an FEA and the accompanying risk analysis and to adjust the project focus and approaches accordingly. These results, which potentially take into account risks that were not identified at the initial appraisal, can therefore be used to improve project implementation.
Footnotes
1 Triangular distributions are parametrically parsimonious (they only require three judgments about “lowest possible”, “most likely” and “highest possible” values, as elaborated by Hardaker et al. 2004, pp. 52-55) and are often the preferred choice of users when data is scarce.
2 The design of distribution curves (i.e. choosing the standard deviation, mean, maximum or minimum values, etc.) can be based on historical data, the opinion of project experts or the analyst’s own subjective estimations about the future.
Key Resources
Reference Material on the Operational Risk Assessment (ORAF), (World Bank)
| Risk assessment frameworks list and rate risks that are both qualitative and quantitative. For each risk, such frameworks provide mitigation measures that should be considered in project design and, if appropriate, in financial and economic analysis.The ORAF is a mandatory tool used by the World Bank to appraise its projects. |
The @RISK software, compatible with Excel, is easy to use and requires only a minimum knowledge in statistics/probabilities. At project level, @RISK is useful to integrate risk analysis into standard FEAs. | |
Handbook for Integrating Risk Analysis in the Economic Analysis of Projects (ADB, 2002) | The Handbook provides guidance on integration of risk analysis in the design and analysis of projects. It illustrates the quantitative risk analysis software through case studies based on actual ADB projects in the agriculture, education, health, and power sectors. It also focuses on the use of risk analysis to achieve poverty reduction. |
Economic analysis of agricultural projects (World Bank / J.Price Gittinger, 1984) | Provides those responsible for agricultural investments in developing countries with sound analytical tools they can apply to estimate the income-generating potential of proposed projects. |
Simple and hands-on guidance for the undertaking of EFA illustrated by good examples for each category of projects. Provides standards on how to present assumptions and findings and for the information to be included in project design documents, their annexes and working papers. | |
Integrating risk management tools and policies into CAADP: options and challenges (FAO, NEPAD, 2013) | Analyzed different price and disaster risk management tools as well as the challenges of their application in Africa, and documented country experiences to guide decision makers in the mitigation of the negative effects of production and market and price volatility. |