عربي
English
Español
Français
中文
Русский
 
 

EAF Steps

Activity 2.2 - Issue Prioritisation and risk assessment

Purpose

To qualitatively assess the risks of each of the identified EAF components (i.e. target stocks, bycatch, habitat, ecosystems, social, economic and governance) to determine their relative priority for direct management or other actions. This approach is a formalised system which enables the assessment of risks where insufficient information is available for fully quantitative methods.

Overview

Most robust prioritization processes are based on risk assessment principles because these involve assessing the uncertainty in achieving your objectives. This is because risk is defined as “the impact of uncertainty on achieving objectives” (ISO, 2009). All risk assessments essentially involve the calculation of the magnitude of potential consequences and the likelihood that each of these consequences will occur; these scores are combined to generate a risk level (Scandol et al., 2009). Essentially, the higher the probability of a ‘worse’ consequence may actually occur, the greater is the level of risk.

Qualitative risk assessments use words rather than numbers to describe the magnitude of potential consequences and the probability (likelihood) that those consequences will occur (HB 436, 2004). The C×L matrix method therefore combines the scores from the qualitative or semi-quantitative ratings of consequence (levels of impact) and the likelihood (levels of probability) that a specific consequence will occur (not just any consequence) to generate a risk score and risk rating (IEC/ISO 301010, 2009). Other qualitative risk methods are available and these are described separately.

4 x 4 Risk Matrix – numbers in cells indicate risk value, the colours/shades indicate risk rankings (see below for descriptions).

  Consequence Level
  MinorModerateMajorExtreme
Likelihood Levels1234
Remote11234
Unlikely22468
Possible336912
Likely4481216

This C x L risk assessment process involves selecting the most appropriate combination of consequence and likelihood levels that fit the situation for a particular objective based upon the information available and the collective wisdom of the group (including stakeholders) involved in the assessment process. These scores are multiplied to generate an overall risk score.

Using the Risk Matrix above, if the assessment group concludes that the most appropriate combination for an assessment of a particular objective is that - it is possible that a major consequence could occur, this is a Consequence level 3 and a Likelihood level of 3. These two scores are multiplied to generate a Risk Score of 9 which for this system would equate to a High Risk (see Risk Levels and Outcomes below) which is an unacceptable level of risk, and therefore increased management actions would be needed to achieve the objective.

To correctly assign the levels of consequence and likelihood, it is important to recognise that these form a pair, they are not to be chosen independently. It is the likelihood that, given a particular fishing management strategy, a particular level of impact may be the result (either from an accumulation of small events or from a single large event). It is assessing the likelihood of an outcome being generated not the likelihood of an activity occurring. This type of error must be avoided as it results in over-rating risks.

When making decisions about what are appropriate combinations of consequence and likelihood, if more than one combination of consequence and likelihood is considered plausible, the combination with the highest risk score should be chosen (i.e. this is consistent with taking a precautionary approach).

Whichever final combination of consequence and likelihood is chosen, it is very important that the justifications for choosing this combination of levels is recorded. Other parties who were not part of the assessment process may need to be able to see the logic and assumptions behind the decisions. It also greatly assists the review of the risk in the future if you know why the levels were originally chosen.

Each of the consequence tables needs to have descriptions of the levels of consequence – if using a four-category system this could be minor, moderate, major, extreme that are relevant for that objective whereby the moderate level should always describe the maximum acceptable level of impact.

Determining the acceptable level of impact is a very important part of the risk assessment (and management) process because it defines how the process operates. The descriptions for each of these levels provided in the Appendix tables have been developed based on experiences gained across multiple fisheries; however, they can be changed to suit local circumstances. Given that a level of impact in one circumstance may be acceptable but not in others the description of what level of impact is ascribed to what level of consequence can also vary. So, for example, the same level of impact could be considered a moderate consequence for one objective but a high consequence for another.

There is also a table that outlines four levels of likelihood from remote to likely in both words and probability ranges.

The starting set of consequence tables based on the 4 x 4 matrix system for use in fisheries management is provided in Appendix 1 but they can be adjusted for local purposes. These cover the most common types of fishery-related objectives including:

  • Target Stock Objectives - stock sustainability (spawning biomass), MSY, MEY
  • Ecosystem Sustainability (trophic impacts); habitat sustainability (habitat damage), protected species (incorporating social acceptability aspects)
  • Economic Objectives - Economic returns, MEY
  • Social Objectives – Food Security, Social Attitudes, MSY
  • Governance - Political

Risk Levels and Outcomes for a 4 x 4 matrix

Risk levelsRisk ScoresLikely Management ResponseLikely Reporting Requirements
Negligible 0 - 2NoneBrief Justification
Low 3 - 4No Specific ManagementFull Justification needed
Medium 6 – 8Specific Management/ Monitoring NeededFull Performance Report
High 9 - 16Increased management activities neededFull Performance Report

EAF Tool Tips

When is it good? This method works well in EAF planning when there is some level of quantitative information and/or a good level of qualitative information available to base judgements. It also required having the potential consequences reasonably well understood so that they can be easily structured into different levels which have clear written descriptions.

The advantage of this C×L risk assessment system is that it can be used to assess the risk to any fishery related objective including ecological, social, economic, political and occupational. In addition to the set of consequence tables provided in the Appendix, consequence tables can be user-defined and can therefore be modified or developed to suit the particular objectives and acceptability associated with any objective in that location. This is most important for the objectives related to non-target species and especially iconic species, for which the acceptable levels of impact can vary greatly among countries and regions.

The four-category system outlined above and as now used in Pacific and Africa is much simpler to use than the original six-category system. The number of categories can, however, be easily varied to the most appropriate level of detail for the situation, but more categories can increase disputes in choosing the levels unless the descriptions are very precise and clear, and the additional resolution is warranted. Using more categories may increase the difficulty of assessments without materially affecting the outcome. The descriptions of the different levels in each of the tables can be modified to address local circumstances and new tables can be developed for new objectives.

When is it not so good? It does not work well with stakeholders who have had minimal training or their language skills and formal education will make this a difficult concept to grasp. It also does not work well with specific issues/objectives that are hard to categorise into levels (which includes many perception based social issues). In these situations the simpler non formal or category based methods will be better options to use (see Synergy below). It also works badly when the facilitator doesn’t understand the concepts and allows the process to generate spurious combinations. It can also fail where individuals participating have highly preconceived ideas on what they think are the risk levels (this can be both low and high) in such cases outcomes could be improved through the use of a panel rather than open forums.

General Tips for Use

  • An experienced facilitator is required to make this system work efficiently. The facilitator needs to understand the basis of risk assessment, how this method operates and must be aware of how the descriptions in each of the tables are defined to assist the group make good decisions about the most appropriate C × L combinations.
  • It is essential to have a training session with all participants before they begin the formal assessment process if they are to participate effectively.
  • Ensure participants actually read out loud the FULL descriptions of both consequence and likelihood together when they propose a suitable combination - not just the category levels - as it is common for people to unconsciously reinterpret the levels based on their biases on what outcome they want.
  • The discussion process to assign risk levels needs to be undertaken using a language that the participants are very familiar with. The process can be confusing enough without adding language inconsistencies. Therefore all the supporting material needs to be in the language that will be used.
  • Getting up-front agreement that the levels and descriptions for each table are sufficiently unambiguous - especially what is the maximum level of impact that is considered acceptable.
  • It is vital to ensure that when choosing the combination of consequence and likelihood that the selected likelihood score relates to the likelihood of a particular consequence level actually occurring, NOT just the likelihood of the activity/event/management occurring. This is an extremely common error to make.
  • When defining levels of consequence, it is important not to use language that is associated with uncertainty. For example don’t say, “a stock is probably above BMSY)”, as this will cause confusion between the specification of consequence and likelihood. Consequence statements should be worded as propositions that can have a formal likelihood associated with them.
  • It is NOT necessary to have full certainty for issues to rate its risk, nor does uncertainty automatically generate a high risk. The level of uncertainty is only a component of the risk calculation process. Risk assessment is therefore making the MOST informed decision you can that includes uncertainty.
  • Not assessing the risk for an issue because there is a lack of information essentially means that the current actions are rated as being acceptable.
  • The combination of consequence and likelihood chosen should be based on the risk of something happening within a defined time period - not the risk of it happening at any point in the future. A convenient time frame to use is the timeframe of the management plan - which is usually around five years.
  • A large discrepancy in scores between individuals often reflects that they are really assessing different issues, have different ideas of acceptability or have different knowledge bases. Be sure they are really using the right table. Don’t assess the risks to a ‘protected’ species, which has high social value and therefore a low acceptable level impact using the target species table. Alternatively, ensure that participants are using the descriptions for the levels not creating their own interpretation of what the levels should be.
  • Most ecological assets – especially target stock levels – have more than one fishery objective (e.g. – ecological, economic, social). It is important to separately rate the risks associated with each relevant objective, the ecological risk may not always be the highest and it is important to clarify this before specific operational objectives and performance levels for management are set.
  • If a large group is participating in the risk assessment workshop, it can be more efficient to have the final risk score combinations chosen by a smaller ‘expert’ panel (this panel can include non-technical people). The broader audience can provide their input during an open discussion phase and comment on the written outcomes. Detailed reporting of discussion can help with disputes over selected risk scores that may occur subsequent to the risk assessment workshop.

EAF Tool Pedigree

A number of qualitative risk assessment methods are available (IEC/ISO 31010, 2009), one qualitative method originally adapted for use in Australian fisheries ESD framework (Fletcher et al., 2002) is based on the consequence-likelihood (probability) matrix method as outlined in the Australian/New Zealand Standard Risk Assessment methodology (AS/NZ 4360, 2000; HB 436, 2004). It has been thoroughly tested in many Australian fisheries (Fletcher 2005) and has subsequently been refined for broader use in assessing risks to ecological, social, economic and political objectives including use in other countries (e.g. Fletcher, 2010, FAO, 2011).

EAF Tool Synergy

This tool works very well with the component tree or component list approach where the issues have been sorted into very clear categories of assets, outcomes and systems that easily link to the high-level fishery objectives.

Qualitative risk assessments may also benefit from being used in combination with the objective- impact matrix, which summarises the types and levels of impact on each of the main objectives/outcomes.

This method can also be used in combination with both less formal methods for issues where the data are considered too minimal or the issue lends itself to more perception-based assessment.

Similarly, for issues where formal quantitative risk or stock assessments already exist, these methods should generally be used, but applying this method and seeing if they are consistent may provide either comfort in both assessments or raise some interesting questions as to which is more accurate rather than which is more precise!

Finally the outcome from the qualitative assessment may generate the catalyst for the completion of more quantitative approaches if the risk level and priority is sufficiently high enough. Using a tiered approach will be the most cost effective method for management.

EAF Tool Usage

Medium

Cost

Low, Moderate

If there is someone within the team or within the relevant agencies who can facilitate this risk assessment process then it is a low cost option. The main costs would be generated if it is necessary to employ a suitably skilled facilitator/risk assessment leader to run the risk assessment process but even this should only be for a few days. There will be some costs in collating the required information for the risk assessment, but this is covered by the scoping document. The other major cost will be in conducting the meeting for example getting the participants to the venue and other associated costs. If all participants are based locally local then these costs can be very small.

EAF Tool Capacity

Moderate

The method has a moderate level of complexity and will usually require the workshop group to have some period of instruction before they can participate effectively. The main capacity required for this method is to have a facilitator that has sufficient knowledge and experience in risk assessment and specifically the application of this C×L method so that the group can be guided appropriately. The facilitator should also have some understanding of fisheries management to ensure that they can tell when the group may be going ‘off track’ or is mis-interpreting the process. It is likely that there will be many people within an area who are skilled at this type of risk assessment methodology but who may not have applied it to fisheries. One option could be to use such a professional in combination with a fisheries expert to co-facilitate. Given the benefits of such methodology and its universal application it would be preferable to train one or more people in this approach.

Background Requirements

Low – Moderate

Qualitative risk assessments can be used with whatever quantity and quality of data are available and the collation of the data and other relevant information should have been covered in Step 2. This system can be applied even where there are minimal quantitative data but is likely to be more satisfactory where there is some information (even high levels of qualitative information) available to provide a defensible specification of consequence and likelihood. Where substantial datasets are available this approach may be a precursor for the use of more quantitative techniques if the assessment indicates that the risk is sufficiently high for these more costly approaches to be applied.

Participation

Moderate - High

Whilst the method can be applied by just one person, it can be completed by a small group, a panel, or it can also be completed in an open workshop format with many people involved. For assessments that are largely on technically based issues, the main input will usually be based on expert opinion. Those that assess community objectives can include a wider set of opinions.

It is often possible for the different consultation approaches to be used in combination, e.g. using a smaller group to generate risk values that can then considered by a broader stakeholder group. Alternatively, there can be general stakeholder input to discussions but the final decisions on scores are made by a smaller, expert group

Because the method requires documentation of the reasoning used to determine the initial risk values, these documents can be circulated to obtain input and comment from those stakeholders who were not part of the workshop process.

Time Range

Short

The process can be completed very quickly. All the risks associated with a particular fishery can be calculated within a day or two during a single comprehensive workshop (which is often held in combination with an issues identification process). There should also be time allowed to document the outcomes of the meeting and include summaries of the supporting information that was used to determine the risk scores.

If a staged series of meetings is required, each additional meeting should only take a day or so to complete. The time required is essentially to gather the relevant information (this should have done in Step 2) or people with the information and time to organise the workshop or forum to complete the assessment process and the time to complete the report.

Source of Information

Fletcher et al (2002) ESD Reporting Framework for Australian Fisheries: The 'How To' Guide for Wild Capture Fisheries.  Internet resource
Fletcher WJ. (2005) Application of Qualitative Risk Assessment Methodology to Prioritise Issues for Fisheries Management. ICES Journal of Marine Research; 62:1576-1587  Internet resource
Fletcher (2010) Planning processes for the management of the tuna fisheries of the Western and Central Pacific Region using an Ecosystem Approach.  Internet resource
AS/NZS ISO 31000 (2009) Risk management – Principles and guidelines. Sydney, Australia: Standards Australia.  Internet resource
Australian Website on the Ecosystem Approach  Internet resource
International Standards and Risk Assessment  Internet resource Internet resource Internet resource

Other Relevant References

AS/NZS (2000) Environmental Risk Management: Principles and Process. Based on AS/NZS 4360:1999, Risk Management HB 203-2000. 81pp.
AS/NZS (2004). HB 436 Risk Management Guidelines: Companion to AS/NZS 4360: 2004. Standards Australia. Homebush, NSW. 116pp
Cochrane, K.L., Augustyn, C.J. and M.J. O’Toole (2008) The implementation of the ecosystem approach to fisheries management in the Benguela Region: Experiences, advances and problems. In: The Ecosystem Approach. Bianchi G, Skoldal HR (eds.). Rome: FAO CABI. 262-292
FAO (2011) A guide to implementing an Ecosystem Approach (EAF) for fisheries in Africa. EAF Nansen Project (in press)
Fletcher, W.J., Chesson, J., Sainsbury, K.J., Fisher, M. & T. Hundloe (2005) A flexible and practical framework for reporting on ecologically sustainable development for wild capture fisheries. Fisheries Research 71:175-183
IEC/ISO 31010 (2009) International Standard: Risk Management – Risk Assessment Techniques. International Electrotechnical Commission Geneva Switzerland. 90pp. 
Scandol, J., Ives, M., M. Lockett (2009) Development of national guidelines to improve the application of risk based methods in the scope, implementation and interpretation of stock assessments for data poor species. FRDC Project 2007/016. Industry & Investment NSW –Fisheries Final Report Series 115, 1884pp.

Appendix

Appendix: Likelihood and Consequence tables for qualitative risk assessment. (All tables and supporting text and figures modified from Fletcher et al., 2002, Fletcher, 2010; FAO, 2011)

Table A 1

Likelihood Definitions – these are usually defined for the likelihood of a particular consequence level actually occurring within whatever is the assessment period (5 years is common).
Level Descriptor
Likely (4)

A particular consequence level is expected to occur

(Probability of 40 - 100%)

Possible (3)

Evidence to suggest this consequence level is possible and may occur in some circumstances

(Probability of 10 - 35%)

Unlikely (2)

The consequence is not expected to occur but it has been known to occur elsewhere

(Probability of 2 -10%)

Remote (1)

The consequence has never been heard of in these circumstances, but it is not impossible

(Probability < 2%)

To correctly assign these levels, it is important to recognise this is a conditional probability of a consequence occurring. It is the likelihood that, given a particular fishing management strategy (e.g., the current allowable catch levels for a tuna fishery), a particular level of impact (e.g., a reduction in spawning biomass to x% of unfished levels) may ultimately be the result (either from an accumulation of small events over time, or from a single large event). It is NOT, as is commonly done when beginning this process, mistakenly assessing the likelihood that the particular fishing activity (i.e., catching the species) will occur. This type of error must be avoided.

Table A2a

Consequence categories for the Major Target/Vulnerable species. The default objective is - maintain the biomass above the level that will generate MSY”
Level Ecological (Target/Vulnerable Species)
Minor (1)

Either not detectable against background variability for this population; or if detectable, minimal impact on population size and none on dynamics.

Exploited Stock Abundance Range 100% - 70% unfished levels

Moderate (2)

Fishery operating at, or close to, the exploitation rate that will deliver MSY..

Exploited Stock Abundance Range < 70% to > Bmsy

Major (3)

Stock has been reduced to levels below MSY and may also be getting into the range where recruitment overfishing may occur.

Exploited Stock Abundance Range < Bmsy to > Bmsy* 0.5

Extreme (4)

Stock size or significant species range contraction > 50% have occurred and recruitment levels reduced affecting future recruitment and their capacity to increase from a depleted state (i.e. recruitment overfishing)

Exploited Stock Abundance Range < Bmsy* 0.5

Table A2b

Consequence categories for the Major Target/Vulnerable species. The default objective is - maintain spawning biomass at least above the level where it is likely not to result in recruitment overfishing
Level Ecological (Target/Vulnerable Species)
Minor (1)

Either not detectable against background variability for this population; or if detectable, minimal impact on population size and none on dynamics.

Spawning biomass 100% - 70% unfished levels

Moderate (2)

Fishery operating at, or close to, full exploitation rate but the long-term recruitment/dynamics are not being adversely impacted.

Spawning Biomass < 70% but > Brec

Major (3)

Stock has been reduced to levels that are now directly affecting future recruitment levels or severely affecting their capacity to increase from a depleted state (i.e. recruitment overfishing).

Spawning Biomass < Brec but > Brec * 0.5

Extreme (4)

Stock size and recruitment levels reduced to an extent that local extinctions or significant species range contraction > 50% have occurred. If it continues it would require listing in an appropriate endangered IUCN category and extinctions could result.

Spawning Biomass < Brec * 0.5

Table A3

Consequence categories for the By-Product Species/Minor bycatch species. The default objective is - To maintain appropriate levels of biomass of bycatch species to minimize any significant impact on their dynamics and the broader ecosystem
Level Ecological (By-product/General Bycatch)
Minor (1)

Take in this fishery is small (< 10%), compared to total take by all fisheries and these species are covered explicitly elsewhere.

Take and area of capture by this fishery is small, compared to known area of distribution (< 20%).

Moderate (2)

Relative area of, or susceptibility to capture is suspected to be less than 50% and species do not have vulnerable life history traits.

Major (3)

No information is available on the relative area or susceptibility to capture or on the vulnerability of life history traits of this type of species AND

The relative levels of capture/susceptibility suspected/known to be greater than 50% and species should be examined explicitly

Extreme (4) N/A Once a consequence reaches this point it should be examined using Table A1.

Table A4

Consequence categories for the Bycatch of Protected species. The default objective is - To maintain levels of catch of these species at acceptable levels
Level Ecological (Protected Species Bycatch)
Minor (1) Essentially no protected species are impacted
Moderate (2) The fishery catches or impacts these species at the maximum level that is accepted
Major (3) The catch or impact by the fishery on the protected species is above that accepted but there are few additional stock implications
Extreme (4) The catch or impact is well above the acceptable level and this is having significant additional impacts on the already threatened status.

Table A5

Consequence levels for the impact of a fishery on the general ecosystem /trophic levels. The default objective is - To maintain any impact on the wider ecosystem by fishing to be within acceptable levels’
Level Ecological (ECOSYSTEM)
Minor (1) Some relatively minor shifts in relative abundance may be occurring but it is unlikely that there would be any measurable changes at whole of trophic levels outside of natural variation.
Moderate (2) Measurable changes to the ecosystem components without there being a major change in function. (i.e. no loss of components or real biodiversity), these changes are acceptable. None of the main captured species play a ‘true’ keystone role
Major (3)

Ecosystem function altered measurably and some function or components are locally missing/declining/increasing &/or allowed new species to appear. The level of change is not acceptable to enable one or more high level objective to be achieved.

Recovery measured in many years to decadal.

Extreme (4)

An extreme change to ecosystem structure and function. Very different dynamics now occur with different species/groups now the major targets of capture and/or dominating the ecosystem. Could lead to a total collapse of ecosystem processes.

Long-term recovery period may be greater than decades

Figure 1The four graphs provide a schematic representation of the types of changes that may be associated with each of the four level of consequence as outlined in Table A 4.
Figure 1

Table A6

Suggested consequence levels for the impacts on habitats. (Three levels – standard, fragile, critical). The default objective is – To maintain the spatial extent of habitat impacts from the fishing activity to a comparatively small percentage of the habitat/community’
Level Ecological (HABITAT)
Minor (1)

Insignificant or barely measurable impacts on habitat(s) which are very localised compared to total habitat area.

(Suggestion – these impacts could be < 5%; < 3%; <2%) of the original area of habitat)

Moderate (2)

There are likely to be more widespread impacts on the habitat but the levels are still considerable acceptable given the % of area affected, the types of impact occurring and the recovery capacity of the habitat

(Suggestion – for impact on non-fragile habitats this may be up to 50% [similar to population dynamics theory] - but for more fragile habitats, to stay in this category the percentage area affected may need to be smaller, e.g. 20% and for critical habitats less than 5%)

Major (3)

The level of impact on habitats may be larger than is sensible to ensure that the habitat will not be able to recover adequately, or it will cause strong downstream effects from loss of function.

(Suggestion - Where the activity makes a significant impact in the area affected and the area > 25 - 50% [based on recovery rates] of habitat is being removed; whilst for critical habitats this would be < 10%)

Extreme (4)

Too much of the habitat is being affected, which may endanger its long-term survival and result in severe changes to ecosystem function and the entire habitat is in danger of being affected in a major way/removed.

(Suggestion this may equate to 70 - 90% of the habitat being affected or removed by the activity; for more fragile habitats this would be > 30% and for critical habitats 10-20%)

Assessing the habitat impacts that may result from each fishery should be done at a regional level, with impacts judged against the best estimate of the original extent of each of the habitats, not their current distribution. The assessment criteria have divided habitat into three categories, which recognise that not all habitats are equal – some are more fragile than others - often due to slower recovery rates. Also some are more critical to the functioning of the ecosystem than others – providing substantially greater levels of fish recruitment or nursery habitat. This is why different levels of impact generate different levels of risk.

Table A7

Suggested consequence levels for economic outcomes. The default objective is - Maintenance or enhancement of economic activity
Level Economic
Minor (1) Possible detectable, but no real impact on the economic pathways for the industry or the community.
Moderate (2) Some level of reduction for a major fishery or a large reduction in a small fishery that the community is not dependent upon.
Major (3) Fishery/industry has declined significantly in economic generation and this will have clear flow on effects to other parts of the community. May result sin some level of political intervention.
Extreme (4) Total collapse of any economic activity coming from what was an industry that the community derived a significant level of their income or employment (resource dependency), including possible debts. High levels of political intervention likely.

Table A8

Suggested consequence levels for social disruptions. The default objective is Maintenance or enhancement of appropriate social structures and outcomes.
Level Social Implications
Minor (1) None, or not measurable. Includes situations where there is no direct involvement by a community in the fishery.
Moderate (2) Some direct impacts on social structures but not to the point where local communities are threatened or social dislocations will occur
Major (3) Severe impacts on social structures, at least at a local level.
Extreme (4) Changes will cause a complete alteration to some social structures that are present within a region of a country

Table A9

Suggested consequence levels for food security. The default objective is Maintaining access to sufficient resources to enable the functioning of local or regional communities
Level Food Security
Minor (1) None, or not measurable. Includes situations where there is no direct impact on the resources used by a community.
Moderate (2) Some direct impacts on food resources of a community but not to the point where these are threatened.
Major (3) Significant and long term (> weeks) impacts on food resources of a community. Likely to lead to health problems
Extreme (4) Changes will cause a complete loss or severe ongoing reductions in of some food resources present within a region of a country leading to starvation and or abandonment of region or requiring aid.

 
Powered by FIGIS