Tools for the Guidebook for evaluating fisheries co-management effectiveness
Task 3.4: Measure the indicators and analyze the results
Analysis is the process of carefully considering, comparing and contrasting information with the intention of helping to clarify uncertainty, or find answers to specific questions. In the case of this evaluation, the analysis of data collected during the co-management system evaluation will help you to address and respond to questions around the weaknesses of the co-management system. Analysis should be a participatory process and involve the evaluation team and stakeholders so that different perspectives on the results may be obtained. There are two stages of the analysis:
- measuring the indicators selected/developed in Task 1.9, i.e. putting a value on each selected indicator; and
- analyzing the results and interpreting the values of the measured indicators.
To measure the indicators;, the relevant information needs to be gathered for each indicator. This may include data from the database, written notes from members of the evaluation team, or other sources, depending on the data management system in place. Measuring indicators can be done in different ways and depends on the complexity of the evaluation and the data collection. The most appropriate way of analysing the data should be determined – whether the data is quantitative or qualitative.
The Guidebook proposes assessment sheets as a relatively simple way of measuring the selected indicators. With this method, each indicator is formulated so that it can be scored on a scale of three.
- When measuring the indicators for the fisheries co-management system good practices, the evaluation assessment sheet can be used. A three-level scoring of the level of completion of each indictor is undertaken: the best practice exists, partly exists or does not exist.
- When measuring the fisheries co-management plan indicators an evaluation assessment sheet is used to analyse the achievement of the co-management plan goals and objectives and associated indicator(s). A three level (fully achieved, partly achieved and not achieved) scoring of the level of completion of each goal and objective (and associated indicator[s]) is undertaken.
Comment boxes are provided on the assessment sheets so that explanations of the level of completion, or achievement score, may be included. Boxes for noting the method/source of information are also provided so that the method/source of the information used to measure the indicator may be described.
Once the indicators have been measured, these results should be analysed. For the results to be useful and to allow for corrective action to be taken as needed, it is important to understand the reasons behind a score.
The results of the co-management system evaluation will be used for the analysis of the co-management plan because there is a causal relationship between the level of completion of the good practices and the resulting achievement of the goals and objectives of the co-management plan. For example, the lack of achievement of an objective may be found to be related to the lack of completion of a best practice.
It is also important to look at relationships between different indicators. These relationships are crucial because looking at a single indicator without understanding its interaction with others can lead to a misleading evaluation. For example, social factors – such as whether traditional knowledge is integrated into fisheries co-management plans – may influence whether management measures are appropriate and operational. Likewise, the existence of fairly allocated tenure rights may influence whether resource users perceive that there are incentives to participate in the co-management arrangement.
The analysis will involve explaining the results of the scores for the indicators and in turn why good practices either exist, exist partly or do not exist, and why the co-management plan’s objectives have been met, or not. This type of analysis should focus on the main purpose of the evaluation and a review of the questions being asked by the evaluation. What are the essential questions that the evaluation wants to address or fully answer?
Depending on the indicator, there may be different views on what the reasons behind a score are. The analysis should be a participatory process that involves the evaluation team and stakeholders so that different perspectives on the results may be obtained. It is helpful to draw these relationships on paper in a diagram.
An example of the analysis of an evaluation of a fisheries co-management system and plan utilizing the analytical framework is:
- An objective of the fisheries co-management system may be: effective stakeholder participation and representation,
- An indicator to evaluate achievement of this objective may be: all main stakeholders are empowered and capable to actively participate in decision-making, including women and youth.
- The results of the evaluation found that the outcome of the objective was not achieved and that there were only a limited number of stakeholders participating in relevant meetings and making their voices heard.
- The framework provides a structure for the analysis of why the objective was not achieved by looking at the relationships between this result and the context of the system (Task 2.1), the process (Task 2.2) and other related indicators.
- In this case, it was found that a possible explanation for there being only a few participants at meetings could be: (1) the resource users were heterogeneous from many different ethnic groups (context); (2) meetings were not organized on a regular basis (process); (3) that the process of implementing the co-management system did not involve enough capacity-building on working together and building trust (best practice indicator).
- The "explanation" for this poor outcome was that there was a limited incentive for the resource users/co-management participants to meet as they did not trust each other.
- An adaptive management strategy was agreed to have more training on building trust and to schedule monthly meetings between managers and resource users so that there could be more direct interaction and discussion.
Suggestions
Socio-cultural factors, gender, etc. need to be considered in analysis and interpretation. Local stakeholders can assist.