Previous Page Table of Contents Next Page

4.1 Assessment Methodology

Ekbom and Bojö (1997), inspired by earlier work by Bojö and Chee (1995), World Bank (1996) and Loksha (1996), elaborated an elementary filter of thirteen criteria grouped into five sequential sections (Table 4.2) as an aide de memoir when mainstreaming the environment into World Bank CAS. Applying this filter to a sample of thirty-four CAS's (twenty-one from Sub-Saharan Africa, thirteen from elsewhere) they concluded that not only had environmental issues made some inroads into CAS documentation, the best CAS integrating the theme across all sections - rather than treating the environment as a separate and distinct component - but that the filter was, moreover, a useful analytic tool for reviewing the extent to which the environment had been successfully incorporated. Furthermore, the exercise threw up a "rich flora of inspiring examples" of effective environmental mainstreaming which were potentially transferable.

Table 4.2 CAS Environmental assessment framework




1. Issues

Are environmental problems/issues described in the CAS?

1. Environmental Problems and Opportunities Assessment
2. Environment and Health
3. Sub-Regional and International Issues

2. Driving Forces

Do (and if so, how well do) CAS's explain the underlying drivers behind the identified environmental problems?

4. Poverty, Inequity and the Environment.
5. Property Rights.
6. Population and Environment
7. Economic Policies and Environment

3. Existing Work

Does the CAS make reference to, embody, National Environmental Action Plans (NEAP) and/or Country Environment Strategy Papers (CESP)?

8. Use of NEAP
9. Use of CESP

4. Actions

What environmental actions are proposed by the Bank - and do they build on existing donor support?

10. Donor Support for Environmental Management
11. Environment and Board Agenda
12. Proposed Actions

5. Process.

Was CAS developed in a participatory way which permitted stakeholder and expert input?

13. CAS Process

Source: Ekbom and Bojö (1997).

A modified version of Ekbom and Bojö's assessment framework was applied by Shyamsundar and Hamilton (2000) when examining thirty-seven CAS produced by the World Bank in fiscal year 1999. Besides amending the review criteria used[30], they derived a four-point ordinal measurement scale allowing the (subjective) evaluation of each criterion which then permitted aggregation to produce a "country score". This not only highlighted national and regional differences in the treatment of environmental issues[31], but also established a framework which allowed the environmental tracking of CAS over time. Significantly, while noting that there was still room for improvement, the authors also suggested that much of this improvement could come through the sharing of best practices (2000:3) - a number of which were highlighted in the accompanying text.

The framework was subsequently directed to assess the focus of PRSPs vis-à-vis environment-related issues by Bojö and Reddy (2001). Forty Interim and Full PRSPs from across the globe were appraised and, while the four-point measurement scale remained largely intact - albeit now ranging from 0 (no mention) to 3 (good practice), seventeen criteria were now encompassed within four principal analytic categories;

(i) Issues (in focus): A description of the major environmental concerns and opportunities.
(ii) Causal Links (assessment): Poverty-Environment Link Analysis.
(iii) Responses (to Environmental Challenges): Environmental Management measures, monitoring and evaluation; and
(iv) Process: Participation and inclusion of environmental stakeholders (2001:7)

The ensuing analysis found that while average scores were low (0.9), there was considerable variation (from 0.3 in the case of São Tome and Principe, to 2.2 in the case of Mozambique), and generally a marked improvement in scores as countries progressed from interim to full PRSP status. Once more, the paper presented a number of concrete examples of good practice under each of the featured categories, intimating that tracking updates were likely as more PRSPs became available[32].

The assessment methodology was subsequently appropriated and adapted by Oksanen and Mersmann (2002) in evaluating the current status and emerging trends regarding the role of the forestry sector in Sub-Saharan African PRSPs. Twenty-five Interim PRSPs, eleven full PRSPs, 17 JSAs and four PRSP Progress Reports were analysed using a four criteria (the criteria equating to the categories employed in the Bojö and Reddy article noted above), four-point scale. The results were then compared to two broader factors - the degree of forest cover remaining in the country AND the existence of an ongoing national forestry policy and sectoral planning process - the latter appearing influential in ensuring forestry representation in PRSPs, as did modest (defined as between 7-40 percent) forest cover. Overall, while twenty-one of the twenty-four Interim PRSPs, and all the full PRSPs, mentioned the sector;

"... in general the sector was incorporated in a rather modest and unsystematic manner. The analysis of the cause and effect linkages between the forest sector and poverty and the treatment of forest related issues was generally weak. Considering this, surprisingly many forest-related responses and actions were proposed in the poverty reduction programmes (2002:123)."

An exploratory review of the relationship between small-scale fisheries and PRSPs in a 2002 SFLP sponsored study across eleven Sub-Saharan African countries was also equally critical of the incorporation process, suggesting that - with the exception of a few countries - the "situation is not very satisfactory." Worse, few fisheries-related responses and actions appeared in the final documents, as sector-pertinent issues were generally "diluted within proposals from the agriculture or rural development sector" (SFLP, 2003:3). While the study encapsulated ongoing work on poverty-profiling within the region, and was understandably strong on process given the envisaged outcomes of the SFLP work programme[33], it unfortunately failed to produce a unifying qualitative methodology which would permit the synthesis of individual country findings.

Rather than derive a new methodology to rectify this oversight however, we instead choose to apply the assessment framework espoused by Oksanen and Mersmann, in effect substituting "fish for forest". Not only is their framework relatively straightforward to apply, it also has the added advantage of permitting cross-sectoral comparisons vis-à-vis the effectiveness of each sector's incorporation into national policy documents[34] - although this is not the task here. Box 5 therefore outlines the assessment methodology applied, and the scoring scale deployed to produce the results outlined in Section 4.2 below.

Box 5
Assessment methodology applied

Criteria 1 (Issue): Were fisheries related issues included in the analysed documents?

Criteria 2 (Causal links): Were the causal linkages between fishery-related issues and poverty related issues analysed in the documents?

Criteria 3 (Responses): Were fisheries related responses and actions defined in the documents?

Criteria 4 (Process): Were links between the document formulation process and fisheries related policy and planning processes detailed in the document?

Each of the four criteria was given a numeric value where;

0 = no mention
1= mentioned, but not elaborated upon
2= elaborated
3= Best Practice

This permits an average aggregate score to be computed for each analysed document, values ranging from 0 (sector is not mentioned in the document at all) to 3 (best practice evident on all four counts).

To produce as comprehensive and systematic a picture as possible of the extent to which the fisheries sector has been integrated into the national development discourse, we apply the above methodology to analyse contemporary PRSPs (Interim and Full) and NDPs produced across the developing world[35]. As such documents - particularly PRSPs - are expected to form the basis for subsequent donor assistance, we also analyse the extent to which fisheries related issues highlighted through such national development agendas are indeed reflected in the subsequent donor support strategies[36]. Section 4.1 details these findings, adopting a regional perspective.

[30] Shyamsundar and Hamilton (2000) conflated the thirteen criteria deployed by Ekbom and Bojö (1997) to six: problem identification, problem treatment, extent of environmental mainstreaming, identification of an environment-poverty link, recognition of linkages between policy interventions and environmental change, and impact of incentives on natural resource/environmental issues.
[31] On a scale of 1 (issue absent) to 4 (best practice), East Asian CAS's gained a mark of 2.97, compared to just 1.81 for the Eastern Europe and Central Asian regions (Shyamsundar et al., 2001:12).
[32] The authors also noted that the review process would be extended to Joint Staff Assessments undertaken by Bank and IMF staff, and Poverty Reduction Strategy Credits (2001:21/2).
[33] These include; increasing the capacity of communities and their partners to participate in planning and management; guaranteeing that fisheries communities needs are reflected in national poverty alleviation planning, and ensuring that policies, institutions and processes are informed by SFLP experiences and knowledge (SFLP, 2003a:1).
[34] This, of course, would depend on some consistency between the subjective assessments of different authors although, given the rather discrete scale involved, likely differences of opinion are reduced.
[35] It should be stressed that the purpose of this research is to measure fisheries incorporation into PRSPs, NDPs and donor support programmes. It is beyond the remit of this circular to ascertain whether the identified links, responses and processes subsequently impact in the manner intended on policy formulation or implementation.
[36] Given the plethora of donors and donor support documents we limit our analysis to the Country Assistance Strategies (CAS) of the World Bank and the Country Strategy Papers of the EU as these are the most widely available. Equally, as said donor support documents follow from - as opposed feed into/inform/interact with - underlying national development strategies the 'process' criteria becomes redundant and is dropped.

Previous Page Top of Page Next Page