74. The evaluation took place some six years after the initiation of country-level work under the Special Programme for Food Security (SPFS). It was undertaken both in response to the request of the Governing Bodies and to meet internal management needs, and was designed with two aims, namely to (i) provide a credible accountability report on the SPFS, containing in-depth analysis and assessment of its continuing relevance, effectiveness in achieving results and overall cost-effectiveness; and (ii) consolidate and enhance the knowledge base of the SPFS for the future by learning from the experience to date, especially by identifying emerging issues, strengths and weaknesses.
75. A representative team of nine senior external consultants13 undertook the evaluation. The FAO Evaluation Service provided operational support. The evaluation team visited FAO Regional Offices and 12 SPFS countries from each of the developing regions (i.e. Bangladesh, Bolivia, Cambodia, China, Ecuador, Eritrea, Haiti, Mauritania, the Niger, Senegal, the United Republic of Tanzania and Zambia). Countries for visits were selected by the evaluation team from a shortlist prepared by FAO of 18 countries representing each of the developing regions. In the shortlisted countries, work had been ongoing in the field for at least three years, with at least three of the four components of the SPFS. The criteria for selection ensured that the team would be evaluating on the basis of substantial experience in implementing the SPFS. Approximately one week was spent in each country by groups, which normally consisted of four consultants. In each country the team held discussions with government, donors and FAO staff, and visited a sample of project sites using a checklist of points based on the terms of reference to facilitate their inquiries with farmers, national development agencies and SPFS staff.
76. The full text of the report, as contained in document PC 87/4 (a) Independent External Evaluation of the Special Programme for Food Security, is divided as follows. Chapter 1 of the report is the Introduction that summarizes the Terms of Reference and discusses evaluation modalities and arrangements. Chapter 2 provides some background on the SPFS covering the rationale for a focus on food security and the development of the programme concept. Chapter 3 deals with the planning and design of specific SPFS programmes and the actual process of project formulation. Chapter 4 is focused on SPFS implementation and management and considers the role of FAO, the organization and management structures, the effectiveness of national inputs, South-South Cooperation (SSC) and the roles played by other international agencies and donors. Chapter 5 is devoted to assessing pilot field operations in terms of selection of sites, target beneficiaries and technologies for testing. The approach used in implementing the SPFS field initiatives is also discussed, as are the results achieved. Chapter 6 considers the impact of SPFS on national policies and the donor community, and also briefly deals with the cost-effectiveness of SPFS initiatives. Finally, Chapter 7 brings together the material presented in the earlier chapters by summarizing the background and strengths of the SPFS as viewed by the evaluation team. This provides the foundation, which the evaluation team uses to propose the approach to be applied in planning and implementing the SPFS in the future.
77. When the SPFS started it had what the evaluation team feels was a rigid design. It was also required initially that it be implemented in those areas where there was the potential for rapidly increasing production. These areas were characterized as being where there were irrigation possibilities. It was envisioned that the production focus would help solve food security problems both at the household and national levels.
78. It soon became apparent that the early “micro”-oriented production focus was insufficient to ensure progress in solving the food security problem and that “macro” and “meso” type issues were important in enabling production increases to occur, and in ensuring benefits accrue to the producers. Over time, the implementation of the SPFS has become less rigid and more flexible as exemplified in the evolutionary table provided by the FAO Regional Office for Latin America and the Caribbean. Thus, practical realities in implementation have compelled SPFS programmes to incorporate elements in the early part of Phase I that were previously conceptualized as being dealt with in Phase II or at the earliest during the expansion part of Phase I. Such initiatives have been found to be particularly appropriate when they can be addressed at the local level without requiring changes in policy.
79. In addition to the design flaws, which became apparent in implementing SPFS activities, another problem of a more conceptual nature became apparent to the evaluation team during the visits to the case study countries. This relates to the likely trade-offs between fulfilling the goals indicated in the guidelines for the SPFS for addressing food security at both the national and household levels. In general, the stipulation of initiating SPFS activities in higher potential areas is likely to be better in addressing the issue of improving national food security. Poverty, and hence individual household food insecurity, is likely to exist in such areas but by the same token it is likely to be less acute than in less promising agricultural areas.14 As a result, in the case study countries, in general the sites selected for SPFS activities have been of relatively high productivity, compared with the more marginal areas where the degree of malnourishment in rural areas is higher but the potential for increases in agricultural productivity is lower. Thus, although in the opinion of the evaluation team the areas selected for SPFS activities are likely to be the best as far as potentially improving national food security, in terms of improving individual household food security, the impact of SPFS would probably have been higher in more marginal areas. This suggests trade-offs between the stated laudable goals of improving both household and national food security.
80. The rigid type design that characterized the SPFS at its inception had another downside in the sense that, because it was not flexible, it was not amenable to being adjusted to the specific priorities and strengths of countries and consequently sometimes probably inhibited the development of collegiate relationships and a sense of ownership on the part of some SPFS countries. It was also introduced as a stand-alone programme not linked to other ongoing or planned activities of other agencies including NGOs.
81. Another issue which became apparent during the visits to the case study countries was that the time initially planned for the pilot part of Phase I of the SPFS, namely two or three years, was far too short, and the number of selected sites was too small to have any major impact on production and food security strategies. Participatory approaches, if they are to be effective, take time, and this combined with annual variations in climatic and socio-economic conditions and the time required to develop sustainable input distribution and product marketing outlets, obviously all imply the anticipated time period was too short. In fact, in the visited countries, where explicit SPFS activities were still being implemented (i.e. excluding Zambia), the major focus, except in Senegal, was still on the sites and communities where such activities were initiated up to five or six years earlier. Success of the SPFS type of approach is very dependent on the strength of the institutional structures, including extension, input distribution, marketing and credit systems. Where there are deficiencies in this, it is very unlikely that a two to three year period will be sufficient to demonstrate impact. Evidence of implementing the expansion part of Phase I (i.e. extending SPFS activities to all agro-ecological zones in a country) was only found in Senegal, although plans do exist on paper for other countries. Also, there is no country that has entered Phase II of the SPFS.
82. Given the above issues, what can be done to improve the design and implement SPFS activities in the future that will improve their potential efficacy, impact and acceptance to both national programmes and potential donors? To address this, the evaluation team first assessed what the current strengths are of the SPFS, which help provide a useful foundation on which future initiatives can be built. Based on this, the team then agreed on what should be the major focus of the SPFS. This then led to a consideration as to what might be the optimal strategy for the SPFS to adopt.
83. The SPFS, as it currently exists, has a number of positive characteristics or strengths, not always shared by other donor and FAO-supported programmes, that deserve recognition and can be usefully built on in designing and implementing future SPFS-related initiatives. The major characteristics of the SPFS are as follows.
84. The SPFS is currently being implemented in 62 countries. However, a major concern of the evaluation team is whether, given the limited resources (i.e. financial and human) available to FAO, it has the capacity to deal adequately with all the countries currently eligible for SPFS. Currently, the criterion for eligibility to participate in/benefit from the SPFS is generally based on being a low-income food-deficit country (LIFDC), although a few of the countries currently included in the SPFS do not fit that criterion.
85. The advantages of using this criterion are that:
86. On the other hand, the problems of using this criterion are:
87. After consideration of the above factors, the evaluation team has come to the conclusion that some form of prioritization of LIFDC countries is necessary in order to prevent the limited resources of FAO being spread too thinly and to improve the prospects for impact of the SPFS. The issue is what criteria should be used in the prioritization process. The evaluation team feels the criteria would differ according to whether the country is planning an SPFS project or whether the country is desirous of continuing an SPFS activity started earlier.
88. For a country wanting to initiate an SPFS activity. The evaluation team suggests that four criteria should be considered once a country (i.e. usually an LIFDC) has expressed interest in an SPFS activity:
89. For a country wanting support for continuing an SPFS activity initiated earlier. This would be based on the same criteria as those for initiating an SPFS activity given in the preceding paragraph, plus three others, specifically whether:
90. Because of the position of FAO, as a global organization, needing to be sensitive to, and equitable in its treatment of FAO member countries, particularly the poorer ones, the evaluation team suggests that this prioritization exercise should not be viewed as a means of excluding certain countries. Rather, it is suggested that low priority as far as FAO-supported SPFS type initiatives are concerned should be translated into high priority for other FAO initiatives (e.g. help in strengthening and/or upgrading of the extension service), particularly for those that could help rectify the deficiencies identified as reasons for the country receiving low priority for SPFS. Also, in some situations, prior to proceeding further to continue SPFS-related initiatives (i.e. in the SPFS terminology, the expansion of Phase I) it may be appropriate for FAO to assist food security development in other ways including the policy setting, institutional capacity building for development in marginal areas, and assistance in such areas as early warning, disaster preparedness and organization of targeted food safety net programmes.
91. The evaluation team fully supports the emphasis placed on food security by the SPFS but as indicated earlier, it is concerned about the possible trade-offs between household and national food security. To resolve the issue, the evaluation team suggests consideration is given to the following:
92. Based on the evaluation team’s observations earlier in the report, it recommends that three basic principles should underlie the initiation or extension of SPFS initiatives in specific countries. These are:
93. In the design exercise itself, there are six specific areas that the evaluation team believes should receive greater attention since they impact not only on the way strategies for household food security are developed but also on the potential degree, sustainability and multiplier impact of SPFS-related activities. These are discussed below.
94. Explicit consideration of seasonality. For poor households heavily dependent on agriculture as a means of livelihood the degree of food security varies seasonally. Technological recommendations have to take into account the “normality” of good and bad years. For the rural poor household, food security is influenced by the agricultural production cycle, the amount of food stored, and the cash flow. This clustering of factors is most pronounced in seasonal rainfed marginal areas, but even in irrigated areas, it is a crucial issue for farming households with very limited resources. In both rainfed and irrigated areas, diversification activities not so dependent on water availability are important in cushioning households from the negative impacts of seasonality, and increasing their resilience to shock and negative trends through diversification of their production systems and income sources. The evaluation team believes it is extremely important to use the seasonal nature of food security, as a rational systematic starting-point for designing strategies to improve household food security during all periods of the year. The components, which might appropriately be termed “counter-seasonal strategies” (CSS), consist of:
95. In the countries visited by the evaluation team, SPFS has mainly dealt with CSS1, not very much with CSS2, to a certain extent with CSS3a (but not in the most food-insecure areas), only sporadically with CSS3b (i.e. a few food-processing activities), and in a few cases with microcredit support (CSS4). If future SPFS-related initiatives are to have a clearly visible identity and boundaries sharply defined around the issues of seasonal hunger and counterseasonal strategies, this will require continuation of efforts to increase food production at different times of the year, to improve implementation of strategies relating to food storage, post-harvest losses, grain banks, marketing, processing and credit, and to enter into partnerships with other agencies that have relevant experience/expertise, for example in income-generating activities (i.e. including off-farm) and rural financial institutions. Mutually satisfying partnerships with other agencies require that FAO does not claim ownership and most of the limelight, and will enable FAO to focus on areas in which it has more experience. Also, much greater effort should be made to work with farmers’ organizations (i.e. such as in Senegal), farmers’ groups and communities.
96. More explicit consideration of environmental issues and ensuring congruency between production and ecological sustainability. Although the concept of the SPFS emphasizes the importance of increasing production without undermining ecological sustainability, the evaluation team believes, at least in the case study countries, it has not always received much explicit attention. The explicit emphasis has tended to be very much on yield-increasing technologies. Although in a couple of countries (e.g. Ecuador and Cambodia), there was some attention to IPM and organic production methods, there was less attention to environmental issues than the evaluation team would have anticipated. Therefore, the team recommends that in the future more explicit attention is given to designing strategies that will ensure congruency between production and ecological sustainability than has been the case to date. Attention to this issue will become even more critically important as SPFS-related activities are extended into more marginal areas.
97. More explicit attention to gender equality. The evaluation team believes mainstreaming gender equality in a location-sensitive manner in the SPFS country programmes needs to receive more explicit attention. The recently developed FAO Gender and Development Plan of Action provides guidelines for the SPFS to mainstream gender equality, partnership with country government gender policies.
98. More explicit attention to linkages. It is desirable for the SPFS to use a more systematic, explicit and planned approach to establishing linkages, not only with donor agencies but also with other developmental actors/agencies (e.g. NGOs). Moves towards doing this need to be initiated even at the design stage since they can play a potentially important role in determining the specific character of the strategies that are planned and are feasible, and in increasing the scope and potential impact of SPFS-related activities that are implemented. Examples of two specific and potentially very useful linkages are with developmentally oriented NGOs and with research institutions. Development-oriented NGOs could sometimes provide support in implementing savings/credit schemes, grain banks, community marketing and processing facilities, and supporting diversification in terms of types of activities not mainstreamed into SPFS diversification initiatives (e.g. crafts and off-farm employment). FAO, because of its mandate, has to concentrate on agriculturally related activities, but potentially important contributors to household security or the sustainable livelihoods of poor households are non-farm sources of incomes that could be facilitated through the development of creative linkages with other agencies that are focusing on such areas. This would be coherent with the right to food, as discussed in the documentation for the World Food Summit: five years later.
99. More explicit attention to macro and meso-level institutional and policy issues. At the macro level this relates particularly to public distribution of food, pricing policies, subsidies and WTO issues. Meso-level type issues include credit, finance, input distribution, market identification, and development, community negotiation, agreement and action in terms of watershed development and land tenure related type issues. All such issues will often need to be dealt with in partnership with other agencies.
100. Acceptance of a longer time period for achieving impact. A more realistic time period (e.g. up to five years) is required to develop and incorporate the above considerations adequately.
101. Given earlier comments about a more flexible design with reference to SPFS initiatives, the evaluation team believes there would be merit, before the design exercise per se, in implementing one or two activities, namely:
102. An exit strategy as far as FAO is concerned in terms of handing over responsibility at the end of the implementation period has to be thought through in advance and gradually implemented. It is recommended that a participatory log-frame approach is used in the design exercise and that during the implementation phase it becomes a participatory dynamic log-frame, which is periodically revisited to facilitate monitoring and evaluation with respect to the objectives, indicators, means of verification/measurement, and hypotheses/assumptions/risks associated with the project. The latter column is of particular importance since it relates to the policy environment and offers a bridge with the constraints analysis.
103. Finally, four complementary strategies need to be developed. These are to:
104. We commend the independent external evaluation team for preparing a succinct, helpfully critical and constructive review of the Special Programme. From the outset, when the SPFS was launched in 1994, we have seen the need to adapt the Programme to respond both to the lessons emerging from experience gained in its implementation as well as to changes in the broader development environment. We welcome this report as a most important contribution to this learning process.
105. The report provides a wealth of useful observations on the Special Programme that will facilitate a well-informed debate on its achievements and its future. We expect that the representatives of those countries which are hosting SPFS activities, financing them or contributing to their implementation through providing SSC will have much to say in this debate, and we will be particularly interested in their observations.
106. The evaluation team’s work was deliberately focused on countries which had entered the SPFS early in its existence because it would be here that there would be the longest track record for review. Many of the concerns expressed in the report about the design and implementation of these early SPFS projects have, we believe, been largely addressed both in later projects which were not covered in this evaluation as well as in SPFS Phase I Extensions. We accept, however, that there is room in the SPFS for further improvement and evolution. Some of this is a matter of closing the gap – which exists in any large-scale development programme and from which the SPFS is not immune – between intent, as set out in guidelines, and what actually happens on the ground. But there is also a need for more fundamental adjustments.
107. In responding, we do not wish to burden the FAO Governing Bodies with detailed comments on the team’s analysis but to concentrate mainly on the report’s recommendations as to how the SPFS might be further improved in future. While we need to learn from the past and take note of the team’s observations, especially when these are critical of certain aspects of the Programme, it is on the future that we need to focus our efforts. Should members, however, wish to seek our views on specific comments and statements in the analytical sections of the report, we will be pleased to share them.
108. However, there are three broad themes on which our perception of the SPFS does not converge entirely with that of the independent review team.
It is in this context that we feel that the results to date of the South-South Cooperation (SSC) initiative and its impact on the processes of innovation are encouraging, especially if compared with other more conventional modes of technical assistance. We believe that this justifies the continued application of the main principles and elements of the current guidelines, adapting them, as required, to country-specific situations. What distinguishes the SSC model being applied by FAO in the SPFS from more traditional forms of technical assistance is that it places strongly committed technicians with good practical skills as change agents out in rural communities where they can interact directly with front-line extension workers, local leaders and farmers. In this way, they can break the conservatism which often inhibits innovation, bring on a daily basis new and very practical ideas directly into the farming environment and encourage groups of farmers, fishers and animal producers to adapt and test approaches to livelihood improvement that have been found to work well.
To increase the multiplier effect of SSC and to avoid creating long-term dependence, all SSC technicians are expected to share their experiences after practical work in rural communities with national staff in training of trainers. This is a part of an exit strategy, which normally limits engagement periods to three years. The strategy, adapted to local conditions in each participating country, includes the training by SSC experts and technicians of trainers at the national level who in turn train other trainers at regional and community level, thus creating a snowball effect.
The costs per SSC expert and technician (about US$12 000 and US$7 200 per year respectively, shared between the source, host countries and FAO initially but later paid by bilateral and multilateral donors) are very low relative to conventional technical assistance (typically in the range of US$120 000 to US$200 000 per expert per year depending on funding source). This makes it possible to field in stages a critical mass of technicians under SSC arrangements, thereby stimulating a process of locally adapted change from below in many rural communities.
This formula is still new and we shall clearly need to make adjustments on the basis of feedback from the countries involved, but we see it as an exciting and affordable way through which developing countries can transfer successful experiences amongst each other, contributing in a very practical manner to food security and agricultural development. We, therefore, intend to continue playing a catalytic role in helping interested countries engage in SSC agreements and in assisting them in mobilizing the necessary financial resources.
It would be presumptuous to imply that the SPFS alone has prompted a shift in policy orientation. We believe, however, that through its high visibility, its demonstration that it lies within the capacity of countries to bring about rapid improvements in the output of small farmers, and its links into the World Food Summit process, the Programme can claim some responsibility. At national level, the decisions of a number of LIFDCs to extend the programme nationwide (e.g. China and Pakistan) as well as of nine other countries to commit their own funds on a significant scale to implement the SPFS countrywide, would seem to indicate that it is beginning to impact on policies.
109. We agree fully with the team’s assessment of the strengths of the SPFS and with the recommendation that it is on these that future SPFS activities should be built. We concur with their views on the importance of flexibility in the Programme and on the need to balance microlevel measures to improve production and livelihoods with addressing meso and macro issues, which could impinge on both the production and the distribution of benefits. We accept the case for a longer time horizon for pilot activities under Phase I and for larger and more sites, representative of all major agro-ecological regions of a country. Indeed, wherever resources allow, this is the direction in which recent SPFS initiatives, including those financed by developing countries from their own resources, are moving.
110. We also agree on the need to update and improve guidelines and to ensure that they are widely accessible and used. In line with the team’s proposal, an SPFS Guideline Technical Committee will be established. Work is already in progress to develop cost-effective methodologies for impact monitoring, covering both production gains and their impact on household income and food security. FAO Management also accepts that subsidies, whether on inputs or on interest rates, should be avoided except where these are consistent with national policies or might be required to indemnify participants against pioneering risks (which would otherwise be met by the state through their funding of additional on-station research). The widespread use of subsidies noted in the report is not in line with the underlying concept of the Programme that it should promote replicable innovations. One of the practical problems faced by many farmers interested in taking up new practices, however, has been the collapse of credit, input supply and marketing systems in a large number of developing countries. In such situations, it has been necessary to intervene in a pragmatic way to bridge the gap between what should be done and what can be done to enable innovators to have access to the necessary inputs and equipment.
111. In considering options for the future of the SPFS, our observations are as follows.
112. The development context and environment in which the SPFS is now operating has changed considerably since it was launched almost eight years ago. The proclamation of the Millennium Development Goals, the expansion of debt relief programmes, the launching of the Comprehensive Development Framework and of Poverty Reduction Strategy Papers and the thinking on new modes of development financing emerging from the International Conference on Financing for Development, all pose challenges and offer new opportunities for the SPFS which the Organization will address.
113. The Committee welcomed this important evaluation, which it felt had been fully independent, thorough and objective, providing balanced and constructive criticism to strengthen the SPFS. Although the evaluation had been costly, the Committee suggested that independent external evaluations of other selected FAO programmes could be useful. The positive response of Senior Management and the intention of the Organization to draw on the evaluation in reinforcing the SPFS, were also appreciated. The Committee also found the explanations of the findings by the evaluation team leader useful and informative.
114. The evaluation had emphasized the potential role which the SPFS could play in overcoming food insecurity. Impact from the SPFS could only be assured when there was national ownership and when development occurred from the bottom up with broad stakeholder involvement. This required time and a flexible design responding to individual country requirements.
115. The Evaluation Report had pointed to examples of positive impacts from the SPFS as well as to many of the problems faced by the Programme. Several Members drew attention to their own experiences with the SPFS, which had been reflected in national policies and resulted in desires to expand the programme. The commitment of FAO Management to put in place a more effective and practical monitoring and evaluation system for national SPFS programmes was very much welcomed, with emphasis on assessing economic viability and replicability of approaches, as well as their immediate impact on household food security.
116. There was agreement on the primary importance of household food security, and the Committee discussed how this was to be best addressed within the context of the SPFS. Several Members stressed the priority need of the poor who cultivate marginal sites and who have limited access to food. Others noted that the poor were not restricted to marginal areas and that in LIFDCs a primary concern was to assure an adequate national food supply. The Committee emphasized the need for a better gender balance in the SPFS.
117. The Committee discussed the findings of the Evaluation Report with regard to the desirability of prioritizing cooperation under the SPFS. Several Members felt that it was essential to address the needs of all LIFDCs and noted that several non-LIFDCs had also demonstrated their desire for the programme by substantially funding work by FAO in their own countries. Other Members of the Committee emphasized that, in a situation of finite resources, prioritization to achieve real impact from Regular Programme resources was essential. In their view this would also facilitate the greater mobilization of donor funds.
118. The Committee agreed that SSC brought a valuable new dimension to technical cooperation. Some Members emphasized that the programme was playing a key role in extending experience between countries. Others also noted that there was a need to relate SSC closely to the specific needs and absorptive capacity of host countries and that greater engagement of national expertise could also be important. The Evaluation Report contained useful practical ideas to increase the efficacy of SSC, including the need for gender balance and family visits by cooperators.
119. The constructive criticism in the report provided valuable assistance to FAO Management as it sought to increase the impact of the programme. The Committee welcomed the information it received on the flexible and people-centred approach which, it was informed, had come to be a feature of the SPFS in more recent years. The Committee welcomed the initiatives now being taken by Management to strengthen the SPFS and implement recommendations of the evaluation team. The Committee was informed of the arrangements now being put in place to secure a fuller engagement of the technical expertise of the Organization and to ensure greater integration of environmental aspects, policies on food security and FIVIMs. Preparations were being made to set up a Technical Support Group, update the Concept Paper on the SPFS, revise guidelines and to train concerned staff in project design and implementation approaches, with an emphasis on participative methods. Preparation of a Monitoring and Evaluation Manual was in progress.
120. In conclusion, the Committee welcomed the commitment evidenced by Management to use the recommendations of the evaluation team to strengthen the SPFS and its approach. It requested a follow-up report at its May 2003 session on progress made in implementing the SPFS and in introducing the many positive changes that Management had referred to in its responses to the report.
12 The full text of the report and the Senior Management Response is found in document PC 87/4 (a).
13 Mr Dunstan S.C. Spencer, Team Leader (Sierra Leone); Mr Pierre Spitz (France); Mr Frank Anderson (Australia); Mr Manuel Contijoch Escontria (Mexico); Mr Antanas Maziliauskas (Lithuania); Mr David Norman (USA); Ms Maija Sala (Finland); Mr Vijah S. Vyas (India); Mr Mahgoub G. Zaroug (the Sudan).
14 While the numbers of relatively poor may be fairly high in high potential areas, the type of strategy that the SPFS focuses on, namely improving agricultural productivity, may at best only benefit such households indirectly (i.e. through creating seasonal employment opportunities since many are likely to be landless). Some types of diversification activities may provide some direct benefits but the most useful strategy for helping such households is likely to be outside the remit of specific SPFS activities, e.g. creation of off-farm employment.
15 However, such data do not cover all countries, are not as reliable as LIFDC data, and are not as widely accepted as LIFDC data.
16 The fact that demand to take part in the SPFS is strong and not confined to LIFDCs would suggest that some of the team’s assertions about lack of national ownership might be questioned.
17 PC/87/REP, paras. 21-28.