PC 92/6b)


Programme Committee

Ninety-second Session

Rome, 27 September – 1 October 2004

Synthesis of Findings of Two FAO Internal Evaluations of Work at Country Level
(FAO Response to the Continuing Crisis in Southern Africa and FAO Post-Conflict Programme in Afghanistan)

Table of Contents

I. Introduction

II. The Results of FAO Programmes

III. Relevance of the FAO Responses and Priorities for the Future

IV. Issues in Planning and Management of FAO’s Programme

ACRONYMS

 

1. During 2003, the Evaluation Service carried out evaluations of FAO’s work in southern Africa1 and in Afghanistan2. These were commissioned by the Emergency Operations and Rehabilitation Division (TCE), on behalf of the Technical Cooperation Department as a whole. Both evaluations were internal, examined FAO’s emergency, rehabilitation and development programmes in their entirety, and had their emphasis on assessing implementation and outcomes in order to learn lessons for the future. The evaluations gave attention to issues raised and recommendations in the Thematic Evaluation of Strategy A.3: Preparedness for, and effective and sustainable response to, Food and Agricultural Emergencies3. These were the first evaluations carried out of the totality of FAO’s programme in a country or geographical area in many years. They provided a cost-effective supplement to thematic evaluations by drawing lessons on general issues for the Field Programme and identifying location-specific country issues. The evaluations were able to cover small interventions which would not justify individual evaluations, including FAO-TCP and emergency projects. It could be desirable to institutionalise this form of evaluation, utilising a mix of TCP and project evaluation funding to contribute to FAO as a learning organization and the provision of substantive accountability.

2. The evaluations were both conducted by teams comprised of independent external consultants and members of the FAO Evaluation Service. The Afghanistan evaluation was externally led. For each evaluation, there was a desk review of available written materials followed by country visits. The southern Africa evaluation visited Angola, Malawi, Mozambique and Zambia with approximately one week in each country. The Afghanistan evaluation team spent two and a half weeks in country. For both evaluations, extensive consultations were carried out. In southern Africa, this involved participation in a workshop by the FAO Sub-regional Office for Southern and East Africa, FAO Representatives and Programme Officers and FAO Emergency Co-ordinators. For Afghanistan, workshops were held to discuss the evaluation team findings with FAO staff in Afghanistan and in the Regional Office for Asia and the Pacific. Reports were discussed and commented upon in draft within FAO before finalization.

3. In analysing FAO’s response, it is important to recognise the very different nature of the emergencies in southern Africa and Afghanistan.

4. Southern Africa: The evaluation finds that, although there are distinct differences between them, many countries of southern Africa are in the grip of a downward spiral of poverty contributed to by HIV-AIDS, failures of governance (including conditions for economic growth), declining soil fertility and inadequate donor investment. There has been an increase in rural households subject to continuous food insecurity and in households subject to seasonal food insecurity. The increase in the number of vulnerable households means that any crisis, whether it be due to climatic shocks, civil disturbance or economic mismanagement, becomes increasingly difficult for the communities to absorb. Thus, the sub-region is suffering from continuing food insecurity which is worsened by each crisis. The 2002 drought was seen as part of this context. There was no large-scale displacement of people from their homes, but the food situation deteriorated drastically.

5. The evaluation finds that the momentum to tackle HIV-AIDS is growing but it is certain that infection rates in the active population in rural areas will continue to rise for the coming years and deaths from AIDS will rise more steeply from the currently infected population. In agriculture, the number of single-parent households, orphan households and households headed by the elderly will grow. This will further compound with the effects of malnutrition, frequent pregnancy, and diseases such as malaria to lower the productivity of labour. At the same time, despite the increase in illness and death among the active working population, the rural and urban populations will continue to grow, albeit more slowly.

6. The emergency situation of Angola is different and parallels with Afghanistan are closer, as the evaluation took place some 14 months after the April 2002 peace accord, with assistance primarily addressing the needs of communities with returning displaced people and the displaced people themselves.

7. In Afghanistan, over twenty years of political and military conflict, combined with major climatic and natural constraints, have caused a series of humanitarian crises, weak law and order, and growth in illicit drug production and trafficking. Following the fall of the Taliban regime at the end of 2001, and the installation of an interim administration, the joint UN Country team in the appeal for the Immediate and Transitional Assistance Programme assessed the situation as one of the most desperate in the world, with over one million displaced persons and high levels of malnutrition. FAO had been working in the country for over 40 years, against a backdrop of civil war, civil unrest and political and economic isolation. The success of UN appeals gave the Organization a major opportunity to launch an assistance programme which made FAO’s operations in Afghanistan its second largest in global terms after Iraq. However, the extent of disruption posed major problems for all agencies in responding to the emergency. There was a huge influx of agencies, a government just finding its feet, continued fighting in several areas and an infrastructure destroyed by years of neglect and conflict.

8. The major differences between the FAO support in southern Africa and that in Afghanistan and also Angola, was that in the latter two cases, development assistance had largely ceased and had to be restarted following an immediate emergency response. In Afghanistan, government had to be virtually re-established and in Angola, major parts of the country had little government structure, whereas in the rest of southern Africa this was not at all the case and development assistance had continued alongside a crisis response to the 2002 drought. In Afghanistan, evaluations and audits of several agencies’ work have been highly critical of these agencies’ performance, perhaps making inadequate allowances for the realities on the ground. Many of the problems FAO met with in delivering its programme were thus found to be beyond the Organization’s control.

9. It almost goes without saying that information is a prerequisite for any meaningful policy or action and the better the information base the more appropriate that action is likely to be, whether it be an emergency or development response. In southern Africa, the recent emergency brought the importance of early warning and vulnerability information sharply into focus and information was the only area in which all funding agencies spoken to by the evaluation team gave a priority to the role of FAO. Similar recognition was found in Afghanistan.

10. In all of the southern African countries visited, FAO has been active in the establishment of information systems. In Angola, rapid rural appraisal of the agricultural sector provided part of the basis for an agricultural recovery and development options review. In Mozambique, part of the assistance to the comprehensive agricultural development programme (PROAGRI) was for information, in particular agricultural census. Also in each of the countries, FAO supported the development of early warning and food security information systems. Emergency coordinators frequently had, as part of their terms of reference, the establishment of an information system to supply information on donor and NGO response (such systems were generally linked to forecasting of input requirements). At the regional level in southern Africa, support was given to development of the Southern African Development Community (SADC) capacity, especially for remote sensing. SADC was found to provide an essential and unique service to its members in remote sensing and related GIS analysis, a clear case of the regional solution being the most cost-effective. However, this function needs to be further strengthened and is not going to become self-sustaining within SADC within the foreseeable future. The field is also advancing quickly, meaning that with technological change, investment in improved systems and training remain a continuing necessity.

11. The evaluation also found that food security, nutrition and vulnerability information functions had not been firmly established in government departments in southern Africa. There has been enduring donor involvement in Mozambique, but in other countries earlier donor support was not sustained. The underlying reasons for the lack of sustainability in information systems were found to have included: i) the exclusive reliance on civil service capacity; ii) a great deal of overlapping of initiatives by various donors, WFP and also by FAO itself, which encouraged fragmentation of effort and a lack of sustainable critical mass; and iii) systems which tried to collect too much data on a regular basis. Other problems were the lack of attention to the information needs of potential users and to means of disseminating and packaging information to make it usable. In general, donors and NGOs have been the largest users of information and a major challenge is to make information relevant to national users.

12. In Afghanistan, prior to the change in regime, FAO had developed a geo-referenced database containing agriculture-related information. This was outdated and incomplete but served as a starting point. Two projects were initiated post-conflict, one for the immediate provision of data to FAO, WFP and other agencies to respond to immediate needs, and another longer-term project, which was designed to provide information for decision-making on food security, nutrition and livelihoods and also to develop capacity in government, partner agencies, and communities. These projects came into Afghanistan at a time when the government situation was very fluid and also a large number of agencies were establishing information systems. Within FAO it also seems that there was an internal lack of cooperation as one project was operated by TCE as an emergency project and the other by RAP as a development project. The evaluation found that despite this, effects were being seen: a crop monitoring system was being established; and cropping surveys had been carried out. There was general agreement on the quality of this work which supported the overall FAO/WFP crop and food supply assessment report. The presentation of this FAO/WFP report created the first real forum for discussion between the government and the international community on food security and food aid issues.

13. In southern Africa, the evaluation found that donors and FAO had now recognised the need for much greater coherence in their support. A networked approach, also involving some non-governmental actors, was making an encouraging start in Mozambique. In Afghanistan, the FAO projects were working closely with the Famine Early Warning Systems Network (FEWS-NET) and with WFP. The southern Africa evaluation concluded that sustainability could not be looked for in the medium term based on government funding and government staffing. It suggested that over time FAO support the formalization of governmental and non-governmental relationships, as well as basket funding from donors, which it considered would continue to be major users of the information systems. Such basket funding should also be sought for SADC. Although it is unlikely that all encompassing systems can be designed addressing every aspect of vulnerability, the maximum degree of complementarity should be the aim. In general, basic systems should collect the essential minimum of data and supplement this with special studies when required.

14. The Afghanistan evaluation concluded that setting up food security information systems takes more time than is generally realised. In countries where institutions are in disarray and there is paucity of data, FAO/WFP crop and food supply assessment missions (CFSAM) provide a useful prime set of analyses for better understanding food aid and agricultural relief needs. Such missions were perceived by all to be an area of FAO comparative advantage, as FAO is neutral and the missions are underpinned by a firm methodology. It was recommended that the Organization develop a two-stage approach in complex emergencies to food security information. In the immediate post-crisis response, FAO should concentrate on the preparatory work for a CFSAM through support to crop assessment surveys, using existing networks of field workers (e.g. WFP food aid monitors). This stage of quick information assembly requires further methodological underpinning and guidelines. As the institutional environment stabilizes, FAO should aim at setting up in-country capacity for food security assessment. The role of FAO in this phase and the extent of its involvement in the full range of food security and nutrition aspects should very much be based on the Food Insecurity and Vulnerability Information and Mapping System (FIVIMS) principles, a careful assessment of the various governmental and non-governmental actors involved, as well as gaps in existing systems. The model followed in Afghanistan was found to provide a sound example for flexible application elsewhere.

15. It was stressed that the use to which information is to be put should always be uppermost in the minds of the system designers. In the first phase, information should be geared towards humanitarian and relief needs, but it can contribute to the establishment of grass-roots monitoring capacity. Later, emphasis will shift to information for strategy and policy for rehabilitation while also ensuring information is provided in a timely way on vulnerability. The evaluation team emphasised that information systems were a distinct area and served different audiences.

16. In both southern Africa and Afghanistan, the evaluations found that information systems and policy were probably the two areas on which there was the greatest degree of agreement from stakeholders and partners that FAO had a key role. In southern Africa, the extent to which the Organization could play a substantive role in policy was found to be closely linked to the capacities of the FAOR and to the extent FAO was involved as a substantive player in support to government-donor coordination mechanisms for the agricultural and food sectors. Thus, the role in Mozambique was found to have been particularly effective and in Zambia there had been some impact. Specialist support through longer-term projects had also been a major factor in both cases, as had the involvement and support of the FAOR in dialogue with government. Individual policy advisory reports were found to have had rather limited impact.

17. In Afghanistan at the end of the war, there was a rush of policy and strategy documents. FAO was involved both in the preparation of the multi-donor Medium-term Development Framework led by the Asian Development Bank and separately prepared a “Strategy for the Early Rehabilitation of Agriculture”. This was criticised by the Afghan Government as not separating the overall government strategy from that of FAO, not dealing fully with vulnerability, which was the main concern at the end of the war, and failing to address adequately where FAO had a comparative advantage. The evaluation found that this document had limited ownership within FAO. On the other hand, FAO’s contribution to the Medium-term Development Framework was better appreciated but a follow-up strategy document which defined implementation in terms of projects was also found by the evaluation to have received limited attention. FAO’s role as a policy interlocutor at this early stage was limited by not having an FAOR in place. Due to the interest of the Ministry of Finance in a coordinated approach across sectors and a prominent role to the International Financing Institutions, the Asian Development bank took over from FAO as coordinator of the Consultative Group on Natural Resource Management. The Organization has, however, emerged as the main assistance to policy development within the Ministry of Agriculture and is providing continuing support by utilising inputs from a variety of projects.

18. In Angola at the end of the civil war, FAO also sent a mission to update development strategy. This concentrated on national priorities and it was too early at the time of the evaluation mission to judge its impact.

19. Part of the conclusion of the evaluation with respect to the policy process in southern African countries has wider implications for FAO’s work. It was “concluded that there have been significant changes in the way national policies and strategies are determined, and this has implications for the way in which FAO assists countries and how the Organization can make a contribution to reinforcing democracy and improved governance within democracy. An important feature of the way in which policy making has changed is the growing role of parliaments and thus of parliamentarians. The fact that many parliamentarians are elected by rural constituencies makes it more difficult to neglect the needs of the rural areas. On the down-side, essentially populist short-term decisions become more likely and the role of formal strategies and policies may be diminished. The influence of civil society organizations, the media and the business community has also grown. Donors appear to increasingly base their resource allocations on their agendas for policy change and there is greater agreement among donors on the changes they are working for. The role of civil services in policy making has reduced, both because of the increased influence of the other parties referred to above, and because of declining capacity. The law has become less easy for the executive to ignore, due to the role of parliament, the media and civil society organizations. This means that policy making and implementation becomes a more iterative, ongoing process. It may also be based to a greater extent on what is seen to work and catches the popular imagination. FAO can no longer prepare a policy with the civil service and expect it to be implemented. The nature of the process necessitates continuous support and advocacy. There is a need to facilitate the public debate, assisting the issues to be understood and answers derived nationally. Politicians, the media and civil society need to be targeted for information. Thus, aspects of information system development become closely related to support to policy development.”

20. The southern Africa evaluation also noted that immediate post-conflict situations provide a window of opportunity for change which can subsequently become more difficult. The experience in Afghanistan has, however, also shown that real dialogue on policy and strategy is often difficult immediately post conflict, especially when a large number of actors are involved and national interlocutors are poorly established. There seems to be greater effectiveness when the focus is on agreeing priorities between the national stakeholders and the international community, while leaving the agreements on particular interventions to the government and donors, being prepared to assist in this where requested. Even in a post-conflict situation, FAO’s greatest effectiveness in policy thus seems to have been in commitment to a longer-term process.

21. Emergency assistance: FAO’s role in coordination of agricultural assistance in major emergencies was an important issue for both evaluations. In Afghanistan, apparently some donors originally looked to FAO for a coordination and immediate strategy development role, but there was also a major problem of a large number of agencies and of competition. FAO, which had no FAOR in place, lost momentum but perhaps at a time, immediately post-conflict, where rather little could have been achieved. The Organization then gradually built up its role through the provision of information systems and substantive participation in fora such as those for seeds.

22. In southern Africa, it was found that where FAORs committed themselves at the time of the formulation of the UN emergency appeal and strategy (CAP), agricultural issues were much better covered than where this input had to rely on a newly-recruited emergency coordinator, who had neither the seniority nor the background knowledge. FAO also more immediately gained a leadership role in coordination. The southern Africa evaluation found that this role was facilitated by FAO having been granted sufficient funds to be a major contractor of NGOs in agricultural relief work. The NGOs were thus willing to exchange information and respect some coordination by FAO. FAO was also able to undertake gap filling where programmes funded by others were not active. Something of a virtuous circle was created in that the donors recognized FAO’s coordination role and funded it, thus FAO was better able to perform the role. On the other hand, in Afghanistan the evaluation concluded that FAO’s major role in distributing inputs through NGOs and other partners was seen partly as competitive by other agencies and these agencies considered that it detracted from the credibility accorded to FAO in strategy development or coordination.

23. Development assistance: FAO’s role in coordination for development assistance was examined in southern Africa. Here, the evaluation finds that FAO can play a useful role vis-à-vis donors, when the donors and government look to FAO in this capacity and it performs the role as part of the UN country team. It is a role that grows when FAO makes substantive inputs into the coordination process, which is closely linked to government policy dialogue with the international community, and the FAOR’s personal capacities are critical.

24. Institutions in the broad sense of the word underlie a country’s capacity to develop, but of the various aspects of FAO assistance examined, the southern Africa evaluation found the least evidence of sustainable impact in institution-building and with the exception of information systems discussed above, FAO assistance in this area also appeared to have been declining. In several of the countries visited, the capacity of civil services was found to have weakened. This is due to policy changes on the role of government, budgetary constraints, resignation of staff to take up higher paid jobs elsewhere, and increasingly losses due to HIV-AIDS. However, it also means that capacities built relying on the civil service are often unsustainable. Individual FAO-TCP projects with their short duration were, in particular, found to have yielded very limited sustainable impacts in institution-building, but examples of limited impact from longer-term assistance extended from veterinary services to marketing.

25. The evaluation concludes that in southern Africa, there needs to be a very realistic appraisal of the justification for establishment of capacities at national level in relation to the stage of development and the potential of regional institutions, such as COMESA, SACU and SADC, bearing in mind that many of the countries have relatively small populations and very small economies. Activities are thus often more cost-effective through sub-regional organizations but these organizations are also often institutionally weak with few resources to fund recurrent budgets. The evaluation notes, that in several countries, it is both government policy and the de facto situation on the ground that development activity, including agricultural extension, is carried out with government – NGO partnerships. The NGOs often channel resources to extension workers and other civil servants to enable them to assist in development activities. It is concluded that at the present stage of development and public sector capacity for provision of goods and services, continuing donor support is justified to carry out central functions. Information systems are particularly crucial in this context and transfer of development knowledge to rural populations is another key area, with extension services largely dysfunctional. It is, however, often unreasonable to expect that government will be able to entirely fund or staff these even in the medium term. Such capacities can probably best be continued with networked solutions and public-NGO-private sector cooperation.

26. On the other hand, in Afghanistan (post 2001) FAO was only able to undertake the work it did in seeds and animal health as a result of former institution building efforts. Improved seed production had been established during the 1990s working with government at central and provincial levels and with NGOs, who contracted Afghan farmers as seed growers. A similar approach but also with more community involvement and the development of barefoot vets/extension agents occurred in animal production and health. With the change of regime, much of the community, NGO and provincial government aspects of these programmes were able to continue with little change. This Afghanistan experience thus reinforced the concept of institution building in depth, involving central government, local government and NGO partners, rather than the more traditional approach of concentrating on central government capacity and relying on central government to translate this into capacity at local level. The evaluation team concluded that FAO should now play an important role in strengthening institutional capacity at the central ministries which had been so devastated by the conflict and years of isolation. It also concluded that FAO projects which have been free standing and working more with non-governmental partners should now be generally fully integrated into government institutions to maximise their capacity building effects.

27. FAO work in emergencies is always intended to protect or rehabilitate agriculturally based livelihoods. This has normally taken the form of distributing seeds and tools to allow planting for the forthcoming season but as is evident from the discussion in this note, longer term rehabilitation was an immediate concern in strategy development for Afghanistan. In southern Africa there was diversified technology development (see below). In both cases, the evaluations welcomed the move towards more tailor-made responses in emergencies.

28. Seeds: Both evaluations gave attention to the provision of seeds and other inputs in an emergency to help in agricultural rehabilitation. In Afghanistan, it was concluded that all agencies in the country had had inadequate information on seed needs and there may have been over supply and some undermining of markets, due to free distribution. FAO could have been expected to take a lead in providing reliable information but was subject to many of the same limitations as others in the immediate post-war situation. The evaluation team found that although FAO had suffered in reputation during the first seed campaign, the second, a few months later in 2002, was much more orderly and in line with needs. Also through its support to what has now become the National Seed Council, FAO made a major contribution to much improved information sharing.

29. In southern Africa, experience was more positive right from the beginning and FAO was able to work through strong NGO networks. Although the seriousness of these should not be exaggerated, the biggest problems were timeliness of delivery, appropriateness of varieties and quality. As in Afghanistan, emergency purchasing on the regional market stretched capacity to the limits. The Afghanistan evaluation noted that despite the failure of one seed company to deliver in the first season, under FAO rules, it was allowed to bid and again awarded the contract in a second season when once again it failed to deliver and the project funding was returned to the donor. In southern Africa, the evaluation team was also told of similar problems of seed delivered too late for planting and that it was difficult to get adequate account taken in the central FAO placement of seed contracts of guaranteed delivery dates and previous performance of the company. Instances were reported of the seeds provided not being of the variety specified and written on the label and of poor germination rates.

30. Both evaluations recommended that these problems should be addressed and the southern Africa evaluation specifically recommended that there should be clauses in contracts with seed companies which provide heavy penalties for late delivery and provide for retention of monies until after the germination and purity is known. Also, an FAO intra-net southern Africa data base could be maintained on the performance of seed companies. To speed up and simplify the preparation and clearance of seed purchase and also to help ensure that all seed does make some contribution to genetic improvement, an agreed FAO seed list should be developed for southern Africa, by agro-ecological zone, and kept updated. This, the evaluation considered, would be valuable to other agencies, not just FAO.

31. Both evaluations address the issue of the disruptive effects of seed and other input distribution on input markets. In southern Africa, all purchasing was done within the sub-region, thus giving a boost to production in the commercial sector. An issue in both situations was the extent to which free distribution would mean farmers under-valued the seed and also local trade was disrupted or even destroyed by the input distribution schemes. In southern Africa, input fairs with vouchers were piloted with initially encouraging results. The advantage of seed fairs was found to be that they provided a stimulus to local trade and gave farmers choice, while allowing some control on the quality of inputs provided and limiting the potential to divert input vouchers to other purposes.

32. Locusts in Afghanistan: The locust campaign was found to have been timely and cost-effective in saving food which would have had to be replaced with food aid. It was noted that FAO was working on development of an integrated pest management programme to reduce the need for future emergency interventions.

33. Irrigation rehabilitation was an important part of the emergency response in Afghanistan. Part of this was providing an information base for future planning, but immediate rehabilitation work was also initiated through the provincial irrigation departments which were trained in the application of community based self-help approaches. The evaluation team found this to already be yielding benefits, but was concerned for sustainability of the capacity being created. This was, however, likely to be secured through a World Bank financed project executed with FAO technical input. Dissemination of treadle pumps was part of the emergency response in southern Africa with very encouraging results but there was some concern about marketing the incremental vegetable production.

34. Livestock: The emergency response in Afghanistan built on previous work which had established a cadre of community livestock extensionists and was to some extent a continued development activity, in that it combined artificial insemination services for cattle, fodder relief and animal health. The programme also included a policy advisor and livestock census. The evaluation found that the programme had contributed to rehabilitation of the herds and flocks, reaching around five percent of the national cattle herd, almost half the sheep and some 20 percent of goats. The nomadic flocks were more difficult to cover.

35. Livestock were found to have been neglected in the emergency and recent development responses in southern Africa, both in the initial appeal and in donor response to requests. The priority was for the control of transboundary epidemic diseases, but the limited TCP funds available for this had not proved effective. Since the evaluation FAO has secured more substantial funding for livestock disease control, but at the time of the evaluation, it was concluded that in line with the policies of EMPRES, considerably more attention needed to be given to livestock emergencies and the role of livestock in vulnerable households’ food security and survival strategies. The evaluation states that in most countries of southern Africa, with the continued decline of government services, a new strategy is required for epidemic diseases which can bring in additional players and resources. FAO can support development of this new strategy and include donors in the thinking from the start in order to get buy-in. In the view of the evaluation team, elements in this could include: a) rethinking of early warning systems for livestock disease, including the role of non-governmental actors, especially traders, and the integration of livestock information systems with those on other aspects of vulnerability, including crop and vulnerability assessment; b) cross-border cooperation more locally organized, relying less on central veterinary departments in distant capitals; c) reinforcing legislation, where necessary; d) greater involvement of NGOs and private veterinary auxiliaries (barefoot vets) in all aspects of prevention and treatment; and e) development of international agreements, including for the support of control in weaker countries by those economically and institutionally stronger.

36. The emergency responses and dissemination of technology: The evaluation in southern Africa found valuable examples of FAO using emergency assistance to disseminate available technologies, in particular technologies which had been originated with NGOs. This included conservation farming in Zambia and treadle pumps and mini irrigation schemes in Malawi. Improved planting material was disseminated, especially in Malawi, for cassava. Strong points were that all this work built on existing experience and training was included. There was cooperation at local levels between the NGOs and government extension agents. The main concerns were lack of attention to marketing, particularly in the irrigation. There was also a concern that emergency funding was of short duration but this was to some extent offset by the continued commitment of the NGOs and the early impact of the improvements which could be sustained by farmers. The evaluation team noted that this partnership between NGOs and the government extension agents appeared to have been more cost-effective than the government-run SPFS schemes and that the emergency-funded programmes with the NGOs had secured a greater self-help input from the farm families. It was, however, too early to ascertain eventual sustainability.

37. In Afghanistan, FAO had been assisting the development and production of improved seeds for many years. It was able to draw on this existing capacity to disseminate further improved germplasm. At the height of the seed distribution, about 18,000 tons of quality declared seed was being produced from this base but there were major problems of quality, a problem largely overcome in subsequent seasons. At least 50 percent of the irrigated wheat currently grown in the country is attributed to the FAO programme going back over 20 years. The impact of FAO genetic material in rainfed areas is also significant, but less so. These varieties give an average yield increment of about one third for irrigated areas and of about a quarter under rainfed conditions.

38. Localised emergencies: The southern Africa evaluation considered FAO’s response to more localised emergencies. It noted that in many cases, donors are reluctant to provide emergency assistance, tending to see patterns of recurrent disaster as ongoing problems to be tackled rather than emergencies. Although it is evident that these are very serious for the affected communities, it is argued by these donors that response is something which should largely be handled by local coping mechanisms and government and NGO relief. The evaluation concluded that in southern Africa such emergencies formed part of the pattern of continuing vulnerability and governments could not be expected to respond to these unaided. It was, therefore, important for FAO to work with governments, donors, other members of the UN community (in particular WFP) and major NGOs to establish mechanisms of coordinated and joint rapid response at the national level, avoiding the necessity to formulate individual requests.

39. Both evaluations examine the underlying issues in agriculture and human development as they relate to the crises and consider the relevance of the current FAO response and where FAO’s focus should lie in future. Both place emphasis on policy work, including major sub-sectoral issues. Food security assessment and information were also found to be areas of common priority and FAO comparative advantage.

40. In Afghanistan, high priority is placed on institution building and reform, including of the agriculture sector ministries. It is considered that capacity building should be integrated into most interventions. Well targeted agricultural relief will remain a priority for vulnerable groups. In southern Africa, the emphasis in this respect is placed on the policies and safety-net programmes to be put in place for continuing vulnerability. Both evaluations stress building on success in emergency operations, carrying this forward into the rehabilitation and development phases. In southern Africa, this applied particularly to technology and the farmers’ field school approach. In Afghanistan, it included work in seeds, plant protection, small-scale irrigation and horticulture. FAO was considered to have had a positive experience in the development of community-based sustainable livelihood approaches. Also in Afghanistan, the role for the Organization in partnering others for a comprehensive livelihoods-based approach to the reduction of poppy growing, was an important priority.

41. In its analysis of the crisis in southern Africa and agriculture’s role in addressing that, the evaluation emphasised that the best buffer for vulnerable households is a food surplus and income within the community and that the best buffer for vulnerable communities is rising prosperity in the country. Agricultural development focused on basic food security for all, thus needs to promote both growth and safety-nets for households which are in continuous or periodic food deficit. In the rural economies of southern Africa, agriculture remains the main potential motor for economic growth as well as the main potential origin of decline. It was concluded that there will be a growing number of rural households which cannot be viable. These will need to receive assistance for food, either continuously or on a seasonal basis. They also need to be provided with the means to produce what they can for themselves, with the least requirement for physical effort.

42. Households with depleted and weak physical labour, including from the effects of HIV-AIDS, need, to an even greater extent than others, to increase economic returns and food production per unit of labour and to reduce the need for physical strength. Probably the single most important way of reducing this physical effort is to increase the value of production per unit of area. The ways to do this include irrigation, higher value crops and fertilizer application, but other factors of importance include: i) income from small livestock; ii) use of farm power especially animal drawn equipment; iii) spreading the work of cultivation during the dry season; and iv) reducing domestic labour. Intensification was also considered to be the main solution to the problems of declining soil fertility on fragile dry lands and overall it was considered that irrigation had to come much more to the fore in providing for intensification. There was also potential in some countries to facilitate movement by households that wished to transfer to higher potential areas.

43. The evaluation was concerned that while social and medical responses to the problems of southern Africa were a priority, the role of agriculture in household, community and national viability was not receiving adequate attention. Particular factors identified by the evaluation in an enabling environment for equitable economic growth include removing barriers to trade at sub-regional level and within countries and finding ways of encouraging both job creation in commercial agriculture and the growth of small-scale entrepreneurs. Land tenure issues were important for both local food security and growth. The evaluation concluded that FAO should support these developments in its policy work and, where possible, should assist implementation in these priority areas through projects.

44. Both evaluations found that definition of FAO priorities for the immediate response to emergency and for rehabilitation and development was an area in which improvements needed to be made. In southern Africa, FAORs complained that FAO’s response, especially through TCP, was fragmented. The evaluation concluded, however, that this was only really true in one of the countries studied and elsewhere FAO had generally concentrated its limited resources. They recommend that FAO’s assistance to a country should be formulated and integrated within a coherent framework. Such a framework should highlight priorities based on FAO’s comparative advantages and should be prepared in close collaboration with local partners.

45. The southern Africa evaluation arrived at similar conclusions to the Decentralization evaluation4 and recommended the development of rolling priority frameworks under the leadership of the FAOR which would provide the setting in which FAO should work with donors for the mobilization of funds and provide TCP support. The Afghanistan evaluation drew similar conclusions but with respect to transition situations. It stresses the need for FAO to develop a clear and shared vision with partners and states “where there is a dynamic pattern of sometimes contiguous, vulnerable and less vulnerable groups, a multiplicity of actors and a combination of humanitarian, rehabilitation and development support, it is important to define a common vision as early as possible of what FAO’s role should be. This role should be articulated around two main objectives: (i) improving resilience of the most vulnerable through protection, restoration and strengthening of livelihoods; and (ii) laying the basis for an enabling environment for growth in which agriculture has a role to play. This strategy for FAO’s role in a post-conflict situation should: a) be based on a needs assessment; b) differentiate clearly between FAO’s role and the broad needs of the sector; c) be prepared in close coordination with local partners; and d) build on FAO’s previous experience in the country.”

46. In emergencies, identification and targeting of beneficiaries and needs assessment were found to be areas for improvement by both evaluations. The southern Africa evaluation noted that it is also important to find out what intended beneficiaries actually want. There appeared to still be a general reliance on rather standard packages of inputs, without taking into consideration the diversity of farming conditions, state of vulnerability and capacity to produce. The team commented, for instance, on the issue of tools, where the drought or flood had not actually destroyed tools and farmers' interest in receiving certain varieties and not others. It also felt that it was useful to distinguish the extent of vulnerability in designing responses and draws a distinction between those households which: a) need no assistance even in an emergency situation and have the potential to produce more for the benefit of the community; b) with assistance will be able to move towards household sustainability; and c) can produce something but will still remain non-viable and will continue to need food assistance.

47. The Afghanistan evaluation argues that emergency agricultural inputs distribution requires a comprehensive assessment of needs, based on an analysis of livelihood vulnerability and its causes. The challenge is to carry out such an assessment within a relatively short timeframe, using quick and dirty methods while being comprehensive. Particular care should be given in the analysis to the relevance and role of agricultural inputs distribution as a means to maintain and/or restore livelihoods.

48. Both evaluations refer to the difficulties of leaving final targeting of individual households to the communities concerned but both argue that with the exception of returning displaced persons, this is generally the only practical solution. Communities should however be informed of both the criteria and the reasons for them in providing inputs or other assistance to households.

49. The Afghanistan evaluation argues that project design criteria in general need to be clearer and include such elements as the use of simplified log frame analysis and measurable indicators to facilitate monitoring and evaluation for better management and accountability. Stronger support to the FAO Project Cycle Review Training Course would help in the task of spreading good practice throughout the Organization.

50. The southern Africa evaluation also notes that many emergency projects are second or third generation, not the immediate response to a calamity, there is thus more time to undertake both needs assessment and project design. It then goes further in addressing the issue of time and the need for sound appraisal in the formulation of emergency response immediately following a disaster by suggesting that maximum flexibility regarding projects formulation for donor funding should be allowed in an emergency situation. The definition of process rather than key details needs to be built into the projects which should be designed in detail after the budget has been approved, often in consultation with beneficiaries.

51. Both evaluations confirmed many of the findings and recommendations of the evaluation of Strategic Objective A3. A particularly valuable feature of the response in southern Africa was found to be the extent to which FAO had closely coordinated its response within the UN system. This had included full support to joint missions and placing its regional coordination staff in the joint UN agency secretariat, RIACSO5.

52. In Afghanistan, following the cessation of the main hostilities, FAO immediately appointed a senior officer (the Director, TCE) as focal point. On the ground in Afghanistan, commendable efforts were made to fill management positions early on and to field other staff but these were hampered by the difficult conditions. For a variety of reasons, there was a high turnover in most of the first year. The FAOR, when he did take up his duties, was new to an FAOR post and relatively new to FAO. The evaluation found that this lack of one person in charge on the ground severely hampered FAO’s work. The evaluation also found that there was a serious problem between the development projects operated from RAP (including those for seed production and for information) and the emergency and rehabilitation projects operated by TCE in Rome. Even with an FAOR in place, these divided lines of responsibility were reflected in the weak unity of purpose on the ground. Although in Angola, there was not a problem of lack of an FAOR following the peace agreement, difficulties arose between the FAOR with implementation authority for development projects and the emergency coordinator.

53. The Decentralization evaluation recommends that FAO should have a core of FAORs/Senior Emergency Coordinators for rotation into major complex emergency situations, replacing the existing FAOR. The Afghanistan evaluation concluded that if such an arrangement had been in place for Afghanistan some of the difficulties might have been avoided. This is also the case in Angola. The warnings of the Decentralization evaluation report on assigning responsibility to offices or individuals without an emergency mentality also need to be heeded, as the Afghanistan evaluation found that some of the difficulties were due to those concerned with development projects not giving sufficient attention to meeting the immediate needs of vulnerable people.

54. The Afghanistan evaluation also emphasises the immediate appointment post-conflict of a senior focal point until such time as an FAOR of the right calibre is fielded and the appointment of a senior advisory group and coordination groups, as was done for Afghanistan. This evaluation concluded, as did the Decentralization evaluation, that in complex emergency situations it was not desirable to have projects operated by both TCE in Rome and operations units in the Regional Offices. Given the emergency nature of the situation, TCE was the logical operations point for all projects until an adequate FAOR could be fielded with the necessary infrastructure.

55. This evaluation also identified a problem with more complex rehabilitation interventions, where the support costs for emergencies were not adequate to cover the desirable level of technical support from FAO and the TCE delivery focus could reduce the extent to which the necessary analysis was undertaken.

56. For any programme, whether it be development or emergency, inputs which arrive after the planting season are of little use, at least for that season. In the case of the immediate aftermath of emergency, the generally stated intention is to ensure a food crop for the following season. This issue was addressed at length in the evaluation of Strategic Objective A3 and both the evaluations of Afghanistan and southern Africa gave it attention. They concluded that there was a need for measures to speed up the whole process of funding, input purchasing and distribution and concluded that there were major problems associated with the lack of flexibility and degree of delegation of authority by FAO. Both evaluations called for realism in emergencies on when it will be possible to meet the next season and when it will not. The southern Africa evaluation concludes that where it is not possible to meet the next season, it may be better to plan more carefully for a fully appropriate response in the following season. The evaluation thus considered that FAO should establish a minimum acceptable period between the completed formulation and approval of an emergency project and the latest date for satisfactory planting. The evaluation team suggested that this was some three months, unless there is the capacity in terms of management and input availability to carry out all operations locally, when it may be of the order of two months.

57. Both evaluations call for efforts to improve the quality and the timeliness of delivery, including procurement. It should be recognised that there is a lack of full understanding of where the principle bottlenecks in the delivery chain are and the reasons for these. Since these two evaluations took place, a number of studies were carried out looking at improving FAO’s efficiency in delivering its services to member countries (and going beyond the scope of emergency situations). Recommendations have been made and some of them also taken up in the Decentralization evaluation. In emergency contexts, the two evaluations made some specific recommendations for:

  1. Training on administrative procedures, including procurement procedures of staff and FAORs;

  2. Increased levels of authority to Regional Offices and FAORs for procurement;
  3. Modification of FAO procedures to facilitate rapid procurement from a shortlist of pre-qualified suppliers, without going to external bidding; and
  4. Introduction of clauses in contracts with input supply companies which provide heavy penalties for late delivery and provide for retention of monies until after the germination and purity of seed is known.

58. Both evaluations commented negatively on the very short contracts and high turnover in emergency personnel. This was in part due to the temporary nature of the funding but the evaluations concluded that there was room for more continuity and the southern Africa evaluation also concluded that this could reduce the number of personnel transactions involved, improving this aspect of efficiency.

59. The southern Africa evaluation called for realism in expectations from internal monitoring systems. It found examples of NGOs collecting and analysing a surprising amount of information on beneficiaries’ response. In general, however, monitoring of results was not strong. In some countries, the personnel on the ground, whether NGO or government, are limited in their capacity to understand, and thus communicate to, the beneficiaries’ requirements for complex information. Informal inspection had sometimes been weak and has an important role to play. Information needs to be kept simple and deal with key essentials. Basic monitoring does, however, need to be a contractual obligation of the partner organizations delivering assistance and where the target is settled communities needs at a minimum to: a) identify beneficiaries by category, geographic location and gender; b) specify the assistance provided to each beneficiary; c) report on use of the assistance (e.g. were all seeds planted); and d) cover results of the assistance supplied, in terms of benefits compared with production which was not assisted. More complex and evaluative information is generally best handled in independent sample studies carried out post-season under separate contracts with, for example, universities as was being done in one case in Zambia.

60. The Afghanistan evaluation found that FAO had tried to put in place an overly heavy intranet supported evaluation system in Afghanistan. System design and implementation had been hampered by conflicts on methodology by units in FAO headquarters including the Evaluation Service. It was concluded that the system was now yielding some useful outputs but could not be sustained without continuing external support. It was recommended that FAO develop a viable model for a programme monitoring and information system which can be rapidly deployed in complex emergencies.

61. Both evaluations emphasised the important place of monitoring and evaluation in emergencies and emphasised that this should be provided for in project budgets and external evaluation should become more the norm. The evaluations also considered that overall FAO programme evaluation for individual countries and groups of countries should be institutionalised.

 

AIDS

Acquired Immunodeficiency Syndrome
CAP UN Consolidated Inter-agency Appeal
CFSAM  Crop and Food Supply Assessment Mission
COMESA Common Market for Eastern and Southern Africa
EMPRES

Emergency Prevention System for Transboundary Animal and Plant Pests and Diseases
ESAF

Food Security and Agricultural Projects Analysis Service
FEWS-NET Famine Early Warning Systems Network
FIVIMS Food Insecurity and Vulnerability Information and Mapping Systems
GIS Geographical Information System
HIV Human Immunodeficiency Virus
NGO Non-governmental Organization
RAP FAO Regional Office for Asia and the Pacific
RIACSO Regional Inter-agency Coordination and Support Office
SACU Southern Africa Customs Union
SADC Southern African Development Community
TCE FAO Emergency Operations and Rehabilitation Division
TCP FAO Technical Cooperation Programme
UN United Nations
WFP World Food Programme

_________________________

1 June-July 2003: J. Markie and C. Tarazona, FAO Evaluation Service; M.J. Watt and M. Zaroug, consultants; and G. Hemrich, FAO-ESAF.

2 October 2003: N. Chapman, consultant-team leader; A. Fitzherbert and R. Lough, consultants; and R. Sauvinet-Bedouin and C. Tarazona, FAO Evaluation Service.

3 PC 88/5 a), September 2002.

4 The Evaluation of FAO’s Decentralization PC 92/6a), September 2004.

5 RIACSO Regional Inter-agency Coordination and Support Office