Indicative Rolling Workplan of Thematic and Programme Evaluations 2003-2007
Table of Contents
Programme of Proposed Evaluations
Action Requested from the Programme Committee
Annex 1: Scope and Approach of the Suggested Evaluations
1. The purpose of this note is to seek the advice of the Programme Committee on the workplan for major independent evaluations to be carried out by the Organization under the direction of the Evaluation Service, in particular during the next biennium 2004-05.
2. At its 86th Session in September 2001, the Programme Committee discussed topics to be covered in future evaluations. The Committee “welcomed the consultation process with the Director-General in selecting topics for major programme evaluations to be presented to the Committee” and “agreed that increasing emphasis be given to thematic evaluations in the context of the Strategic Framework and the Medium-Term Plan”. It also recognized “the need for flexibility in planning evaluations” and “decided to review on a biennial basis the planned coverage of topics for possible adjustments”.
3. The Evaluation Service will complete seven major evaluations for presentation to the Governing Bodies in the current biennium 2002-03, as follows:
- Animal Health Component of Programme 2.1.3 (May 2002);
- Emergency Prevention System for Transboundary Animal and Plant Pests and Diseases (EMPRES) – Desert Locust (May 2002);
- Special Programme for Food Security (May 2002);
- Strategic Objective A3: Preparedness for and Effective and Sustainable Response to Food and Agricultural Emergencies (September 2002);
- The Codex Alimentarius and Other FAO and WHO Food Standards Work (May 2003);
- Statistical Activities of FAO (May 2003); and
- Crop Production (including programme entities within Programmes 2.1.2 and 2.1.5) (September 2003).
4. An evaluation of decentralization is being initiated in 2003 to be made available to the Governing Bodies in 2004 (see Annex 1). In addition, the current plan for the period 2002-05, as reviewed by the Programme Committee in 2001, included three additional topics as indicated below. However, as the Programme Committee envisaged in requesting the review of the programme of evaluations on a biennial basis, the relative priority of topics should be kept under review in the light of changing circumstances and new priorities arising. The Evaluation Service’s assessment on these three topics is as follows:
- Strategic Objective D1 (Integrated management of land, water, fisheries, forest and genetic resources) – the salient components of activities falling under this Strategic Objective have been covered by recent evaluations on Fisheries and Forestry Resources Programmes and it is proposed that it should be substituted by related Strategic Objective D2 (Conservation, rehabilitation and development of environment at greatest risk), because of the latter’s significance in terms of management of resources related to food and agriculture as well as of poverty alleviation and food security concerns;
- Cross-organizational Strategy on Communicating FAO’s messages – some important aspects of this strategy have been covered by the in-depth review of FAO’s publications and other information products, and hence it is considered more useful to give priority to the Cross-organizational Strategy on Broadening partnerships and alliances; and
- Programme 2.1.3 Livestock (Livestock Production, Information and Policy Aspects) – this topic could be retained but be covered in the second half of the proposed programme. In addition, it is suggested that priority be given to Programme 2.3.3 (Fish Exploitation and Utilization) as this programme has not been covered before and in view of the topical nature of this subject in the fisheries sector.
Programme of Proposed Evaluations
5. The intention would be to produce 5-7 evaluations in each future biennium, depending upon their scope and the available resources. All the evaluations will integrate field and normative aspects as appropriate. In this regard and in order to meet the Organization’s commitment to systematically evaluate the Technical Cooperation Programme (TCP), at least two evaluations per biennium will include substantial coverage of related TCP projects within the evaluation topic. The Programme Committee welcomed the assessment of programmes within the framework of the Strategic Framework and in particular, the comprehensive approach which systematically reviewed normative activities as well as a balanced sample of operational activities1. As a consequence, the proposed evaluations will assess those programmes, programme entities and Field Programme interventions most closely associated with the Corporate Strategies, Strategic Objectives or Cross-organizational Strategies under review.
6. In addition, the Committee will recall that it will also receive a biennial summary of auto-evaluations undertaken by programme managers. These will cover all Technical Projects (TPs) at least once during the projects’ life and all Continuing Programmes (CPs) at least once every six years. The first such biennial summary will be provided to the Committee in 2005.
7. In order to ascertain potential priority topics for evaluation, an analysis was made of the evaluations completed during the last three biennia against: the Strategic Framework (Corporate Strategies and Cross-organizational Strategies); the FAO programme structure; and interest indicated by the Governing Bodies. Department heads were also consulted. On the basis of this, the following topics have been identified, classified into two groups, with higher priority considered for completion in 2004-05 and a second group for reconsideration in 2005 with a view to possible completion in 2006-07. In Annex 1, an indication is given of the scope and proposed approach for each evaluation as well as for the two evaluations initiated in 2003 (crop production and decentralization).
Evaluations proposed with first priority for flexible initiation in 2004-05 (5-7 evaluations):
- Within Corporate Strategy B, Strategic Objective B1 (International instruments concerning food, agriculture, fisheries and forestry, safe use and fair exchange of agricultural, fishery and forestry goods) - building on the Codex evaluation, an evaluation of FAO’s role in the development and support of major international instruments and frameworks, including work related to the PAIAs on Biosecurity for Agriculture and Food Production and Climate Change;
- Within Corporate Strategy C, Strategic Objective C1 (Policy options and institutional measures to improve efficiency and adaptability in production, processing and marketing systems, and meet the changing needs of producers and consumers) – an evaluation of the strategy component, “Promoting the diversification and specialization of production to take advantage of new opportunities as well as of comparative advantages based on different resource endowments”, covering related programme entities in all the Technical and Economic Major Programmes;
- Within Corporate Strategy D, Strategic Objective D2 (Conservation, rehabilitation and development of environments at greatest risk), including related aspects of the PAIAs on Integrated Management of Biological Diversity for Food and Agriculture and Strengthening Capacity for Integrated Ecosystem Management;
- For Corporate Strategy E, Strategic Objective E2 (Regular assessments, analysis and outlook studies for food and agriculture), including the PAIA on Global Perspective Studies;
- The Cross-organizational Strategy (SACOI) Broadening Partnerships and Alliances - this is proposed to replace the SACOI on Communicating FAO’s Messages for the reasons given above. The evaluation would cover key aspects in working with partners. As can be seen from Annex 1, this would address major current issues; and
- Programme 2.3.3 (Fish Exploitation and Utilization) – the programme is a major part of the Fisheries Major Programme, covering many aspects of current interest.
8. The absence of Corporate Strategy A (Contributing to the eradication of food insecurity and rural poverty) may be noted. However, several major evaluations completed in the 2002-03 period related particularly to Strategy A. For example, the evaluations of the Special Programme for Food Security (SPFS); EMPRES (both animal health and desert locust); and of Strategic Objective A3 (Preparedness for and effective and sustainable response to food and agricultural emergencies).
Evaluations proposed with Second-Priority for consideration in 2006-07
- Within Corporate Strategy B (Promoting, developing and reinforcing policy and regulatory frameworks for food, agriculture, fisheries and forestry) – the evaluation would cover work related to the PAIA on WTO Multilateral Trade Negotiations and other trade related work falling in Programme 2.2.4 (Agriculture, Food Security and Trade Policy and in Fisheries);
- Within Corporate Strategy C, Strategic Objective C2 (Adoption of appropriate technology to sustainably intensify production systems and to ensure sufficient supplies of food and agricultural, fisheries and forestry goods and services) – this would cover the PAIA on Biotechnology Applications in Agriculture, Forestry and Fisheries and significant parts of Programme 2.1.5 (Agricultural Applications of Isotopes and Biotechnology);
- Within Corporate Strategy E, Strategic Objective E1 (An integrated information resource base, with current relevant and reliable statistics, information and knowledge made accessible to all FAO clients) – the evaluation would focus on The World Agricultural Information Centre (WAICENT) in particular; and
- Livestock policy and production work in Programme 2.1.3 as originally planned for the current biennium.
Action Requested from the Programme Committee
9. The Committee is invited to provide its views on the priorities among the evaluation subjects proposed.
Annex 1: Scope and Approach of the Suggested Evaluations
10. The emphasis is on formative evaluation to build on strengths and correct weaknesses in the future development and execution of programmes. Except where otherwise stated, evaluations will cover work over a period of approximately six years.
11. The methodology usually includes visits to a representative sample of countries and decentralized offices to review the results of work and its relevance to countries. Questionnaires will be sent to the concerned member countries, development partners and other stakeholders to assess the priority they attach to the area of work and their assessments of its relevance, effectiveness, strengths and weaknesses. Each evaluation will be led by a senior staff member of the Evaluation Service often supported by the participation of external consultants to provide subject matter expertise and regional perspectives. In a few specific cases, team leadership by an external consultant may be considered where there is a requirement for particular expertise, or for independence from FAO itself (e.g. CODEX evaluation). This general approach is consistent with the view of the Committee that external evaluations of selected topics were desirable2.
12. Each report of those evaluations which are led by the Evaluation Service will be reviewed by an independent peer review panel, which will discuss it with concerned managers and representatives of the evaluation team and make comments as appropriate on the report. The panel’s report and the management response will also be submitted to the Programme Committee.
Agreed evaluations initiated in 2003
Crop Production: This evaluation covers crop production activities within Programme 2.1.2, Crops and Programme 2.1.5, Agricultural Applications of Isotopes and Biotechnology and the related Field Programme activities, including TCP projects for the period from 1997.
Decentralization: This evaluation will concentrate on the results of the decentralization process initiated by the Organization in 1994, with a view to assessing its strengths and weaknesses. The evaluation will be formative with its primary purpose being to suggest how decentralization could be further strengthened to the benefit of member countries and to increase cost-efficiency. It will examine all the decentralized structures and functions and how these relate to the functions and structures at Headquarters. It will thus examine what functions could be most effectively centralized, as well as decentralized. In assessing this, account will be taken of the views of member countries on the services they require and receive, their efficiency and quality. In addition, the experience of other organizations of the UN system, in particular other Specialized Agencies, will also be considered. The evaluation will also benefit from the output of several internal working groups examining related issues, as well as the relevant reports of the Office of the Inspector-General. In view of the complexity of the evaluation and the need for particular expertise, it may need to be led by a senior external consultant with inputs from Evaluation Service staff and specialist consultants. Visits will be made to a cross-section of decentralized offices and countries and a questionnaire survey will be conducted. The findings and recommendations of the evaluation will probably be finalized in a workshop. Field work for this evaluation could possibly be combined with the evaluation on the Cross-organizational Strategy on Broadening Partnerships and Alliances in the interests of efficiency (see below).
Suggested Evaluations for 2004-2007
Corporate Strategy B - Promoting, developing and reinforcing policy and regulatory frameworks for food, agriculture, fisheries and forestry - Strategic Objective B1 (International instruments concerning food, agriculture, fisheries and forestry, safe use and fair exchange of agricultural, fishery and forestry goods):
- FAO’s role in the development and support of major international instruments and frameworks. This evaluation, which deals with global public goods, would build on the Codex evaluation, and concentrate on areas where FAO has had a particular lead role in either the development or implementation of the instrument. It would thus include coverage of: the International Plant Protection Convention; the International Treaty on Plant Genetic Resources for Food and Agriculture; Prior Informed Consent Procedure (PIC) for Certain Hazardous Chemicals and Pesticides in International Trade; Code of Conduct for Responsible Fisheries and the role of the various international fisheries bodies; the UN Conventions on Biological Diversity, Desertification and Climate Change (including the related PAIA); and the Law of the Sea. Work related to the PAIA on Biosecurity for Agriculture and Food Production (MTP3 paragraphs 101-103) would be covered. An International Conference on Biosecurity will be held in 2005 and the evaluation would be formative in examining FAO’s role in follow-up. Work related to WTO agreements would be excluded (see below). Attention would be given to the institutional aspects of FAO’s role, including its role as a forum for negotiation and the technical value of FAO secretariat inputs. Particular aims of the evaluation would be to draw conclusions on: the adequacy of international instruments in FAO’s areas of mandate; when binding legal frameworks are possible and most likely to be productive and when not; pre-requisites for success in developing and contributing to the development of international instruments; and resourcing the development and implementation of international instruments;
- Work related to the PAIA on WTO Multilateral Trade Negotiations including Codex, IPPC and Pesticides Prior Informed Consent (PIC) (MTP CL 123/7 paragraphs 104-106) and other trade work falling in Programme 2.2.4, Agriculture, Food Security and Trade Policy (taking care not to duplicate recent work in this area) and Programme Entity 233A5, Promotion of international fish trade. This evaluation would draw on the recently completed evaluation of Codex and other FAO and WHO Food Standards Work. An important aspect of the evaluation would be assessment of the usefulness and relevance of FAO work to member countries and the partnerships in this work with other organizations including the WTO and World Bank. This evaluation could include a sample of related TCP projects.
Corporate Strategy C - Creating sustainable increases in the supply and availability of food and other products from the crop, livestock, fisheries and forestry sectors:
- Strategic Objective C1 (Policy options and institutional measures to improve efficiency and adaptability in production, processing and marketing systems, and meet the changing needs of producers and consumers). The evaluation would focus on the component, “Promoting the diversification and specialization of production to take advantage of new opportunities as well as of comparative advantages based on different resource endowments”, including some of the work falling under the PAIAs on Organic Agriculture (MTP CL 123/7 paragraphs 111-113) and on Food for the Cities (MTP CL 123/7 paragraphs 114-116). Programme entities touching on this are found throughout the Technical and Economic Programmes, e.g. 212A1 Alternative Crops and Cultivars for New Opportunities, 214A4 Agri-business Development, 224P5 Enhancing Diversification and Competitiveness of Agricultural Commodities, 233A3 Increased Production from Under-utilized Aquatic Resources and Low-value Catches and 242P2 Appropriate Utilization of Forest Products. The evaluation would include related TCP projects and examine not only FAO’s role vis-ą-vis that of other organizations in identifying new opportunities but also in examining the potential domestic, regional and global markets and disseminating information;
- Strategic Objective C2 (Adoption of appropriate technology to sustainably intensify production systems and to ensure sufficient supplies of food and agricultural, fisheries and forestry goods and services) – covering the PAIA on Biotechnology Applications in Agriculture, Forestry and Fisheries and significant parts of Programme 2.1.5 (Agricultural Applications of Isotopes and Biotechnology). This evaluation could include related TCP projects and follow the approach indicated above.
Corporate Strategy D - Supporting the conservation, improvement and sustainable use of natural resources for food and agriculture:
- Strategic Objective D2 (Conservation, rehabilitation and development of environments at greatest risk) - including related aspects of the PAIAs on Integrated Management of Biological Diversity for Food and Agriculture (MTP CL 123/7 paragraphs 123-125) and Strengthening Capacity for Integrated Ecosystem Management (MTP CL 123/7 paragraphs 126-129). The evaluation would cover work on conservation of forest and fisheries eco-systems, mountains, desertification and wetlands. It would include a sample of related TCP projects.
Corporate Strategy E - Improving decision making through the provision of information and assessments and fostering of knowledge management for food and agriculture:
- Strategic Objective E2 (Regular assessments, analysis and outlook studies for food and agriculture) - including the PAIA on Global Perspective Studies (MTP CL 123/7 paragraphs 136-138). In addition to the State of Food and Agriculture; World’s Forests; World’s Fisheries and Aquaculture; Food Insecurity in the World; and Agriculture Towards 20XX, this evaluation would examine more specialist analysis and outlook work, such as that for commodities and animal genetic resources. It would cover regional studies, such as those for forests in Africa and studies for regional economic groupings. The evaluation would be carried out over a period of at least 12 months to enable Evaluation Service staff to assess the value of studies to countries during other missions. The principal use of consultants would be to examine the quality of work. This evaluation may include a workshop to explore the potential for increased partnership and input from other organizations into this area of the Organization’s activities;
- Strategic Objective E1 (An integrated information resource base, with current relevant and reliable statistics, information and knowledge made accessible to all FAO clients) - this evaluation would concentrate upon WAICENT content and technical performance in terms of its usefulness and accessibility to external users, internal users and in its ease of maintenance. Attention would be given to the development of partnerships within WAICENT to achieve genuinely global information access with quality assurance. Field visits by Evaluation Service staff would be included partly in tandem with other missions for discussion with users and partners. The evaluation would take place over at least one year to gain structured statistical information on who accesses WAICENT and for what purposes.
Cross-organizational Strategy (SACOI): Broadening Partnerships and Alliances (MTP CL 123/7 paragraphs 820–843):
- As has been recognized by all major UN Conferences and Summits, in particular by the World Food Summit: Five years later, partnerships are essential at international, regional, national and local levels, if progress is to be made in overcoming hunger and poverty, promoting economic and social development and conserving the environment. This evaluation would examine all forms of partnership in an overview and through case-studies. The intention would be to draw lessons on types of partnership and modalities that can work most effectively in different situations and for different purposes, including the limitations and benefits of partnerships. It would lead to a clearer analysis of conditions when partnership is likely to be cost-effective and when it is not. The Strategy specifically addresses partnerships in the UN system, IFIS, NGOs and CSOs, the private sector and with local authorities. The evaluation would also cover partnerships with developed country agencies for development programmes, as well as technical level partnerships with academic and research institutions, including those under the International Alliance against Hunger. Field work on partnership at country level would be combined with the evaluation of decentralization with a view to assessing the views of actual and potential partners.
13. These evaluations would follow the general approach as indicated above and include related TCP projects:
- Programme 2.3.3, Fish Exploitation and Utilization: This evaluation would cover the Programme as a whole;
- Programme 2.1.3, Livestock: As mentioned above, the coverage would be on livestock policy and production aspects.
1 CL 123/12, Report of the 88th Session of the Programme Committee, para. 51.
2 CL 123/11, Report of the 87th Session of the Programme Committee, para. 21.
3 All MTP references relate to document CL 123/7, the Medium-Term Plan 2004-09, FAO, August 2002.