The evaluation addressed activities to directly assist countries in policy development, and not normative policy work. The evaluation concentrated on policy formulation and excluded policy implementation, although the latter was a major criteria for judging whether policy was successful (paras. 9-10).

The evaluation was carried out through questionnaire surveys to developing member countries, review of work by missions to 21 countries in each of the developing regions, and through extensive review of materials and discussions in Headquarters and the Regional Offices. Independent consultants participated in the country missions and the report was reviewed by a panel of experts from outside FAO (paras. 12-15).

The evaluation found that policy was an increasingly important area for developing countries and the Peer Review concluded that FAO should accord its work in this area greater overall priority. There had been substantial impacts from FAO's work in the form of contributions to policy development processes and direct policy impacts were found reflected in policy implementation (paras. 70-73). It was also concluded that:

i. The need for policy in agriculture, forestry, fisheries, food security and sustainable livelihood development, as well as the sustainable management of the natural resource base, is growing, particularly as a result of globalisation and the greater reliance on indirect policy instruments to provide an enabling framework for development (paras. 3-5);

ii. FAO has a comparative strength in support for development of fisheries and forestry sector policy and in development of policies in agriculture at the sub-sector level and in the technology-policy interface (costs and benefits of choices). The Organization also has a valuable role to play as part of the UN team in ensuring that concerns for agriculture and food security are fully reflected in macro-policy and in developing the capacity of agricultural ministries to influence this process. The Organization has well-qualified competitors in agricultural sector review work (paras. 22-26);

iii. Although FAO's assistance has always met a need and fallen within the Organization's strategic priorities, these areas of comparative strength are not fully reflected in the pattern of FAO's work which has over-emphasised agricultural sector review (para. 27);

iv. Design of technical cooperation could be improved to have better impact on the policy process continuum (paras. 32-36). Administration of individual policy interventions could also benefit from reduced response times (paras. 38-40). The productivity and effectiveness of Regional Policy Assistance Branches could be raised (paras. 55-59);

v. The technical quality of FAO's policy work is as good, or better than, that of other agencies, but significant areas for improvement were found, including in the integration of multi-disciplinary input and in ensuring effective normative underpinning, especially for the costs and benefits of policies for sub-sectors and the technology-policy interface. There were areas of sub-sector policy support such as research and extension (innovation and learning) where FAO needs to increase its capability (paras. 50-51);

vi. FAO had been particularly effective in introducing more consultative approaches into policy making between government departments and with civil society, but this aspect could be further strengthened (paras. 61-64);

vii. Joint action with development partners, particularly the IFIs and full engagement in the UN country process has been essential for effective mainstreaming of FAO's policy input and its integration into macro-policy. The FAO Representative is essential for this and in continuing dialogue with the agricultural ministries (paras. 65-68).

Recommendations are made for:

viii. Developing clearer priorities in policy work including:

changing the balance of FAO's policy work in favour of fisheries and forestry and work at the sub-sector level and the technology-policy interface (paras. 77-79);
developing priorities for groups of countries (regions, levels of development) (para. 80);
emphasising the LIFDCs (para. 80); and
increasing attention to advocacy and awareness raising (paras. 83-86).

ix. Working in capacity building, with the emphasis on issue identification (rather than sophisticated techniques) and strengthening the broad-based capabilities in agricultural ministries to participate as partners in macro-policy discussion and analyse the cost-benefits of their own programmes (paras. 81-82). In training, FAO should regard itself as much as a resource for others as an executor of training and more clearly identify target audiences (para. 87);

x. Strengthening work through partnerships, with full-commitment to the UN country processes, particularly to ensure adequate attention to food security and the agricultural sectors in macro policy, including policies for poverty alleviation and livelihoods (paras. 89-92);

xi. Strengthening the flexibility in the responsiveness of FAO technical cooperation, including:

strengthening the ratio of non-staff resources to staff through elimination of posts and mobilisation of donor funds. In particular it is envisaged that the ratio of staff to non-staff resources be stabilised and the possibility considered of establishing policy cooperation funds on a global or regional basis (para. 96); and
some streamlining in arrangements for FAO-TCP (para. 97).

xii. Strengthening integration and working procedures in the Regional Offices with clear regional priorities, better defined lines of responsibility and more interdisciplinary work (paras. 98-101);

xiii. Providing greater support to FAORs in their policy role (para. 102);

xiv. Improving services to the CIS and similar countries through the establishment of specialist capability (para. 103);

xv. Developing the quality of FAO policy development cooperation, including:

the issue of guidelines for policy work (para. 104);
improving project design, in particular as regards recognition of the point on the policy continuum that the project is intended to impact, the expected policy outcome and the consultative process (paras. 105-107);
improving country intelligence, using a web-based solution (para. 108); and
establishing a task force for policy work to better define priorities and to develop the normative underpinning and related guidelines (paras. 109-110).

Findings of the External, Independent Peer Review Panel

The Peer Review Panel strongly endorses the findings and recommendations of the Evaluation. The Evaluation process was exemplary. It combined the evaluation by teams, including independent consultants, of a broad sample of policy interventions/projects at country level; discussions and questionnaires with member governments and development partners; preliminary internal reviews of findings; and an independent External Peer Review Panel. This process provided for dialogue, a thorough exploration of issues and many valuable insights.

The overall conclusions that policy development is an area of high priority in the service of member countries and that FAO should concentrate on areas in which it can develop relative strength for policy support, are shared by the Panel members. The Panel also attaches particular importance to the following findings and recommendations within the report, along with areas in which they can be strengthened, and wishes to highlight them for management's attention:

1. The priorities within policy assistance need to be identified. In the view of the Panel, broad agricultural sector reviews should have a lower priority than supporting national policy formation processes. This can be done by assisting agricultural and rural development ministries in diagnoses of particular policy issues and in helping integrate the work of those agencies into larger national policy processes. Attention should also focus on areas that are neglected from a policy point of view, for example, livestock.

2. Based upon FAO's Strategic Framework and mandate to work on food security and agricultural aspects of poverty alleviation, policy assistance should be oriented toward rural development policy, when appropriate, rather than just agricultural policy. We recognise the danger of the Organization spreading itself too thinly, but effective work on rural agricultural development and rural poverty alleviation does require a broader rural development orientation. This is applicable to middle-income countries, as well as lower-income countries, where rural poverty remains a persistent and under-recognised issue.

3. FAO can play an even stronger role than it presently does in strengthening national capacity for analysing agricultural and rural policy issues. Its role in supporting national development of policy should be catalytic, thereby promoting national "ownership" of policy reforms. FAO's basic role in the policy arena is to promote sound processes of policy analysis, formulation, and implementation. In this regard, it should continue to encourage participatory modes of policy formation that include not only other line ministries, but also NGOs, the private sector, and other UN and bilateral agencies. The panel emphasises the essential role of FAO both in providing alternative views to government and in supporting processes that promote national ownership of policy.

4. The Organization can play a pro-active role vis-�-vis the UN and donor countries concerning policy formulation to ensure that the importance of food security, agriculture and the rural sector are well-recognised.

5. In many circumstances, FAO can work in close cooperation with IFIs and other donors. In other situations, it may be called upon independently to help Member Governments formulate policy. These are not incompatible but FAO's independence and the role in which it is acting in any one situation needs to be clear both internally and to other stakeholders. This independent role is very valuable but may require careful management in situations in which FAO is also communicating with IFIs in regard to policy issues.

6. The Panel believes that increased resources need to be made available for country-level policy work. To provide effective policy assistance, there is need for flexible mechanisms and rapid response. The TCP has been the main mechanism for policy intervention. It has serious shortcomings, in that its operational requirements in many cases are inconsistent with quick, quality response. In particular, limitations on amounts that may be spent on international expertise and requirements for use of TCDC are often not appropriate for policy work. The Panel also supports the recommendation that the ratio of non-staff to staff resources be increased under the Regular Programme, and that the ratio be protected against budgetary cuts.

7. Policy expertise is now dispersed among many units of FAO and there is poor coordination and exchange of information across units. This could result in duplication of effort and certainly results in loss of potential benefits from cross-fertilisation of ideas and experiences. There is need for both the recommended task force with its chairperson as a "focal point" to provide linkage and coordination, and to be a clearinghouse of knowledge. The Panel recognised that within the Organization, many creative linkages exist based on personal networking and professional relationships, but these could be strengthened through a more formal structure for policy work. Such coordination is not cost-free. Additional incentives and mechanisms for facilitating such inter-unit collaboration need to be developed and the focal point must have resources for this. Its value-added has been demonstrated in several areas, for example, the work on trade.

8. FAORs are FAO's presence in the country and the focus for continuing dialogue. They are a critical point of entry for government requests for policy assistance. Therefore, the recruitment criteria for FAORs should reflect this skill requirement. Further, training is needed to ensure that FAORs have a general awareness of policy issues, especially those which are regional priorities. They also need stronger technical support and information. The CIS and other low-income Eastern European countries should have FAORs as a matter of urgency to facilitate policy assistance work.

9. Successful institutions are continuous learning organizations. If FAO is to strengthen its commitment to policy analysis and advice, there is an urgent need to implement enhanced training on policy issues throughout the staff of the Organization. This would include refresher training courses for existing policy specialists and attendance at professional meetings. Strengthening training has obvious budget implications, but this activity should be considered a high priority to enhance and preserve the excellence of FAO expertise. Attention to quality will also require closer matching of staff with the requirements of posts.

10. Knowledge management, including improved country intelligence and the strong link between normative work and country cooperation needs to be strengthened. Case studies about how policy issues have been approached and successfully addressed are in great demand and should be written up for both the use of the house and clients. The Organization does not do an optimal job in learning from its own experience and that of others in making this kind of specific information widely available.

11. The operational definition of policy should allow for inclusion of the implementation phase when appropriate. Good policy work often is built around a recognition from the beginning of the channels for implementation, and it may include assistance in designing the implementation phase and monitoring it.

12. Finally, the Panel feels that a strengthened policy role will require larger budgetary allocations for that purpose, along with a clear institutional mandate to carry forward the report's recommendations.

12 January 2001

Composition of the Peer Review Panel

Mr. Malcolm D. Bale, Sector Manager, Strategy and Policy, Rural Development and Natural Resources Unit, East Asia and Pacific Region, The World Bank, Washington, D.C.

H. E. Jo�o Carrilho, Vice-Minister of Agriculture, Republic of Mozambique

H. E. Dr. Ing Kantha Phavi (Ms.), Secretary of State for the Ministry of Women's and Veteran's Affairs, Kingdom of Cambodia

Dr. Roger Norton, advisor on agricultural development policy and natural resource management

Ms. Raquel Pe�a-Montenegro, Director, Latin America and Caribbean Division, IFAD

Dr. Julian Thomas, agricultural development specialist and former Chairman of the FAO Finance Committee and the Africa Group of Representatives to FAO

Mr. William Valletta, policy and legal specialist for land tenure and property rights in developing and transition economies

Views were also submitted in writing by Dr. Simon Maxwell, Food Policy Specialist and Director Overseas Development Institute, UK.

Management Response

1. The evaluation, covering specifically advocacy and policy assistance work in direct support of member countries, is fair, thorough and objective. It went through a commendable process, combining dialogue and questionnaires with countries and development partners, independent high-level external advice and sustained in-house consultation. The recommendations made are constructive and timely. They converge, together with the ongoing internal review of Field Programme development and that of the TC Department, towards a clearer definition of the mandate of the Policy Assistance Division and its linkages with other parts of FAO. The principal findings of the evaluation are endorsed, and in a number of key areas action has already been initiated to implement the required changes.

2. It is agreed that agricultural and rural development policy is an increasingly important area for developing countries, and that these require assistance in formulating appropriate policies to achieve food security and sustainable livelihoods, in particular for their rural poor, and the sustainable management of their natural resources. FAO will give greater overall priority to policy work and, to the extent possible, allot larger budgetary allocations for this purpose. To this end, limited additional resources are proposed under the SPWB 2002-03, most of which as non-staff resources to the decentralised offices, an area of need that is highlighted in the report. It has to be recognised that under a reduced scenario for overall resources this increase will be difficult to maintain, bearing in mind other high priority activities such as reversing the declining trend of FAO's non-emergency field programme, as well as the many important normative activities which are also strongly supported by the membership.

3. Priority themes for FAO's work have been established in the World Food Summit Plan of Action and the Strategic Framework. Management concurs that greater advocacy of these themes should be made, as embodied in the FAO strategic objective A central place for food security as well as the cross-organizational strategy Communicating FAO's Messages. Priorities for policy work by the Organization will be based on its comparative strengths as identified by the evaluation. Management agrees with the suggested emphasis on policies at the sub-sector level (rather than comprehensive agricultural sector reviews) and on ensuring effective normative underpinning of policies.

4. Participatory policy formation processes, the technology-policy interface and integrating costs and benefits in sub-sector policy work will be enhanced through closer and deeper collaboration between all policy units at Headquarters and in decentralised offices. In the Regional and Sub-regional Offices, in particular, this will be achieved by a better integration of policy assistance branches and units with the technical department groups and multi-disciplinary teams. The technical expertise available in the different technical divisions will be strengthened in policy work, including in the areas of extension and research, while TCA policy officers will be given, where possible, a more technical orientation. The importance of cross-sectoral aspects of policy work, for instance environmental dimensions of agricultural policy formulation and implementation and gender and HIV/AIDS issues, should also be stressed.

5. The recommended focus on LIFDCs is very apt, notwithstanding absorptive capacity constraints that may arise. This focus will have to be properly balanced with the demands of middle-income countries for policy assistance in such areas as emerging development themes. It is also agreed that policy assistance to CIS and similar countries has to be expanded, by establishing relevant specialist capability working on these countries subject to the overall constraints of available resources. The absence of FAO Representatives in CIS countries has been mentioned in the evaluation. Although the 119th Council approved the proposal of the Director-General for the establishment of additional country offices under certain conditions, these countries have not generally come forward to participate in the scheme. Costs for liaison or representational tasks in such countries have therefore not been included in the PWB growth scenario. However, the Organization is reviewing its plan in this regard.

6. The evaluation underlines the critical role of FAO, within UN teams and at country level, in ensuring that agriculture and food security issues are fully reflected in macro-policies, including for poverty alleviation and sustainable rural livelihoods. This role is not always recognised by some of our partner institutions or by some countries themselves. A corporate paper aiming to refine the conceptual basis for FAO's work on linkages between poverty, food security, agriculture and rural development, is being finalised by the Economic and Social Department. Arrangements are also underway to enhance collaboration with other UN agencies and IFIs in UNDAF, CCA, CDF, HIPC and PRSP exercises. Recently, additional financial support has been provided to FAO Representatives through a special TCP facility, to help strengthen their prompt response to countries' needs and contribute to these processes.

7. As suggested by the evaluation, FAO's contribution to promoting consultative approaches for policy development and engaging civil society will be expanded further. This will be done by implementing more effectively FAO's policy and strategy for cooperation with NGOs and civil society organizations (including rural women's organizations and groups), which was set out in the Strategic Framework and developed thereafter.

8. It is agreed that, at present, policy expertise is spread throughout many units in FAO, and that there is insufficient coordination and exchange of information despite the existing interdisciplinary cooperation, both ad hoc and institutionalised. The recommendation is therefore endorsed to establish a policy Task Force and focal points to better define priorities; to strengthen the normative underpinning of policy work; to develop mechanisms for facilitating inter-unit collaboration; and to develop policy guidelines on how to effectively relate to and facilitate the policy process at country level. Indeed, the findings of the evaluation reinforce Management's belief that improvements in all these areas will increase the impact that FAO has made on both the process and implementation of policy in developing member countries. An inter-departmental coordination mechanism will be established to foster the interface between policy and technology as well as between policy and operational activities, with particular reference to rural development. Focal points for policy work in the Organization will, subject to the availability of resources as determined by the membership, be allocated additional resources to assure the policy interface between normative analytical work and advocacy/assistance work.

9. The need for FAO to improve "country intelligence" is already being acted upon by the TC Department, which has initiated the establishment of a web-based country information system for internal use, primarily for enhanced country focus and Field Programme development. Once operational, this system will be managed by the policy assistance branches and units in close collaboration with the FAO Representatives. It will not cover specialised domains of information for which databases are maintained by the technical departments.

10. Management also concurs that FAO's capacity building activities in policy formulation and analysis should be enhanced, particularly to strengthen the broad-based capabilities in agriculture ministries to participate as partners in macro-policy discussion. The focus and activities of TCA in capacity building are being reoriented, in order to reach higher-level decision-makers in government and have a better impact on both policy processes and content. This shift will be gradual, however, as it requires a strengthening of TCA's professional capability and resources to provide effective assistance to developing member countries in building up their capacities in policy development.

11. The evaluation states that in order to raise the productivity and effectiveness of the policy assistance branches and units, working procedures in the Regional Offices should be strengthened with a view to improving inter-disciplinary integration in their work. The analysis by the TC Department of the performance of these units and their impact on both Field Programme development and policy assistance has led to similar conclusions. Consequently, directives are being given to establish Country Task Forces in Regional Offices and to field better targeted multi-disciplinary programming missions, in close collaboration with Governments and FAO Representatives. These mechanisms will help to more clearly identify and develop regional priorities, as suggested in the evaluation, and to translate them into demand-driven services to Member Nations. Management accepts the concept of a common MTP programme entity, since it would provide added incentives to carry out policy work in an integrated and effective manner. The framework for such a common programme entity will be developed in consultation with the decentralised offices. An in-depth review is also being conducted of existing capabilities in the policy assistance branches and units.

12. The evaluation correctly stresses that the FAO Representative is a critical point of entry for government requests for policy assistance, and is essential in assisting countries in the policy process. Management concurs with this emphasis. The criteria used in the selection of FAORs have been defined by the Organization^*. The concerns expressed by the evaluation on the capacity of the FAO Representatives are therefore more a matter of appropriate briefing and training, and the suggestions and recommendations made in this regard have been noted. It should also be pointed out that the Organization is reviewing staff resources made available at the country level to assist with the follow-up to normative and operational activities, particularly project operations.

13. Information and direct technical assistance that are currently provided by the TCA branches and units to the FAO Representatives have to be reinforced. These decentralised offices will give timely support to the FAORs when they request policy advice.

14. The recommendations to improve the design of policy-oriented projects are well taken and will be applied to future interventions. Those on the shortcomings in the design of technical cooperation have also been noted. Among these, streamlining TCP arrangements will be done to the extent that the existing rules of TCP projects will be adhered to, and keeping in mind that these projects, like other FAO projects, are formulated and executed at the request of governments. To strengthen the flexibility of response to policy needs and improve budgetary reporting, administrative procedures will be streamlined and decentralised decision-making further enhanced.

15. The suggestion made in the evaluation that the ratio of resources to staff be strengthened through elimination of posts is accepted in principle and will be examined on a case by case basis. As mentioned above, an increase in non-staff resources is proposed in the SPWB 2002-03. Finally, it is fully agreed that donor funds have to be mobilised to establish policy cooperation funds, both globally (for policy interface between normative and technical divisions in FAO) and regionally (possibly modelled on the Horn of Africa initiative).

I. Introduction

CONTEXT

1. The present evaluation forms part of the regular schedule of evaluations carried out on subjects recommended by the FAO Programme Committee and reported to the FAO Conference in the Programme Evaluation Report. In line with requests by FAO Governing Bodies to undertake evaluation on FAO objectives as identified in the Strategic Framework for FAO 2000-2015, this evaluation addresses Corporate Strategy B - Promoting, developing and reinforcing policy and regulatory frameworks for food, agriculture, fisheries and forestry (Objective B.2 National policies, legal instruments and supporting mechanisms that respond to domestic requirements and are consistent with the international policy and regulatory framework). It may be noted that FAO-TCP work to develop national legislation was evaluated in 1999¹ and reported to the Programme Committee in 2000. The present evaluation is thus restricted to policy work.

2. As the evaluation addressed only FAO's work in development of national policy, it did not aim to draw conclusions on the relative priority which should be accorded to policy work in comparison with other FAO strategic objectives. The evaluation sets out to examine FAO's past performance and capacity in meeting policy needs and countries' and development partners' expectations from FAO. In making this assessment, the evaluation adopts a forward looking stance in examining past performance in order to derive practical recommendations for future FAO work.

THE IMPORTANCE OF COOPERATION WITH COUNTRIES IN POLICY DEVELOPMENT

3. The place of policy on the development agenda has been steadily increasing in importance. First of all, it was appreciated that distortions and lack of clarity of purpose in macro-policy instruments had contributed to diminished growth, inequitable income distribution, unsustainable use of natural resources and high levels of indebtedness linked to unproductive investments. More recently, the pace of globalisation has meant that if countries are to reap its benefits, rather than suffer from its disadvantages, they must restructure their economies to maximise trade opportunities and introduce national measures in line with their international commitments. Governments have thus been withdrawing from direct involvement in production and commercial activity, as well as the provision of certain services.

4. With these changes, the emphasis has shifted to more indirect policy instruments to provide the enabling framework for development. At the same time, countries face critical old and many new challenges, including hastening the generation and uptake of improved technology and management systems; making the most of debt relief for poverty reduction; living with AIDS; and living with climate change. The role of agriculture, fisheries and forestry and of rural people has remained critical in ensuring:

national and household food security;
contributing to poverty alleviation, livelihoods and economic development in rural and to some extent urban areas;
sustaining the natural resource base; and increasingly
in middle-income countries, providing leisure and environmental benefits for urbanising populations.

5. To achieve these benefits, action is required within FAO's mandate on many facets of policy, including at the:

macro-sector interface - implications for agriculture, food security and rural livelihoods of policies in such areas as trade, taxes, money supply, property rights, water, employment and international investment;
sector level - fisheries and forest management, promotion of private sector services in agriculture, etc.; and
specific subject matter areas - irrigation; research, innovation and communication; degraded land management; pesticides; rural finance; genetic resources; food safety nets; emergency preparedness, etc.

FAO'S ROLE

6. FAO provides assistance to countries for development of national policies in all aspects of agriculture, livestock, forestry, fisheries and related natural resource management as well as in food and nutrition policy. This is done through:

provision of a forum for norm setting and international agreement which is then reflected in countries' national policy. Examples of this extend from the commitments of the World Food Summit, to the Code of Conduct on Responsible Fisheries, the negotiations with respect to plant genetic resources and the establishment of food quality and safety standards (Codex);
provision of international databases drawn on when establishing policies;
awareness raising on policy issues and experiences through publications, meetings, training, etc.; and
at the national level, consultancies to assist directly in policy development and in capacity building.

7. The evaluation concentrates on points c) and d). Work is thus included which provides support in:

translating policy commitments and norms accepted at international level into national policy;
identification and analysis of policy issues and options;
policy formulation;
national policy dialogue; and
development of capacity for policy development.

8. What FAO actually does in policy work to directly support countries, in approximate order of importance in terms of resource commitment, consists of:

advisory work, financed primarily from extra-budgetary resources but supplemented by FAO Regular Programme inputs;
ongoing dialogue with governments supported by the FAO Representatives;
development of materials for training and awareness-raising publications on policy issues;
training; and
regional meetings to raise policy issues.

SCOPE AND METHODOLOGY

9. Scope: In order to arrive at a manageable scope for the purposes of the evaluation, policy was considered at the level of central government, government departments and intergovernmental bodies. The policy work included refers to the definition and articulation of goals, priorities, principles and strategies by government but not the detailed development of the actions and instruments for execution of those policies (i.e. resource allocation, legislation, programmes of service and infrastructure provision and projects). Planning was also excluded, as in general it refers to policy execution. Thus, although assistance in such areas as legislation and planning often have policy implications, for the purpose of this study the activity was only included if "policy" was a significant proportion of the activity.

10. The adoption of this scope of work in no way infers that policy instruments have less importance than the policy decision itself. Indeed one of the main criteria used in the evaluation for assessment of the quality of policy work was whether the instruments for implementation of the policy were defined and were realistic. The eventual effects and impact were judged largely in terms of the extent to which policy was translated into policy instruments. Similarly, the essential importance is recognised of the statistics, field information and other data, which must be assembled and maintained in information systems for policy making.

11. Work falling within the above scope by all FAO technical and development units was included. Particular attention was given to the Technical Cooperation Programme (FAO-TCP) funded from FAO's Regular Programme budget ². Policy was thus not restricted to economic policy and encompassed the:

interface between the agricultural, fisheries and forestry sectors and macro-policy;
overall sector or sub-sectors, such as agriculture or aquaculture;
cross-cutting issues, such as the environment;
more micro issues, e.g. access rights to resources (water/land); and
positions adopted in international negotiation.

12. Methodology: The evaluation was extensive in its coverage, gaining lessons through wide confirmation, rather than a very detailed case study approach on a few interventions. It involved the following actions:

circulation of an internal questionnaire to FAO units to provide an initial database on policy activities. This did not provide a reliable source of information on all policy work but provided a basis for steps b) and d) below;
a questionnaire survey to government departments in the agricultural, fisheries and forestry sectors in developing countries in which FAORs are posted on their needs for, and satisfaction with, FAO policy work. This questionnaire obtained a 53% response, giving an indication that the departments with which FAO deals attach some priority to the Organization's involvement in policy work ³;
discussion in FAO with the main units involved in policy work, including Regional and Sub-regional Offices and FAORs;
field visits ⁴ by two-person missions (one FAO Evaluation Service, one senior independent consultant) to 21 countries ⁵ covering each of the developing regions, including Europe. During country visits, 66 individual FAO policy interventions/projects were evaluated and discussions were held with the government, civil society and development partners, including the UN system, IFIs and bilateral agencies;
visits and email discussions at Headquarters, and in the decentralized offices and with development partners and comparators including the World Bank, IFAD, IFPRI and UN specialised agencies;
an email questionnaire circulated widely in FAO on some initial conclusions and recommendations followed by an internal seminar; and
an independent external peer review workshop of the draft evaluation report.

13. Both questionnaires to countries and missions were required to provide scores for relevance and to score the performance of projects and other interventions. This was supported by qualitative analysis. There is no basis in this type of evaluation for quantitative analysis of impact (IERR) but impact was scored and examined qualitatively. The main underlying criteria for evaluation were:

relevance to country needs;
conformity to FAO priorities;
efficiency in carrying out work;
technical quality of work;
quality of process; and
effects and impact of the work in contributing to the development of viable policies at the national level.

14. It was important in assessing the adequacy of FAO's work to have a basis for comparison and throughout the evaluation an effort was made to obtain an overview of the policy work undertaken by other agencies. Independent consultants had worked for and had a knowledge of several organizations in addition to FAO. Through questionnaires, countries were asked to compare FAO with other partners. In this context, it may be noted that policy work is more identifiable in FAO than in the other UN specialised agencies consulted, although all of them are increasing their emphasis in this area.

15. The methodology of the study meant that it concentrated on FAO's work in countries where there had been a relatively high policy input. Thus, the profile of FAO on policy work is lower in many of the 80 percent of developing countries not visited. No missions visited small island states where on occasion a single intervention can have considerable impact ⁶. Similarly at FAO Headquarters and in the Regional Offices, some units have developed a greater policy orientation in their work than others. This means that both inadequacies and successes are attributed in certain disciplines and by implication to certain units, while others, which could be equally important for discussions of policy, such as those dealing with crop production, land management, and research and extension, have not received the same level of attention.

_______________________

* See in particular Manual Section 118 and, more recently, background documents to the 119^th Session of the Council.

1 Programme Committee - PC 83/4(b) Synthesis of Recent Field Project Evaluations.

2 The annual thematic evaluation of TCP projects was combined in the current evaluation.

3 Response rates were: Africa 50%, Asia and Pacific 65%; Latin America 52%, Near East 30%.

4 Countries were selected for visits on considerations of regional balance and the number of FAO policy interventions, especially FAO-TCP policy interventions. In selecting countries for visits, a secondary criteria was diversity in subject matter and type of policy intervention. In the countries visited, all FAO policy interventions since 1994 were normally covered.

5 Africa: Malawi, Mali, Mozambique, Senegal, Tanzania (mainland and Zanzibar); Asia: Bangladesh, Cambodia, Laos, Thailand; Europe: Estonia, Lithuania, Moldova, Slovak Republic, Turkey; Latin America: Brazil, Mexico, Peru; Near East: Egypt, Morocco, Tunisia, Yemen.

6 Account was taken of policy projects reviewed in the Caribbean during the 1999 evaluation of legislation work.

PROGRAMME COMMITTEE

Eighty-fifth Session

Rome, 7-11 May 2001

Evaluation of FAO's Policy Assistance (Cooperation with Member Countries in the Development of National Policies (1994-99) with particular attention to FAO-TCP)

Table of Contents

Executive Summary