Hundred and Twenty-seventh Session
Rome, 22-27 November 2004
Director-General’s Report on the Proposal to carry out a Comprehensive and Independent External Evaluation of FAO
ANNEXE I - A Comprehensive and Independent External Evaluation of FAO
ANNEXE II - Inventory of Reviews with External Inputs since January 2002 of Potential Relevance to an Evaluation of FAO
1. On 21 October 2004 two Members1 of the Council submitted a request to the Director-General calling for an additional item entitled “Comprehensive External Evaluation” to be added to the Provisional Agenda of the 127th Session of the Council. Subsequently, one of these Members wrote again on behalf of the North American Group providing a background paper and Draft Resolution in support of the requested Agenda Item. These are provided to Members of the Council in CL 127/LIM/4.
2. Under the provisions of Financial Regulation XIII:
- 13.1 Before taking any decision involving expenditures, the Council or any commission or committee appointed by the Council or the Conference shall have before it a report from the Director-General on the administrative and financial implications of the proposals.
3. This document responds to that requirement.
4. The Director-General wishes to put on record his preparedness to support the proposal for a comprehensive external evaluation and, should the Council decide to proceed with it, he also wishes to assure the Council that the Secretariat will be fully responsive to the needs of the evaluators in their work and in providing support to the work of the Members. He makes specific suggestions in this regard in this report.
5. He particularly welcomes the reaffirmation in the Draft Resolution of FAO’s mandate as expressed in its Constitution, and of the objectives established in the 2000-2015 Strategic Framework and agrees that the evaluation could make a significant contribution to the forthcoming revision of the Strategic Framework.
6. He also appreciates the recognition of important international meetings, in particular, the World Food Summit and its follow-up. Further recognition of the need for linkages between the various outcomes is also important, as is the need for resources to support the implementation of their respective plans of action. The Director-General considers that some reflection of this important element in any decision of the Council might be appropriate.
7. The background document provides a clear indication of the intended scope of the evaluation. The chapeau in paragraph 1 envisages a comprehensive evaluation covering:
8. This is encompassed in the first of the objectives which follows immediately from the chapeau in sub-paragraph (i) which covers an “assessment of FAO’s strengths and weaknesses”, a planning term which usually refers to the outcome of an internal analysis: “that is, those aspects of an organization that help or hinder the accomplishment of is mission and the fulfilment of its mandate”2. This is entirely consistent with the chapeau paragraph.
9. The proposal envisages a Committee of the Council, which would presumably be established under the provisions of Article VI 2. of the Constitution which states that the Council may establish “committees and working parties to study and report on matters pertaining to the purpose of the Organization and consisting either of selected Members or ...” rather than General Rule XXV 10 which states that “The Council may also establish ad hoc committees consisting of a limited number of Council Members, to meet between Council sessions for the purpose of examining, and reporting on, such questions as the Council may have referred to them.”
10. General Rule XXV 6. states that “The travelling expenses of not more than one member of the delegation of each Member Nation on the Council, properly incurred in travelling, by the most direct route, from the member's capital city or duty station, whichever is less, to the site of the Council's session and return to his or her capital city or duty station, shall be borne by the Organization.” Similar provisions exist for the Programme Committee3 and for the Finance Committee4. Given these precedents, the Council may wish to clarify:
11. The estimates below assume coverage at the level of the Committees from extra-budgetary resources.
12. The Council may wish to consider giving guidance that the exercise should be carried out in such a manner as to ensure that the full diversity of the Membership of the Organization is reflected in the modalities for the evaluation.
13. In particular, the provision requiring that the team include professional evaluators and consultants from different regions needs to be strengthened if the results are to have the support of the entire membership. A recent experience elsewhere in the system, which had a similar provision, resulted in only two junior team members from certain regions in a team dominated by other regions. The Council may wish to insist that regional representation within the team is substantive.
14. The background paper and draft resolution do not explicitly define the appropriate qualifications of the experts needed for the evaluation. While the Director-General recognizes the usefulness of evaluation experience, he is convinced that this is not enough in itself. FAO is a complex technical Organization and its effectiveness, performance and impact can only be realistically assessed by practitioners in its various sectors. The Council, if it agrees, may wish to emphasize that team members should be competent, internationally recognized experts with a sound knowledge of the relevant sectors of FAO’s mandate.
15. The Committee will need some form of secretariat to prepare for and support its meetings. In addition, it presumably will wish to receive reports on the progress of the evaluation vis-ŕ-vis the timetable and the Terms of Reference as the work of the team will need to be subject to day-to-day attention of a type which cannot be provided by a committee.
16. The Council should decide if the Director-General should assign a small team from the Secretariat to act as Secretariat to the Committee. In such case, the costs of the Committee Secretariat should be covered by additional extra-budgetary resources to meet the needs of the Committee.
17. Attention is drawn to the deadlines associated with the submission of a translated document to the Council in November 2006. Working back from the Council start date of 20 November, the document will need to be despatched by 9 October. Given the need for document production and translation into five languages, it should be delivered for processing no later than 1 September 2006.
18. In addition, sufficient time will be needed to develop the Secretariat’s response which will also need to be translated. Precisely how much time will be necessary is difficult to say but if it is a fully comprehensive report, it may require several weeks (at least a month) to develop well thought-out responses for each recommendation.
19. While the scope is clearly defined as explained under “Purpose of the Evaluation” described above, there are some scope issues on which the Council may wish to give guidance.
20. The Council may wish to further clarify sub-paragraph (v) which is to “Provide further input for the definition of FAO’s future resource requirements”. It is hoped that this would be measured against the needs of Members as expressed in the Strategic Framework and in the various editions of the Medium Term Plan and not be limited to the adequacy or otherwise of resources for the current much reduced Programme of Work.
21. The background document is silent on the relevance of the very recent “Independent Evaluation of FAO’s Decentralization”5 and whether the External Evaluation would build on that work. The Director-General notes that the Programme Committee “... welcomed the depth and independence of the evaluation of FAO’s decentralization. The report was comprehensive in identifying and addressing the very complex and inter-linked series of issues for the Organization in strengthening its institutional performance. It contained a wealth of analysis, suggestions and ideas, in addition to the formal recommendations. The Committee, thus, emphasized its appreciation of the evaluation team’s work.”6
22. Currently the Secretariat is preparing a comprehensive response and action plan for the Programme Committee including:
23. The Programme Committee has welcomed this approach but it could be a largely wasted effort on the part of the Secretariat if the “Independent Evaluation of FAO’s Decentralization” were to be superseded by a new external evaluation.
24. The Director-General believes it is important that this issue be clarified. The Council may wish to provide guidance as to whether it intends that the evaluation should take into consideration recent evaluation and oversight reports submitted to the Programme and Finance Committees8 including, in particular, the report of the “Independent Evaluation of FAO’s Decentralization”.
25. There are both procedural and monetary aspects to the financial implications.
26. The seventh paragraph of the Preamble of the draft Council Resolution: "Further decides that extra-budgetary resources will finance this evaluation”.
27. Paragraph 4(i) of the proposed document provides that “the Committee of the Council will decide how contributions will be received, managed and disbursed”. In addition, paragraph (iii) of the Draft Council Resolution states that the Committee will “agree on modalities for receiving contributions ... and for disbursing such contributions”.
28. On the assumption that there is agreement on the fact that the required voluntary contributions will be deposited with FAO, paragraph 4(i)of the proposed document and operative paragraph iii) of the draft Council Resolution do not take into account that there are already binding provisions regarding the receipt, management and disbursement of voluntary contributions by FAO. Following the FAO Rules and Regulations, a trust or special fund should be established to receive such contributions. In particular, Financial Regulation 6.7 provides that “Trust and Special Funds and voluntary contributions shall be administered in accordance with the Financial Regulations of the Organization, unless otherwise provided for by the Conference.”
29. The proposed Committee may establish the purposes of the Trust Fund and determine the activities to be funded under the Trust Fund. However, the proposed text is too general and does not indicate that the reception, management and disbursement of funds should be carried out in accordance with FAO Financial Regulations. The Director-General recommends that this be stated clearly.
30. Attention is drawn to the requirement in the Finance Manual that “In principle, the Organization makes no financial commitments and disburses no funds under a Trust Fund until funds have been received9”. In the interests of ensuring that the proposed timetable is adhered to, it is stressed that funds should be received promptly after the Council makes its decision so that administrative arrangements can be made for the first meeting of the Committee.
31. The Draft Resolution requires that the Evaluation be funded from extra-budgetary resources which in theory might mean that there are no financial implications for the Regular Programme. However, the Director-General is concerned that:
32. In particular, no detail is provided of how the estimate of US$ 2 to US$ 3 million was developed or of the proposed composition of the experts, evaluators and consultants.
33. For this reason, the Director-General provides an assessment of the total financial implications10 based on the Organization’s experience of evaluations. These assumptions are made solely to prepare a reasonable estimate and not to prejudge any decision the Council may wish to make.
34. These consist of the cost of Committee meetings and would be close to US$ 25,000 each assuming one day sessions in all five languages or about US$ 38,000 for a two day session depending also on the length of the in-session documentation. The total estimate below assumes six one-day meetings.
35. On the assumption that the council approves travel reimbursement, it is estimated that on average five Members would travel from their capitals (almost US$ 30,000 per meeting) – this figure may, of course, vary.
36. In addition, it is expected that the final report will be substantial (estimated at 70,000 words) and that this, along with a detailed management response, will cost just over US$ 190,000 for translation and production in all languages.
37. As indicated in paragraph 15-16 above the Committee will need a Secretariat for the reasons stated. The cost of US$ 486,000 covers a D-1 and one professional post plus secretarial support.
38. The background paper includes the entirely appropriate requirement that the Secretariat should provide support for the evaluation as requested by the Council and its organs including such background papers that the Committee or its experts may request. The demand for support from the Secretariat could be significant as was the experience of the previous Review in 1988-89. However, this will reduce the capacity of the Organization to deliver the original Programme of Work approved by the Conference unless some resources are made available from the proposed Trust Fund to allow the replacement of staff where necessary.
39. It is very difficult to estimate the extent of support and therefore no amount is included in the estimate. However, it is hoped that the Council will agree that the Trust Fund should cover the cost of any incremental direct costs arising from the provision of such support.
40. This is the larger part of the total team foreseen in these estimates and it consists of a team leader, several senior experts and experts covering the various disciplines and sectors of the Organization’s programmes. This quite large group will manage the evaluation, evaluate the work of the technical departments and prepare the final report. The amount estimated including the cost of travel and secretariat support, is approximately US$ 2.4 million.
41. The potential scope and range of management reviews of areas such as finance, HR management, ITC, etc. is enormous. However, a selection of the key reviews has been calculated at average rates for high level consultants to arrive at a total cost of US$ 1,066,950. It is noted that this may, in some cases be duplicative of relatively recent studies as mentioned in Annex II.
42. The estimate includes the cost of sending three person teams on eight missions to the field covering all Regional Offices, most Sub-regional Offices, most of the Liaison Offices and about 25 countries. This would cost about US$ 688,000 although it noted that it may be largely duplicative of the recent Independent Evaluation of FAO’s Decentralization.
43. In summary, the Director-General estimates that the total cost of such a comprehensive evaluation is likely to cost in the region of US$ 5 million as recapitulated in the table below.
|Summary of Costs||US$ 000|
|Committee of the Council – direct costs||148,158|
|Committee of the Council – travel of Members||176,100|
|Final Evaluation Report and Management Response||192,766|
|Secretariat for the Committee||486,450|
|HQ Programmes and Overall Management of the Team||2,403,657|
|Key Reviews of Non-technical Departments||1,066,950|
44. The Director-General notes that if such estimates correctly reflect the scope of the Evaluation, then there are two potential consequences as it is not viable to envisage a contribution from the Regular Programme Budget which has already been subject to drastic cuts:
45. The following table summarizes the areas on which the Council may wish to take action.
Area of Action
|4||Decide whether to proceed with the Evaluation of FAO|
|6||Recognize the linkage between internationally agreed areas of action and resources made available|
|9||Establish a Committee of the Council and its Membership|
|10||Determine travel entitlements of Committee Members|
|13||Define extent of regional representation|
|14||Provide guidance on expert qualifications|
|15||Decide whether a Secretariat is required for the Committee|
|17/18||Establish a timetable|
|20||Clarify Team’s mandate to consider resources issues|
|Clarify the extent to which the Team may be required to take account of pre-existing oversight reports in its deliberations|
|26-29||Establish a Trust Fund under FR 6.7 to receive voluntary contributions and from which disbursements can be made|
|31-42||Require that all direct costs incurred on the Evaluation be funded from the Trust Fund including: |
|44||Provide guidance on action to be taken in the event of a shortfall in voluntary contributions and, if necessary, decide on priorities for inclusion/ exclusion from scope|
1 United States of America and Canada
2 Bryson, J.M., Strategic Planning for Public and Nonprofit Organizations, Jossey-Bass, San Francisco, p90
3 G.R. XXVI 9
4 G.R. XXVII 9
5 PC 92/6 a)
6 C 127/12, paragraph 48
7 Idem, paragraph 57
8 See Annex II for an Inventory of Reviews with External Input since January 2002
9 MS 250, paragraph 250.113
10 See Annex I “Summary of Costs” and the supporting worksheets.