Annex 1: List of Participants
Annex 2: Agenda
Annex 3: List of Documents
Annex 4: Terms of Reference for External Programme and Management Reviews of CGIAR Centres and Guidelines for External Programme and Management Reviews of CGIAR Centres
MEMBERS OF THE TECHNICAL ADVISORY COMMITTEE
Dr. Donald L. Winkelmann
Chair
Technical Advisory Committee/CGIAR
355 East Palace Avenue
Santa Fe, New Mexico 87501, USA
Dr. André Berkaloff
Université Paris XI
Institut de Microbiologie - Bâtiment 409
91405 Orsay Cedex, France
Dr. Jaques Faye
c/o CIRAD
37, Avenue Jean XXIII
BP 6189 - Dakar Etoile, Senegal
Dr. Richard Harwood
Crop & Sciences Dept.
260 Crop & Soil Sciences Building
Michigan State University
East Lansing
Michigan 48824, USA
Dr. Ted Henzell
182 Dewar Terrace
Corinda, QLD 4075
Australia
Dr. Bram Huisman
Director
Wageningen Institute of Animal Sciences
Wageningen Agricultural University
Marijkeweg 40
6709 PG Wageningen, The Netherlands
Dr. Keiji Kainuma
Vice President
Bio-Oriented Technology Research Advancement Institution (BRAIN)
3-18-19 Toranomon Minato-Ku
Tokyo 105, Japan
Dr. Justin Yifu Lin
China Center for Economic Research
Peking University
Beijing 100871, China
Dr. Magdy A. Madkour
Director of Agricultural Genetic Engineering Research Institute (AGERI)
Agricultural Research Centre
9 Gamaa Street
Giza 12619, Egypt
Dr. Richard Musangi
Villa Maria Enterprises Ltd.
P.O. Box 14330
Nakuru, Kenya
Dr. Cyrus Ndiritu
Director
Kenya Agricultural Research Institute
P.O. Box 57811
Nairobi, Kenya
Dr. C.H. Hanumantha Rao
240-B. Road No. 18
Jubilee Hills
Hyderabad 500-033, India
Sir Ralph Riley
16 Gog Magog Way
Stapleford
Cambridge CB2 5BQ, England
Dr. Ammar Siamwalla
The Thailand Development Research Institute Foundation
565 Soi Ramkhamhaeng 39 (Thepleela 1)
Ramkhamhaeng Road, Wangthonglang
Bangkapi, Bangkok 10310, Thailand
Dr. Peter Tigerstedt
University of Helsinki
Department of Plant Biology
Plant and Tree Breeding
Viikkid 00710 Helsinki, Finland
Dr. Lucia de Vaccaro
Facultad de Agronomia
Institute de Produccion Animal (IPA)
Universidad Central de Venezuela
Apartado 4579
Maracay, Aragua, Venezuela
Dr. Maria de O. Zimmermann
Avenida 85 No. 300, Ap. 501 A
Setor Marista
74166-900 Goiânia, GO., Brazil
COSPONSORS OF THE CGIAR
UNDP
Mr. Philip Reynolds
FAO
Mr. Henri Carsalade
Assistant Director-General
Via delle Terme di Caracalla
00100 Rome, Italy
Mr. Gora Beye
Chief
SDRR
INTERNATIONAL AGRICULTURAL RESEARCH CENTRES
Centro Internacional de Agricultura Tropical (CIAT)
Dr. Grant Scobie
Director General
Apartado Aereo 6713
Cali, Colombia
Dr. Juan Antonio Garafulic
Financial Controller
Center for International Forestry Research (CIFOR)
Dr. Jeffrey Sayer
Director General
Jalan Gunung Batu 5
Bogor 16001, Indonesia
Centro Internacional de Majoramiento de Maiz y Trigo (CIMMYT)
Dr. Timothy Reeves
Director General
Lisboa 27
Apartado Postal 6-641
06600 Mexico, D.F. Mexico
Dr. Tiffin Harris
Director, External Relations
Centro Internacional de la Papa (CIP)
Dr. Hubert G. Zandstra
Director General and Chair, CDC
Apartado 1558
Lima 100, Peru
Dr. Ed Sulzberger
Senior Adviser
Dr. Martha ter Kuile
Chair, Board of Trustees
RR3 Ashton
Ontario, Canada KOA IBO
Dr. Tom Walker
Head, Social Sciences
International Center for Agricultural Research in the Dry Areas (ICARDA)
Dr. Abdel El-Beltagy
Director General
P.O. Box 5466
Aleppo, Syrian Arab Republic
Dr. Mohan Chandra Saxena
Assistant Director General
Dr. John E. Noisette
Director of Finance/ADM
International Center for Living Aquatic Resources Management (ICLARM)
Dr. Meryl J. Williams
Director General
MC P.O. Box 2631
Makati Central Post Office
0718 Makati, Metro Manila, Philippines
Dr. Peter Gardiner
Deputy Director General
International Center for Research in Agroforestry (ICRAF)
Dr. Pedro Sanchez
Director General
United Nations Avenue
P.O. Box 30677
Nairobi, Kenya
Dr. Bruce Scott
Deputy Director General
Ms. Madeleine Shearer
Projects Officer
International Crops Research Institute for the Semi-Arid Tropics (ICRISAT)
Dr. James G. Ryan
Director General
Patancheru 502 324
Andhra Pradesh India
Dr. Kwame Akuffo-Akoto
Director of Finance
Dr. R.S. Paroda
Chair, Board of Trustees
Director General
Indian Council of Agricultural Research
New Delhi, India
Mr. Mark Winslow
Special Assistant to Deputy Director-General
International Food Policy Research Institute (IFPRI)
Dr. Per Pinstrup-Andersen
Director General
1200 17th Street N.W.
Washington, D.C. 20036 - 3006 U.S.A.
International Irrigation Management Institute (IIMI)
Dr. David Seckler
Director General
P.O. Box 2075
Colombo, Sri Lanka
International Institute of Tropical Agriculture (IITA)
Dr. Lukas Brader
Director General
PMB 5320
Ibadan, Nigeria
International Livestock Research Institute (ILRI)
Dr. Hank Fitzhugh
Director General
P.O. Box 30709
Nairobi, Kenya
Dr. Ralph von Kaufmann
Director for External Relations
Dr. Jean Hanson
Programme Leader, Biodiversity
International Plant Genetic Resources Institute (IPGRI)
Dr. Geoffrey C. Hawtin
Director General
Via delle Sette Chiese 142
00145 Rome, Italy
Dr. Emile Prison
Director, INIBAP
Parc Scientifique Agropolis
34397 Montpellier, France
Dr. Dick van Sloten
Assistant Director General
Dr. Masaru Iwanaga
Deputy Director General - Programme
Dr. Thomas Gass
Regional Director, Europe
Dr. Wanda Collins
Chair, Board of Trustees and Chair CBC
World Bank
ESDAR, S-7055
1818 H Street, N.W.
Washington D.C. 20433, USA
Mr. Koen Geerts
Director - Finance and Administration
Dr. Jan Engels
Director - Genetic Resources Science and Technology
Dr. Lynsey Withers
Director, Documentation, Information & Training Group
International Rice Research Institute (IRRI)
Dr. George Rothschild
Director General
P.O. Box 93375
Manila, The Philippines
Dr. Ken Fischer
Deputy Director General for Research
Dr. Ed Sayegh
Director of Finance
International Service for National Agricultural research (ISNAR)
Dr. Stem Bie
Director General
P.O. Box 93375
2509 AJ The Hague, The Netherlands
Dr. Howard Elliott
Deputy Director General
Dr. Amir Muhammed
Chair, ISNAR Board of Trustees
House 3, Street 35, F-8/1
Islamabad, Pakistan
West Africa Rice Development Association (WARDA)
Dr. Kanayo F. Nwanze
Director General
01 B.P. 2551
Bouaké 01, Côte d'Ivoire
Dr. Chris Perry
Deputy Director General
Dr. Just Faaland
Chair, Board of Trustees
Chr. Michelsen Institute
Development Studies and Human Rights
Fantoftvegen 38
N-5036 Fantoft, Norway
Dr. Peter Matlon
Director of Research
CGIAR MEMBERS
Belgium
Mr. Luc Sas
CGIAR Officer
Bureau Agricultural Organizations
Belgian Administration for Development Cooperation (BADC)
Troonstraat, 4
B1000 Brussels, Belgium
France
Dr. Gilles Saint-Martin
Executive Secretary, CRAI
Ministry of Education, Higher Education and Research/DSPT9
173 Boulevard Saint-Germain
75006 Paris, France
Germany
Dr. Ekkehard Kuerschner
Coordinator,
International Agricultural Research
German Council for Tropical and Subtropical Agricultural Research (ATSAF)
Ellerstr. 50
D-53119 Bonn, Germany
Ms Beate Weiskopf
Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ)
Division 423
Postfach 5180
D-65726 Eschborn, Germany
Italy
Dr. Abdelmajid Slama
Director
IFAD
Via del Serafico, 107
00142 Rome, Italy
Dr. Shantanu Mathur
Technical Adviser
IFAD
Via del Serafico, 107
00142 Rome, Italy
Mr. David Kingsbury
Economist
IFAD
Via del Serafico, 107
00142 Rome, Italy
The Netherlands
Mr. Frans Neuman
CGIAR Liaison Office
International Agriculture Centre (IAC)
Lawickse Allee 11, P.O. Box 88
6700 AB Wageningen, The Netherlands
USA
Dr. Dana G. Dalrymple
Research Adviser
US Agency for International
Development (USAID)
Washington DC 20523-1806, USA
OTHER PARTICIPANTS
Mr. A. Aboul-Naga
Deputy Permanent Representative of the Arab Republic of Egypt to FAO
Embassy of Egypt
Via Salaria, 267 Rome, Italy
Dr. Ronnie Coffman
Chair, ICRISAT External Review Panel
Associate Dean for Research
Cornell University
245 Roberts Hall
Ithaca, NY 14853-1902, USA
Dr. Iain MacGillivray
Senior Programme Manager
Food Aid Centre
Canadian International Development
Agency (CIDA)
Place du Centre
200 Promenade du Portage
Hull, Quebec KIA 064, Canada
Prof. Dr. Uwe-Jens Nagel
Agricultural Extension and Communication Sciences
Humboldt University of Berlin
Lenzeallee 75
D-14195 Berlin (Dahlem), Germany
Dr. Samuel Paul
Chair, ISNAR External Review Panel
Chairman, Public Affairs Centre
27, SB1 Colony, Block 3, Koramangala
Bangalore 560 034, India
Dr. Enrico Porceddu
DABAC,
Professor
University of Tuscia
Via S.C. del Lellis
01100 Viterbo, Italy
Dr. Calvin O. Qualset
Chair, IPGRI External Review Panel
Director, Genetic Resources Conservation Program
Hopkins Road at Bee Biology Road
University of California
Davis, California 95616, USA
CGIAR SECRETARIAT
Mr. Alexander von der Osten
Executive Secretary
The World Bank
1818 H. Street, N.W.
Washington, DC 20433, USA
Mr. Ravi Tadvalkar
Principal Finance Officer
Mr. Gordon MacNeil
Senior Finance Officer
Mr. Selçuk Özgediz
Management Adviser
TAC SECRETARIAT
Dr. Shellemiah O. Keya
Executive Secretary, TAC
FAO
Via delle Terme di Caracalla
00100 Rome, Italy
Dr. Guido Gryseels
Deputy Executive Secretary, TAC
Dr. Amir Kassam
Senior Agricultural Research Officer
Ms. Jane Garrioch
Secretary
Ms. Irmi Braun
Programme Clerk
1. Opening Session2. ICRISAT External Programme and Management Review
3. ISNAR External Programme and Management Review
4. IPGRI External Programme and Management Review
5. Centres' Medium-Term Plans 1998-2000
6. CGIAR Priorities and Strategies
7. CGIAR Activity Classification
8. CGIAR Research Priorities for Marginal Lands
9. Future Reviews
10. Future Meetings
11. Other Business
|
Item |
Document |
Number/Source |
|
1 |
Draft Report of the 71st Meeting of the Technical Advisory Committee |
SDR/TAC:IAR/97/2 |
|
2 |
Report of the Fourth External Programme and Management Review of ICRISAT |
SDR/TAC:IAR/96/22 |
|
ICRISAT's Response to the Report of the Fourth External Programme and Management Review |
ICRISAT |
|
|
3 |
Report of the Third External Programme and Management Review of ISNAR |
SDR/TAC:IAR/96/23 |
|
ISNAR's Response to the Report of the Third External Programme and Management Review |
ISNAR |
|
|
4 |
Report of the Fourth External Programme and Management Review of IPGRI |
SDR/TAC:IAR/97/1 |
|
IPGRI's Response to the Report of the Fourth External Programme and Management Review |
IPGRI |
|
|
5 |
Centres' Medium-Term Plans 1998-2000 |
CGIAR Centres |
|
7 |
Report on the TAC/DSE Workshop Efficient Research Planning for the CGIAR Logical Frameworks and the Classification of Activities |
SDR/TAC:IAR/97/5 |
|
8 |
Report of the Study on CGIAR Research Priorities for Marginal Lands |
SDR/TAC:IAR/96/18.1 |
|
9 |
Future Reviews - Progress Report |
SDR/TAC:IAR/97/3 |
|
Terms of Reference and Guidelines for External Programme and Management Reviews of CGIAR Centres |
SDR/TAC:IAR/97/4 |
April 1997
TERMS OF REFERENCE FOR EXTERNAL PROGRAMME AND MANAGEMENT REVIEWS OF CGIAR CENTRES
BACKGROUND
Context
1. The Consultative Group on International Agricultural Research (CGIAR) is an informal association of over 50 members that supports a network of 16 international research centres in agriculture, forestry and fisheries. The CGIAR aims, through its support to the Centres, to contribute to promoting sustainable agriculture for food security in developing countries. Because the Centres constitute the core of the CGIAR, the effectiveness of each Centre is crucial to the continued success of the CGIAR (as a System).
2. Each Centre is an autonomous institution operating within the mandate assigned to it by the CGIAR, and is governed by a legally constituted Board that has full fiduciary responsibility for managing the Centre. To ensure accountability in an essentially decentralized system, each Centre is expected to be responsive to the CGIAR, which provides financial support for its work.
3. The CGIAR has established a tradition of External Programme and Management Reviews (EPMRs) to provide a mechanism of transparency and accountability to the Members and other stakeholders of the CGIAR System. EPMRs are the joint responsibility of TAC and the CGIAR Secretariat, and are conducted for each Centre approximately every five years. As each Centre is autonomous, EPMRs provide a measure of central oversight and serve as an essential component of the CGIAR's accountability system.
Integrated System of Reviews of Each Centre
4. Besides the EPMRs, Centre Commissioned External Reviews (CCERs) are undertaken at each Centre. These CCERs are commissioned by the Centre Boards to periodically assess the quality and effectiveness of particular aspects of a Centre's work. The terms of reference (TORs) for each CCER are determined by the Centre, based on broad principles endorsed by the CGIAR at ICW95 (ref. document entitled Improving the Quality and Consistency of CGIAR's External Centre Reviews, dated October 24, 1995).
5. EPMRs complement the CCERs by providing a CGIAR-commissioned and comprehensive external assessment of the Centre's programme and management, especially its future directions and the quality and relevance of its research. The TORs for the EPMRs (which update the "standard TORs" endorsed by the CGIAR at MTM95) are provided below. Guidelines for undertaking the reviews are issued separately.
TERMS OF REFERENCE
Objectives and Scope
6. EPMRs seek to inform CGIAR members that their investment is sound, or recommend measures to make it so. Members of the CGIAR and other stakeholders can be informed whether the Centre is doing its work effectively and efficiently. EPMRs are both retrospective and prospective; and help ensure the Centres' excellence, relevance and continued viability, and the CGIAR System's coherence. Each review is expected to be strategic in orientation and as comprehensive as the situation warrants.
7. The broad objectives of EPMRs are to: a) provide CGIAR members with an independent and rigorous assessment of the institutional health and contribution of a Centre they are supporting; and b) to provide the Centre and its collaborators with assessment information that complements or validates their own evaluation efforts, including the CCERs.
8. The EPMR panel is specifically charged to assess the following:
a. The Centre's mission, strategy and priorities in the context of the CGIAR's priorities and strategies;b. The quality and relevance of the science undertaken, including the effectiveness and potential impact of the Centre's completed and ongoing research;
c. The effectiveness and efficiency of management, including the mechanisms and processes for ensuring quality; and
d. The accomplishments and impact of the Centre's research and related activities.
9. The topics expected to be covered by the EPMRs are listed below.
TOPICS TO BE COVERED
A. Mission, Strategy and Priorities
- The continuing appropriateness of the Centre's mission in light of important changes in the Centre and its external environment since the previous external review.- The policies, strategies, and priorities of the Centre, their coherence with the CGIAR's goals (of poverty alleviation, natural resources management, and sustainable food security), and relevance to beneficiaries, especially rural women.
- The appropriateness of the roles of relevant partners in the formulation and implementation of the Centre's strategy and priorities, considering alternative sources of supply and the benefits of partnerships with others.
B. Quality and Relevance
- The quality and relevance of the science practised at the Centre.- The effectiveness of the Centre's processes for planning, priority setting, quality management (e.g., CCERs, peer reviews and other quality and relevance assurance mechanisms), and impact assessment.
C. Effectiveness and Efficiency of Management
- The performance of the Centre's Board in governing the Centre, the effectiveness of leadership throughout the Centre, and the suitability of the organization's culture to its mission.- The adequacy of the Centre's organizational structure and the mechanisms in place to manage, coordinate and ensure the excellence of the research programmes and related activities.
- The adequacy of resources (financial, human, physical and information) available and the effectiveness and efficiency of their management.
- The effectiveness of the Centre's relationships with relevant research partners and other stakeholders of the CGIAR System.
D. Accomplishments and Impact
- Recent achievements of the Centre in research and other areas.- The effectiveness of the Centre's programmes in terms of their impact and contribution to the achievement of the mission and goals of the CGIAR.
GUIDELINES FOR EXTERNAL PROGRAMME AND MANAGEMENT REVIEWS OF CGIAR CENTRES
INTRODUCTION
1. External Programme and Management Reviews (EPMRs) of CGIAR-supported Centres are carried out in accordance with the process Guidelines outlined below and the Terms of Reference (TORs) issued separately. Each review is expected to be strategic in orientation and as comprehensive as the situation warrants. To be credible and acceptable, all reviews must strive to be objective, transparent and participatory. The reports must be direct, explicit and frank. These principles are observed throughout the review process.
2. Being a member of a review panel is usually an interesting and rewarding experience. Moreover, Centre management and staff generally welcome the opportunity to discuss with panel members their achievements, concerns and future plans. A healthy atmosphere of mutual respect and collaboration in the interchange of ideas is the key to the success of the review. It helps to ensure that the recommendations of the panel are realistic, are well understood by the Centre management and staff, and will be willingly, or even enthusiastically, implemented.
GUIDELINES
3. EPMRs are expected to maintain high standards of quality and rigor, and be conducted by an independent and objective panel. The EPMR is expected to assess the Centre in terms of its: mission and strategy, programme priorities; quality and relevance of its science; achievements and impact; and effectiveness and efficiency of management, as noted in the TORs.
4. It is inevitable that the conduct of a review requires the collaboration of numerous individuals; as well as a process that enables the various participants to collaborate effectively in a complex assessment that has to meet high expectations and tight deadlines. The main participants in an EPMR are: the EPMR panel Chair and members; the CGIAR Members, TAC and the CGIAR Secretariat; members of the Centre's Board, management and staff; the panel's support team of external consultants and resource persons from the TAC and CGIAR Secretariats; and the Centre's many partners at the local, national, regional and international levels.
Roles and Responsibilities
5. The CGIAR, TAC and the Secretariats. The CGIAR establishes external review policies for the System, and EPMRs are conducted on its behalf, in accordance with the TORs and Guidelines endorsed by the Group. For each review, CGIAR Members are requested to propose Centre-specific issues for the panel to consider, and receive the review report. Once the timing of a particular EPMR has been determined, generally according to the 5-yearly schedule, TAC and the CGIAR Secretariat are responsible for the coordination and management of the EPMR, and they provide guidance on matters of review design and panel composition, in consultation with the Centre's Board and management.
6. Senior staff members of the TAC and CGIAR Secretariats serve as resource persons throughout the review process, accompanying the panel Chair and members during their visits to the Centre and on field visits. Besides substantive briefings on technical and management matters, they assist the panel on process matters, including the logistical aspects of report preparation and production. However, to help safeguard the EPMR panel's independence and objectivity, the Secretariats' resource persons are not normally expected to undertake substantive review, analysis or writing responsibilities on behalf of the panel.
7. The Panel Chair, Members and Consultants. The leadership and task management skills of the panel Chair are obviously critical, as are the expertise and experience of panel members. The Panel Chair is appointed by TAC in consultation with the CGIAR Secretariat and the Centre, The Chair's involvement begins early on, when he/she is consulted regarding panel composition, and briefed by TAC and the CGIAR Secretariat about the review process and key issues and concerns regarding the Centre. Once the review is underway, the Chair is responsible for ensuring that the panel undertakes its assessment and completes the task in accordance with the TORs and Guidelines for EPMRs. Given the magnitude of the task, the complexity of the issues, the fact that many panel members may by unfamiliar with the CGIAR, the importance of maintaining dialogue with the Centre, and the need to produce a report that reflects the consensus of the panel, the Chair's task is a demanding one.
8. Because the report should reflect the judgement of the whole panel, all members of the panel are expected to contribute to all aspects of the review report. Staff provided by the Secretariats assist the panel Chair and members throughout the process, as appropriate. Consultants are also provided to the panel, as needed, for limited periods of time, for assessment of specialized areas. While these consultants and resource persons from the Secretariats (and sometimes a TAC member) support the panel's efforts as members of a team, ultimately the panel is responsible for formulating the assessment and recommendations of the EPMR report.
9. The Centre Board, Management and Staff. The Centre's Board, management and staff play a crucial role in the conduct of the review. They are heavily involved in planning the review, and subsequently in organizing the review and preparing for the panel's visits to the Centre and to the field. Once the review is underway, it entails a significant degree of interaction between the EPMR panel and Centre staff, as part of a valuable two-way learning experience. Throughout the process, the collaboration and inputs of Centre management and staff are essential for the review to run smoothly and for the report to be credible and acceptable.
10. The Centre's Partners. Representatives of national agricultural research systems (NARS), regional fora, bilateral and multilateral agencies, NGOs and the private sector are important partners of CGIAR Centres, and their input is considered essential for the viability of the EPMR review process. As part of the review, representatives of such organizations are consulted for their views on the Centre's strategy, programmes and collaboration. This may be through panel visits and/or meetings, as well as through questionnaires or interviews. The panel may also visit or contact managers and researchers from other CGIAR Centres and other relevant institutions with which the Centre collaborates. Such consultations are valuable as a means of assessing the Centre's role in the CGIAR and in the global context. Given the vast number of collaborators or potential partners of a Centre, such meetings must be limited. Their outcome is considered important, however, and is expected to feed into the panel's assessment of the Centre.
Panel Composition and Report
11. Panel Composition. The review panel is composed of experts in programme and management areas relevant to the Centre being reviewed who can carry out a comprehensive assessment and give the CGIAR their best judgement about the past performance and future potential of the Centre. The panel is expected to make an independent assessment based on its own observations and other information available to it, particularly the evidence provided through CCERs (see below).
12. The EPMR panel normally consists of about five members, including the Chair. Panel members are generally selected for their ability to focus on the institution-wide issues relating to the Centre's mission, strategy, priorities, programmes and management. To ensure adequate coverage of the TORs, the panel composition usually meets the following requirements: a) the Chair and at least two panel members are familiar with the CGIAR; b) at least two panel members have a technical background relevant to the Centre being reviewed; and c) at least one panel member has expertise in institutional governance and/or research organization and management.
13. Panel Report. The EPMR report is expected to present an accurate account of the outputs and what is known about the impact of the Centre during the review period. It is expected that in-depth reviews of particular programme or management areas would have been undertaken earlier through CCERs - and would not normally need to be undertaken by the EPMR panel. This enables the EPMR panel to concentrate on the important strategic issues rather than on specialized detailed assessments of each programme, project or activity.
14. Because research in the CGIAR System is a long-term undertaking, the problems the Centre is working on may not have visible outputs until several years from now. For this reason, the review report is expected to provide convincing evidence on the quality and relevance of the completed and ongoing research, and the efficiency with which the work is conducted, as a surrogate measure of the potential impact of the Centre's current programme of work.
15. Although the EPMR report is expected to be comprehensive, the panel has considerable leeway in deciding on what issues it would focus in depth. The review report highlights the most significant issues faced by the Centre and makes recommendations on how the Centre (or the CGIAR) could address them. It provides assurances and convincing evidence to indicate that other aspects of the Centre's programmes and management (i.e., those not covered by the panel's report in depth) are effective and efficient. It also comments on the effectiveness of the Centre's internal review system on which the EPMR was based, and on how well the Centre has addressed the recommendations of the other reviews commissioned by the Centre.
Integration with Centre Reviews
16. It is expected that some detailed high-quality CCERs would have been completed within 2 or 3 years preceding the main phase of the EPMR. The CCERs are undertaken by specialized external consultants, assisted by members of the Centre Board and staff as resource persons (not participants). They are expected to cover at least portions of the Centre's main research programmes (including their relevance, direction, science quality, achievements, and, to the extent possible, impact) as well as aspects of Centre management (including governance, research organization and management, finance and human resource management).
17. The Boards would decide which CCER reports are made available, at the time of their completion, to TAC and the Secretariats. These reports, along with comments from TAC and Secretariat staff, are made available to the EPMR panel, along with reports of the follow-up actions planned or taken by the Centre's management and Board. Other analytical papers - particularly internal assessments of programme performance and impact - and other background documentation prepared by the Centre are also provided to the panel, at the discretion of the Centre. The Centre is responsible for providing this information in an easily accessible and usable form, so that the EPMR panel's conclusions can be based on a comprehensive and thorough review of all aspects of the Centre.
18. The CCERs - which are often very detailed and comprehensive - provide essential evaluative information to the EPMR panel on particular aspects of the Centre's programme and management. Their availability in advance of the main phase of the EPMR helps create an integrated system of Centre- and CGIAR-commissioned reviews of each Centre, and enables the EPMR to be forward-looking and to focus more on strategic, rather than operational, issues.
19. The EPMR, then, can serve as a vehicle for analyzing, verifying, and synthesizing the information already available through CCERs and other reviews, and for making this information available to a wider audience outside the Centre. While the Centre's Board and management are responsible for ensuring that the internal evaluation system is sound (in terms of scope, coverage, quality and timeliness), judgements on the adequacy of a Centre's quality assurance system, including the processes for undertaking CCERs and other mechanisms of peer review, are the responsibility of the EPMR panel.
Board Assessment Visit
20. Interactions between the Centre Board and the panel form an essential component of every review, given the Board's important role in the CGIAR System. Hence, early in the process, prior to (or sometimes during) the first visit of the full panel to the Centre (see below), the panel Chair along with Secretariat staff and possibly one other panel member or consultant attend a Board meeting, and interview Trustees concerning Board and Centre matters. This helps ensure the participation of the Board in the planning and design of the upcoming review, including the identification of key issues and concerns of relevance to the EPMR.
21. The visit also provides the panel Chair and selected members or consultant an opportunity to review the documentation provided to the Board, interact informally with individual Board members, observe at least one formal meeting of the Board and its committees, and serve as an element in assessing the Board's effectiveness and operations. The preliminary written assessment of the Board is made available to the panel (but not the Centre), and is modified as appropriate during the main phase of the EPMR (see below).
22. In assessing Board effectiveness and operations, the panel takes into account the key legal documents governing the Centre - particularly the Establishment Agreement, the Headquarters Agreement, and the Constitution of the Centre. It also keeps in mind the main provisions of the Guidelines for CGIAR Boards endorsed at MTM95, particularly the guideline on the "Role, Responsibilities and Accountability of Centre Boards of Trustees".
First Panel Visit and Briefing
23. Following the panel Chair's visit for discussions with Board members (or sometimes coinciding with it), the full panel undertakes its first visit to the Centre headquarters for about one week to ten days for initial discussions with Centre management, staff and Board members (when the visit coincides with the meeting of the Executive or Programme Committees of the Board). This initial phase visit enables the panel to obtain an overview of the Centre's current activities and future plans, to identify strategic issues to be covered by the review team, and to prepare preliminary drafts of key sections based on an agreed outline of the report which will be completed during the main phase several months later. During the first two days of this first visit, the panel receives detailed briefings from Secretariat staff and Centre management on the recent developments in the CGIAR and the Centre being reviewed, and on the processes, quality and content of the CCERs made available to the panel.
24. Briefings by the TAC Secretariat resource person cover technical matters such as the CGIAR's mission, priorities, strategies, programmes and impact assessments; and by the CGIAR Secretariat resource person, management matters such as the CGIAR's governance, organization, finance and human resources. These briefings by staff of the Secretariats also cover the CGIAR's expectations regarding the scope and process of the review (as outlined in the TORs and Guidelines for EPMRs); as well as an overview of programme and management issues of relevance to the Centre being reviewed. The resource persons from the TAC and CGIAR Secretariats also provide substantive and process-oriented support as requested by the panel Chair. One of the resource persons, usually from the TAC Secretariat, serves as the panel Secretary.
25. The panel then receives briefings from Centre management and senior staff on the Centre's strategy, priorities, programmes, governance and management. These briefings focus particularly on the Centre's recent developments and achievements, CCER findings and conclusions, and future plans. In addition, the panel seeks additional information from other Centre staff, on a selective basis, as needed; and invites Centre staff members, either individually or in small groups, to voluntarily share their concerns, if any, regarding Centre-wide programme and management issues.
26. To help ensure that these briefings and discussions are as comprehensive and up-to-date as possible, and to enable the panel to obtain a comprehensive overview of the Centre's work, the Centre is expected to provide to the Secretariats and panel members, in advance of the first visit, copies of the recent CCERs and other assessments undertaken, as well as other relevant Centre-related documentation (such as the latest Strategy document, Medium Term Plan, Programme and Funding Request, and other relevant policy documents or analytical papers prepared by the Centre). For the list of documents generally provided to the panel by the Secretariats and the Centre, see Attachment I. Most of these documents are off the shelf, and do not need to be especially prepared for the EPMR.
27. Towards the end of this first visit to the Centre, the panel spends two to three days preparing preliminary drafts and precis of sections based on the outline of the report agreed by the panel and the writing responsibilities assigned by the panel Chair. This ensures that the panel undertakes a significant amount of preliminary drafting prior to the main phase of the review, and continue its assessment of the key issues and concerns during the period between the initial phase and the main phase.
Field Visits
28. To help ensure that the EPMR panel's assessments are adequately grounded in the reality of the Centre's circumstances, the panel is expected to undertake country field visits, jointly determined by the Centre, panel Chair and the Secretariats. The field visits cover the major non-headquarters based operations of the Centre, so as to provide a realistic assessment of the Centre's field operations, working conditions, and interactions with NARS and others in the region. These visits by panel members (as smaller "sub-panels", if necessary) are often for about one week each, and are undertaken before the main phase of the review.
29. A senior staff member from the Centre normally accompanies the (sub)panel members on these field/country visits, but does not participate in substantive discussions with country officials or representatives of regional fora. The resource persons from the TAC and CGIAR Secretariats help coordinate the field visits and accompany the panel members, as requested by the panel Chair. These visits supplement any surveys of NARS and Centre staff, organized by the resource persons from the Secretariats in advance of the main phase.
Main Phase and Report Writing
30. The EPMR panel visits the Centre for a period of about three weeks to undertake the main phase of the review, and to complete and submit its report to the Centre's Board and management. As noted earlier, the Centre is expected to have made available to the panel, well in advance of this visit (through the panel Chair and Secretary), copies of CCER reports and other relevant documents; and the panel is expected to have completed the field visits and been adequately briefed by the resource persons from the Secretariats. The panel is thus expected to be reasonably well informed about the Centre and be familiar with others' detailed evaluations of its specific programmes and activities by the time it undertakes its own assessment of the Centre.
31. The EPMR panel's report is expected to focus on the four topics covered in the TORs - namely, the Centre's: a) mission, strategy and priorities; b) quality and relevance of science; c) effectiveness and efficiency of management; and d) what is documented about accomplishments and impact. The report is expected to be succinct and written in plain language, focusing on strategic issues. It can, where relevant, propose forward-looking recommendations on overall direction and priorities (rather than on detailed programme content or operational management). The writing style is expected to be direct, explicit and frank.
32. Since descriptive material and detailed analysis is expected to be kept to a minimum, a report of about 60-70 pages - with suitable cross-referencing (not summaries) of the CCERs - is expected. However, if the CCERs available to the panel are inadequate in quality, coverage or depth, the EPMR panel's report is expected to compensate for gaps through its own analysis and assessment.
33. The drafting of the EPMR report is completed during the main phase, and the final draft chapters are shared with the Centre management to ensure their accuracy and completeness. The final EPMR report is normally expected to be completed at the Centre; and is presented by the panel Chair to the Centre Board, management and staff before the panel's departure upon completion of the main phase. Simultaneously, the panel Chair formally transmits the document to the TAC Chair and the CGIAR Executive Secretary.
Response and Follow-up
34. The Board and management of the Centre under review are expected to submit a formal written response to the EPMR report, addressed to TAC and the CGIAR Secretariat. Then TAC discusses the report in the presence of the panel Chair and representatives from the Centre (including the Board Chair and Director General), and prepares a commentary, in collaboration with the CGIAR Secretariat, including recommendations for follow-up action by the CGIAR or the Centre. The EPMR report, the Centre's written response, and the TAC and CGIAR Secretariat commentary(s) are then simultaneously distributed to and considered by numerous individuals within CGIAR member agencies prior to its formal discussion by the Group.
35. As a final step in the review process, the CGIAR discusses the EPMR report and the accompanying comments at either its mid-year or annual meeting, and agrees on follow-up action. Occasionally, this follow-up includes a CGIAR-commissioned Mid-Term Review (MTR) to monitor closely the Centre's handling of major concerns raised during the EPMR. Generally, however, the Centre's progress in implementing the recommendations of the EPMR is followed-up by the next EPMR panel five years later, and are reported to the CGIAR - as a mandatory Annex of the EPMR report - thus completing the external review cycle.
CONCLUSION
36. EPMRs provide the CGIAR and other stakeholders very valuable information on the accomplishments and future prospects of each Centre funded by the Group. Because they undertake a comprehensive strategic assessment of all key aspects of the institution, such reports from an independent external panel can provide much needed assurance to the CGIAR Members - as well as to the Centre's Board, management, staff and partners - about the Centre's direction and its institutional capacity to produce the desired research results. If significant changes in direction, scope, focus, or mode of work are required, these too can be made on a systematic and periodic basis, based on Board-endorsed EPMR recommendations. In any case, the Centre and the System benefit from such reviews.
Attachment I: LIST OF DOCUMENTS FOR EXTERNAL REVIEWS OF CGIAR CENTRES
The following is a list of documents for panels conducting external reviews of CGIAR Centres. Copies of relevant documents should be sent 10 panel members in advance of the review. Copies of all documents should be available at the Centre during the review.
|
|
Provided by: |
|||
|
TAC Secretariat |
CGIAR Secretariat |
Centre |
||
|
PROVIDED IN ADVANCE OF THE REVIEW |
||||
|
To all panel members: |
|
|
|
|
|
Terms of Reference and Guidelines for External Programme and Management Reviews of CGIAR Centres. |
x |
|
|
|
|
Review Processes in the CGIAR (1988). |
x |
|
|
|
|
Most recent External Programme and Management Review report of the Centre. |
x |
|
|
|
|
Most recent TAC paper on CGIAR Priorities and Strategies. |
x |
|
|
|
|
Most recent CGIAR stripe studies involving the Centre. |
x |
|
|
|
|
Most recent CGIAR Vision Statement. |
|
x |
|
|
|
Most recent CGIAR Annual Report. |
|
x |
|
|
|
Most recent CGIAR Brochure and Directory. |
|
x |
|
|
|
Most recent Annual CGIAR Funding Requirements document. |
|
x |
|
|
|
Organization and Management of the CGIAR System: A Review. 1993. |
|
x |
|
|
|
Most recent Annual Report of the Centre, and comparable research reports of the programmes, if available. |
|
|
x |
|
|
The latest Board-approved Strategic Plan of the Centre. |
|
|
x |
|
|
The latest Medium-Term Plan of the Centre. |
|
|
x |
|
|
The latest Annual Funding Request. |
|
|
x |
|
|
A paper summarizing the main achievements, constraints and impact of the Centre programmes since the last CGIAR external review. |
|
|
x |
|
|
A brief paper outlining the major issues confronting the Centre. |
|
|
x |
|
|
The current organization chart, with a brief description of the Centre's internal management structure, including the composition and terms of reference of each major committee. |
|
|
x |
|
|
List of senior staff with brief summary of qualifications. |
|
|
x |
|
|
Centre-Commissioned External Review Reports. |
|
|
x |
|
|
List of reports of major planning conferences, internal reviews, expert meetings, etc., which have had a major influence on the direction of specific Centre programmes. |
|
|
x |
|
|
Self-studies, if conducted, assessing strengths and weaknesses of Centre programmes and/or management |
|
|
x |
|
|
Summary of actions taken in response to the last External Programme and Management Review. |
|
|
x |
|
|
A list of staff publications during the period under review. |
|
|
x |
|
|
List of the agreements for cooperative activities with other centres and institutions. |
|
|
x |
|
|
List of ongoing and recently completed contracted projects. |
|
|
x |
|
|
Supplementary documents, provided to relevant panel members: |
|
|
|
|
|
Most recent statements of CGIAR policies of relevance to the Centre. |
x |
|
|
|
|
Most recent CGIAR Medium-Term Resource Allocation document. |
x |
|
|
|
|
Relevant extracts from TAC and CGIAR reports of the most recent discussions on the Centre's programme (e.g. Strategic Plan and Medium-Term Plan). |
x |
|
|
|
|
Overview of Management in the CGIAR Centres. 1990. |
|
x |
|
|
|
Most recent volume of the CGIAR Board of Trustees Directory. |
|
x |
|
|
|
Roles, Relationships and Responsibilities of Trustees and International Agricultural Research Centres. 1984. |
|
x |
|
|
|
Some Thoughts Toward Ensuring the Successful Performance of Boards in the CGIAR System. 1987 |
|
x |
|
|
|
Most recent Annual CGIAR Financial Report. |
|
x |
|
|
|
Most recent CGIAR financial guidelines and manuals relating to: |
|
|
|
|
|
|
- Financial Management. |
|
x |
|
|
|
- Accounting Policies and Reporting Practices Manual. |
|
x |
|
|
|
- Audit Policies and Procedures. |
|
x |
|
|
Charter and other basic documents establishing the Centre, along with subsequent amendments. |
|
|
x |
|
|
Table showing composition of the Board over the last five years, along with an indication of the term of office of current members and their roles on the Board. |
|
|
x |
|
|
Board handbook or rules of procedure. |
|
|
x |
|
|
Table showing allowances, benefits, and salary ranges for each category of staff. |
|
|
x |
|
|
Table showing personal data on internationally recruited staff by programme, including each job title, incumbent's location, period of tenure, gender, nationality, age, salary over the last three years, funding source (excluding names). |
|
|
x |
|
|
Table summarizing turnover of staff over the last five years by staff category. |
|
|
x |
|
|
List of international staff vacancies and how long positions have been vacant. |
|
|
x |
|
|
Brief description of the Centre's information management systems and procedures (e.g., library and documentation, archives and records management, computer and information technology, management information systems). |
|
|
x |
|
|
AVAILABLE AT THE CENTRE (one copy) |
||||
|
Set of minutes covering Board and Board committee meetings since the last External Review (and reports of board committees to the full Board if not included in the minutes). |
|
|
x |
|
|
Staff manual or a description of current personnel procedures for international and locally-recruited staff. |
|
|
x |
|
|
Local compensation surveys used by the Centre. |
|
|
x |
|
|
Reports of external auditors, including management letters, and financial officer's reports to the Board since the last External Review. |
|
|
x |
|
|
Most recent internal audit reports. |
|
|
x |
|
