The changing global context and the diversity, both within and between the regions in which IPGRI operates often raise many complex and sometimes conflicting interests for its work and resources. National programmes, networks, policy makers, scientific institutions, NGOs and donors have continuously changing expectations of IPGRI. For example, while scientific institutions often expect IPGRI to be engaged with them in laboratory and field based genetics research, many NGOs expect the Centre to provide technical and development assistance. Contracting parties to the CBD and the ITPGRFA have different needs and expectations in terms of IPGRI’s support to their efforts to implement the treaties. Each group of donors has different and often changing expectations of IPGRI. Bilateral donors’ expectations tend to be determined by their domestic political constituencies, while private foundations’ decisions on funding to IPGRI are likely to be influenced by economic market forces. Depending on their peculiar domestic political and economic considerations, different groups of donors will exert different influences on the Institute.
In these changing and increasingly complex situations IPGRI is confronted with the challenge of designing its programmatic work and administrative structures to balance the many different interests while at the same time sharply focusing limited resources on specific problems. It cannot and should not respond to all of its clients’ specific demands. IPGRI’s leadership has the responsibility to ensure that the Institute’s mission and vision are not undermined and indeed that IPGRI stays focused and the quality and impact of its work improved. Strategic planning is the process which IPGRI uses to respond to the challenges posed by the changes in the external and internal contexts.
Strategic planning has always been an integral part of IPGRI’s evolution and operations. Its evolution from IBPGR involved the identification and adoption of a clear mission, vision and programme of work on PGR conservation and sustainable use. As it has made the transition from an agency that focuses on a narrow range of scientific issues of conservation to one that takes a holistic view of the problem of PGR management in its wider context and as its constituency has expanded from national PGR ex situ programmes to a range of other conservation and development actors, strategic planning and priority setting have become important. Over the years it has formulated and revised its strategic focus.
IPGRI’s first overall Strategy was adopted in 1993. Since the last EPMR, IPGRI has revised the Strategy and established an institute wide process of strategic planning. In the current Strategy IPGRI made "Eight Strategic Choices" as a set of operational statements that will direct priority setting and resource allocation. These Choices, which provide the overall umbrella for IPGRI’s thematic and regional Projects and activities are as follows: (1) Strengthening NARS; (2) Working with networks (regional and crop activities); (3) Improving conservation strategies and technologies; (4) Increasing use of genetic resources; (5) Managing and communicating information; (6) Addressing socio-economic and policy issues; (7) Conserving and using specific crops; and (8) Conserving and using forest GR.
These Choices all relate directly to the Centre’s Objectives and, appropriately, to the 20 Activities of the GPA (with exception of the strategic choice pertaining to FGR which is not included in the GPA). The Choices also accommodate the new commodities that have been added to IPGRI’s portfolio and those being considered, as well as new areas for socio-economic and policy work. The Institute’s priority setting is largely determined within these Choices. While the Choices do not change, new priority issues are determined and adopted to constitute Projects and Activities.
Generally, strategic planning has become a continuous process in IPGRI. The staff has flexibility in the course of operations to suggest new research issues and questions that fit the Choices. This arrangement is important in the sense that it creates space for continuous creativity by staff and the Institute as a whole. To consolidate this, biannual strategic planning meetings take place. The annual strategic planning is managed within PPRC (see Chapter 8). The establishment of the new PPRC and the Innovation Fund and, more recently, the development of an Institute wide logical framework are intended to improve strategic planning in IPGRI. These mechanisms and tools are also meant to provide broad parameters for planning at the Programme, Project and Activity levels both at HQ and in the Regions.
Regional Strategic Plans have been developed and adopted by APO and SSA with considerable input of national and regional actors through workshops. A survey conducted by the Panel shows that stakeholders highly appreciate IPGRI’s efforts to open its planning process in order to take their needs into account. They rated the organization and management of the process high but some noted that there is need to harmonise the IPGRI regional strategic planning with those of PGR networks.
The Panel commends IPGRI for establishing the mechanisms and tools to institutionalise and improve strategic planning. However, the Panel has noted some lack of transparency in making decisions regarding the justification and relative prioritization across the strategic choices and how these are translated into resource allocation among Projects and Activities (see Chapter 7). There also appears to be a lack of clarity in the criteria for deciding longer term resource commitments across activities and "shifts" often appear to be based on "qualitative" judgements. With the exception of the three year rolling Medium Term Plan, longer term strategic planning documents typically make no clear mention of staff numbers, budgets or the relative shares of different research topics. The Panel strongly suggests that management should build more clarity into the priority setting and Project and Activity acceptance mechanism.
IPGRI professional staff was asked to provide the Panel with details of their "Measures of Esteem". These included honours, prizes, representation on key committees and boards, invited lecture and other relevant achievements, successful grant applications, students supervised and other relevant experience as well as publications. 112 responses were obtained from Honorary Fellows and Associate experts as well as nationally and internationally recruited professional staff.
Averages for most of these measures are meaningless over the range of job descriptions and level of experience. However, the Panel has used this approach for assessing refereed publications. Honours and prizes included election to national academies of science, honorary chairs, national and international prizes for exceptional research work and services to science, agriculture and society. Forty four professional staff had received 134 such accolades. This figure was heavily influenced by IPGRI’s Honorary Fellows, who are employed precisely because of their achievements and academic standing. Whereas such recognition often comes later in careers, it is also clear that many of IPGRI’s younger researchers are making a significant impact in their chosen fields. Fifty six staff had sat on 209 key influential panels and committees. These ranged from Boards of Trustees at other CGIAR Centres and other such organizations, advisory panels of international scientific initiatives, national steering committees and international organizing committees of influential conferences. These measures of distinction were broadly spread over staff at HQ and in the Regions. Invited lecture statistics showed that IPGRI staff is recognized. Sixty two staff has delivered 371 invited and keynote addresses since 1996. One further relevant statistic is that 34 staff members supervised (usually and appropriately so, as co-supervisor) 134 MSc and PhD students. The Panel noted that the Honorary Fellows greatly boost the profile of IPGRI professional staff.
It is clear that, particularly at the more senior staff levels, IPGRI’s research and researchers are being recognized. This recognition is being translated into invitations to further influence other research and political agendas. These results are equivalent to those expected at a successful European research organization. The Panel hopes that the Centre will follow these indicators to judge progress in future years.
The Panel did consider the figures for interaction with postgraduates students rather low. It is clear that not all research staff has the same opportunity. However these interactions do provide one of the best means for scientists without their own laboratories and research programmes to stay current in their fields.
It is clear to the Panel that, as the Institute has grown, there has been increasing pressure on staff that distract them from research and keeping up with developments in their areas. There are obvious solutions such as having line managers try to ensure that their staff attends key conferences in their area and encouraging interactions with nearby universities, particularly in delivering undergraduate or MSc course lectures and in the co-supervision of postgraduate students. The Panel was also pleased to see that there was provision for staff to take Study Leave. It is disappointing that no staff have availed themselves of this opportunity and the reasons for this should be investigated. This tool, which is used by many academic organizations to recharge and reinvigorate hands-on researchers and their programmes, could be further tailored to suit IPGRI’s particular circumstances. The Innovation Fund, which has now been operating for two years, should also provide opportunities for staff to think "out of the box" and be given the chance to develop a new area with potential for the Centre. The proposals submitted for the 2001 and 2002 rounds do indicate that there are good possibilities here. Other opportunities are being considered, such as an IPGRI public lecture series and the organization of workshops at venues where staff can easily attend.
Undoubtedly the risk of loss of contact with new developments in science by IPGRI researchers, who were mostly recruited for their scientific achievements in the first place, is an issue. It does not seem to be a regional problem although staff at some locations are clearly more isolated than most. Others find themselves better placed to interact with local universities than HQ-based staff.
10.2.2.1 Ex ante assessment of projects
The key opportunity for open assessment of Projects at IPGRI is the PPRC, at which new activity priorities are presented and the previous year’s progress monitored and reviewed. Necessarily discussion of each of the two hundred and sixty Activities and 20 Projects over a single week can only be cursory. The size of the committee also discourages frank discussion and criticism. PPRC in its present enlarged form is probably still finding its feet and is too engrossed with administrative detail. It certainly must evolve further.
The Panel also hopes that, when MEC gets into its newly recommended mode of operation with regular formal meetings, there will be more focus on strategic questions. Moreover the involvement of other appropriate staff that are best placed to contribute to particular agenda items and with published minutes, should improve clarity and should enhance the climate of intellectual equity. This, in turn, should ensure that new and interesting ideas and activities receive the support that they deserve.
Although IPGRI itself does not apply external peer review to project proposals, the Panel notes that activities get reviewed externally by the donors before they receive restricted funding.
10.2.2.2 CCERs
A more infrequent review of IPGRI’s Project activities is provided by the CCER system. IPGRI is probably the CGIAR Centre which makes most use of the CCER and this EPMR was provided with 13 CCERs and additional audits (see Appendix V). Most areas are reviewed once in a 5 year period and provide feedback on staff performance and help Project coordinators make appropriate adjustments and identify gaps. In general the EPMR had a very good impression of the value of the CCERs to Management.
The key to most reviews lies in the detail of the ToR. The Panel reviewed these and found them mostly appropriate and comprehensive. However the quality of the CCERs did vary and was closely correlated with the quality of the ToR. Examples of CCERs that have been used to improve Project scope and quality are those for DIT and FGR. The Panel suggests that Management revisit ToRs to ensure that future CCERs take account of lessons learnt.
A final word on the value of CCERs to the EPMR. It has become clearly evident during this process that, although it is necessary that the Panel sample the detail of the science, it cannot possibly provide the Centre with the service that the CCERs do. This Panel has used the CCERs and their recommendations heavily and has found them quite complementary to its own deliberations, which have generally been above the level of individual Projects.
IPGRI’s mandate is in research for development and IPGRI should not be regarded as a conventional research institute. In line with its mandate and mission, it uses two major publication channels to disseminate the results of its work and that of its partners. The first channel consists of IPGRI’s own publications and of invited contributions to books published by third parties. This channel mainly serves IPGRI’s own constituency, the global PGR community. IPGRI’s own publications are generally categorised as original papers, manuals, handbooks, guidelines, technical bulletins, issues papers, proceedings and conference presentations. In addition, it publishes training materials. The most frequently generated publications are IPGRI Technical Bulletins and the Plant Genetic Resources Newsletter. These are produced in series. Articles published in the Technical Bulletins are often reviewed internally and target the scientific community while the Newsletter is peer reviewed and is for wider readership. Refereed journals that reach the scientific community are the second channel for publishing. These publications may influence the ideas and concepts of a wider community of researchers that somehow deal with genetic resources and are a means to register IPGRI’s scientific output and enhance its scientific standing. Approximately 75% of IPGRI’s publications are delivered through the first channel, in line with its mission to serve its global PGR constituency. Therefore, usual standards regarding the number and quality of scientific publications cannot be simply applied to IPGRI staff’s published output record.
The average number of articles authored or co-authored by IPGRI staff and published in refereed journals over the last six year was approximately 50 per year for some 100 IRS and LRP and 15 Honorary Fellows. However, of the 200 peer reviewed publications over the seven year period 1996 - 2002, 76 were authored or co-authored by just three scientists. Of these two were honorary fellows and one a senior scientist. This puts a different complexion on the performance. Much of IPGRI’s work is region specific and therefore it is entirely appropriate that many of the publications are not in internationally renowned journals, though these do not get a high score in an impact rating system.
The number of publications per IPGRI staff member per year is relatively low as compared to other CGIAR Centre[18]. However, it is interesting that IPGRI’s original papers have been highly cited, which reflects a high potential impact.
The distribution of journal articles across IPGRI’s Projects shows a bias towards INIBAP’s activities and to the GRST projects, in particular tissue culture and in vitro research. The figures show that IPGRI staff is relatively more successful in publishing original papers in highly technical areas than in other areas.
The Panel strongly suggests that appropriate steps are taken to improve IPGRI’s staff’s publishing record in refereed scientific journals in order to improve the status and outreach of the Institute and to draw attention to its concepts, strategies and research results. The Panel suggests that the IPGRI Publications Committee develop a clear policy, work plan and annual plan for the publication of results in refereed journals. This should take into account IPGRI’s strategy for raising its profile. The policy should identify conditions and mechanisms to make it attractive for IPGRI’s (senior and junior) staff to publish in peer reviewed journals. Publication targets for individual staff members and time allotments to prepare for agreed publications should be part of discussions held in the context of annual performance appraisals. These targets should, of course, be tailored to the individual job description. In many areas a low publication expectancy and record will be offset by other activities.
Networks play a major role in IPGRI’s operational mode. IPGRI’s contribution to the success of networks, measured in terms of rate of growth, intensity of interactions between the partners and cost efficiency and sustainability of network operations, should be regarded as a major IPGRI output. This success rate seems to be largely dependent on the degree of structural, topical and financial autonomy of the networks. The return on investment of IPGRI’s support to networks is closely related to this level of autonomy. IPGRI is advised to closely monitor network development and functioning in order to maintain a high level of effectiveness and efficiency in its network support.
The Panel obtained some measure of the success of IPGRI’s network activities from an analysis of the stakeholder survey data. The analysis shows that IPGRI’s research is valued as very significant or significant by stakeholders from genebanks, NARS, universities, government agencies, international organizations and NGOs. Highest scores were obtained from universities, genebanks and international organizations. IPGRI’s output in information, a measure of the quality of its own publications, was valued equally high.
In general, the Panel commends the quality of IPGRI’s work. IPGRI obtained outstanding results in the area of conservation and storage technologies, in standardization of documentation and in the development of inter-institutional PGR databases. In addition, its work on in situ conservation has attracted major attention.
The publication records, taking IPGRI’s own publications and journal articles together and the professional standing of IPGRI’s scientists underpin this overall assessment.
Impact studies in the CGIAR are define[19] as falling "along the continuum between academic research...and utilization-focussed evaluation". Impact assessments (IA) at IPGRI are set up to be a form of evaluation rather than fundamental research. Consequently there has been a move to using a mixture of quantitative analysis and more qualitative indicators.
At IPGRI, IA is a relatively new and small area of activity. Although IA is part of one of IPGRI Projects, ‘Understanding and communicating the value of PGR’, it is directly under the Office of the Director General, with supervision from the ADG. The objective of this area of work at IPGRI is to "increase awareness and understanding of PGR research and its role in development and increasing capacity to design and implement genetic resources research projects and programmes"[20] and it is closely linked to planning. In 1999 IPGRI appointed a full time evaluation and impact assessment specialist. IA work receives very limited core funding, but has been quite successful in receiving external funds. Consultants and interns, in particular, are employed for conducting the studies.
Research topics that nominally fall under impact and evaluation related studies range from descriptive type projects looking at institutional and organizational projects such as ‘The Effects of IPGRI Letters of Agreement on partner capability’ or ‘Evaluating IPGRI’s Fellowship Programmes’ to micro level production economics studies on, for example, ‘Economic benefits of coconut genetic diversity as a smallholder crop’. Such studies employ very different academic disciplines in their analysis. Furthermore they would appear to have very different potential value as an input in a public awareness activity.
Out of 19 impact-related case studies and reports completed by IPGRI 15 are concerned with IPGRI’s influence on institution building, policy formulation at the international level and the impact of its training and publication activities. Three deal with the use of germplasm and one on the value of on-farm diversity and are done by the economist shared with IFPRI as part of a new research programme. The new activity aims at assessing the social and economic impact of improved banana varieties in East Africa. IPGRI sees that further economist input would be desirable for conducting studies on on-farm and commodity research. The Panel encourages this development, as there is already a great call for ex post assessments of in situ work, particularly in the regions.
The IA activities have also included development of indicators, both at the Centre level and together with project teams for activities. The new work on network indicators aims at providing a monitoring scheme and guidelines for improving network productivity and for eventually assessing impacts.
IPGRI’s IA studies that evaluate its impact on institution related matters are helpful. The study on IPGRI coordinated projects that included COGENT, the in situ project in Nepal and the leafy vegetables project in Sub-Saharan Africa clearly established that, without IPGRI’s involvement, these projects would not have happened and that they pulled together a wide range of players to make an effective project where the sum of the parts was greater than the whole. Another study on maximising participation in PGR networks[21] shed interesting light on how to influence the ability of members to participate in network decision making. It also demonstrated the problems that can occur if there is excessive external funding at the early stages of the establishment of a network.
IA work aims at providing information and analysis to IPGRI Management for conclusions and action in strategic planning at Centre and activity level. Examples of this reiterative process can be found in the new publications and training strategies which have drawn from the respective IA studies. The IA work is also closely connected to the new theoretical underpinning, the sustainable livelihoods approach, that IPGRI is exploring for its work. The IA staff is collaborating with DFID in getting IPGRI’s programmes better focused towards poverty alleviation. IPGRI is engaging in a strategic planning exercise for its IA activities in which it plans to also involve stakeholders. The Panel agrees that a bottom-up approach to IA is essential for the ownership and strong development of national programmes and for IPGRI’s success in facilitating the process of capacity building.
The Panel agrees with the CCER’s positive assessment of IPGRI’s IA work and commends IPGRI for its innovative IA activities in the early stages of this research. The Centre is at the forefront in the CGIAR in exploring new areas as far as IA methodology and approaches are concerned. The CCER recommendation that the IA function should be divorced from the public awareness function is also being taken on board.
|
[18] Cherfas, J. 2003,
Towards a media strategy for IPGRI. IPGRI, Rome. [19] Mackas, R. and Horton, D. 2002, Expaning the use of impact assessment and other types of evaluation. In: CIMMYT and SPIA (forthcoming), Watson, D.J. (ed.), Summary proceedings, International conference on impacts of agricultural research and development. San Jose, Costa Rica, February 4-7, 2002. CIMMYT, Mexico [20] Watts, J. et al. 2000, Impact assessment in IPGRI: needs, constraints and options. In: Proceedings of a workshop organized by the Standing Panel on Impact Assessment of the Technical Advisory Committee. FAO, Rome, May 3-5, 2000. [21] Watts, J., 2002 One plus one equals three: maximising participation in plant genetic resource networks. Plant Genetic Resources Newsletter 130: 28-35 |
