Posted November 2000
By Marilee Karl
This bibliography covers materials that deal, directly or indirectly, with approaches to evaluating the extent and quality of participation, the costs and benefits of participation to stakeholders and the impact of stakeholder participation on the performance, outcomes and sustainability of projects and programmes.
These have been winnowed out of a much larger mass of materials dealing with evaluation and participation in general and with participatory monitoring and evaluation. Because evaluations explicitly addressing participation are only now beginning to emerge, the amount of materials directly addressing evaluation of participation, rather than participation in evaluation, is still limited.
Some of the materials on participatory monitoring and evaluation, however, do address issues of evaluation of participation. Information relevant to the evaluation of participation is also found in materials dealing with stakeholder analysis, conflict management and impact analysis. The relevant sources are included in this bibliography.
Although the focus of this bibliography is on evaluation of participation in rural development projects and programmes, some material is also included on participation in research and policy, as some of the lessons learned in these areas are also useful for projects and programmes.
This bibliography has been compiled from a number of sources, including:
The Informal Working Group on Participation (IWG-PA) web site's library data base; the FAO library; bibliographies compiled by IWG/PA members; and the web sites of the Department for International Development (DFID), the Electronic Development and Environment Information System (ELDIS), the Inter-American Development Bank (IDB), the Institute for Development Studies (IDS), the International Development Research Centre (IDRC), the International Labour Organisation (ILO), the International Institute for Environment and Development (IIED), the Organisation for Economic Co-operation and Development (OECD), the Swedish International Development Authority (SIDA), the United Nations Development Programme (UNDP), and the World Bank.
Abbot, Joanne and Irene Guijt, 1998, Changing Views on Change: Participatory Approaches to Monitoring the Environment, SARL Discussion Paper No. 2, London: International Institute for Environment and Development (IIED), 96 pp.
This discussion paper reviews participatory approaches to monitoring environmental change. It describes monitoring approaches that develop partnerships of multiple stakeholders for efficient, effective, and socially inclusive monitoring. The drive for accountability and the need for more information to improve planning processes has given impetus to participatory monitoring. This paper discusses various project-led approaches to participatory monitoring of the environment, many of which highlight the importance of indicator definition. The paper acknowledges the need to assess the claimed benefits of participatory monitoring and gives an example of how this has been done. It also looks at stakeholder analysis and negotiating the needs of different stakeholders. An extensive list of references is attached.
Bhatnagar, Bhuvan, 1992, "Participatory Development and the World Bank: Opportunities and Concerns", in Bhatnagar, Bhuvan and Aubrey C. Williams (eds.), Participatory Development and the World Bank: Potential Directions for Change, World Bank Discussion Paper 183, Washington D.C.: The World Bank, pp. 13 - 30.
This paper reports on the results of a survey of 20 World Bank-supported participatory projects. It focuses on the survey questions dealing with the capacity of the Bank to support participation, including the extent of participation, the time and money required, and the staff resources needed. Three preliminary hypotheses on costs emerge:
Bhatnagar, Bhuvan and Aubrey C. Williams (eds.), 1992, Participatory Development and the World Bank: Potential Directions for Change, World Bank Discussion Paper 183, Washington D.C.: The World Bank, 195 pp.
In the early 1990s, the World Bank initiated an internal learning process on popular participation, which included an international workshop in February 1992. This volume presents a summary record of that workshop. It includes background papers, a description of the learning process, and preliminary findings. An important part of the initiative was a survey of 20 Bank-supported participatory projects, which contained questions on the Bank's capacity to support participation, including the costs to the Bank. This volume discusses the provisional lessons of this initiative, focusing on:
Campbell, Jock and Venkatesh Salagrama, 2000, New Approaches to Participation in Fisheries Research, Rome: FAO and SIFAR, 56 pp.
This discussion document explores the options available for expanding participation in fisheries research. Placing this exploration in the larger context of the concepts of participation and the current usage of participation in research, the paper enumerates the potential benefits of balanced participation of different stakeholders. An extensive bibliography is attached.
Cernea, Michael M. (ed.), 1991, Putting People First: Sociological Variables in Rural Development, New York: Oxford University Press, 575 pp.
The main theme of this publication is that "putting people first" in development programmes is an imperative in successful development. The book highlights issues related to natural resources management, particularly water, forests, and fisheries; the environmental implications of development programmes; and the development of human capital through investments in grassroots organizations and participation. It also points to such adverse consequences of development as the risk of greater impoverishment for some marginal groups, the forced displacement and involuntary resettlement of populations, and the deterioration of common income-generating assets. The authors stress the operational relevance of the sociologist's task. They draw lessons, both positive and negative, from many World Bank-assisted development projects, as well as projects supported by other international, bilateral, or national development agencies. In addition to sections on sectoral projects, the book contains sections on evaluation, participation and the collection of social data.
Clayton, Andrew, Peter Oakley and Brian Pratt, 1998, Empowering People: A Guide to Participation, New York: UNDP, 58 pp.
Prepared for UNDP staff promoting participation in UNDP programmes, this guidebook covers the concepts of participation in development, strategies for participation, methods of promoting participation (including stakeholder analysis), the monitoring and evaluation of participation, institutional support for participation and a listing of resources. The chapter devoted to monitoring and evaluation of participation looks at conceptual issues including the need to recognize both the qualitative and quantitative dimensions of participation. It discusses critical issues in selecting indicators of participation and gives examples of both qualitative and quantitative indicators. The process of participation at project level can only be evaluated if it has been monitored, and the some key features and examples of a monitoring system for participation are given. This chapter also looks at issues of the interpretation of qualitative findings.
Department for International Development (DFID), 1995, Guidance Note on How to Do Stakeholder Analysis of Aid Projects and Programmes, London: DFID, 17 pp.
This Guidance Note supplements the DFID Technical Note on Enhancing Stakeholder Participation, and provides practical tips on how to do stakeholder analysis, and how such analysis can be used.
Department for International Development (DFID), 1995, Guidance Note on Indicators for Measuring and Assessing Primary Stakeholder Participation, London: DFID, 9 pp.
This paper points out that work in the field of developing and selecting indicators for measuring primary stakeholder participation is relatively new. Summarizing current usage, the paper reviews the qualitative, quantitative and time (QQT) dimensions of indicators for measuring participation. Means of verification are then examined and the paper concludes with a checklist. A select bibliography is attached.
Department for International Development (DFID), 1995, Technical Note on Enhancing Stakeholder Participation in Aid Activities, London: DFID, 24 pp.
Synthesizing DFID's own best practices and drawing on the experiences of other agencies, this paper covers a range of issues on stakeholder participation in projects and programmes. Part 1 answers the questions of why, when and how to encourage stakeholder participation and why participatory approaches are important for sustainable and effective aid programmes. It examines the key issues in both primary and secondary stakeholder analysis and provides steps for negotiating stakeholder participation in DFID-financed activities. It also briefly looks at the questions of feasibility and cost-effectiveness of participation. Part 2 deals with enhancing participation in policy and projects, including methods and checklists. This section touches briefly on some of the costs involved. A select bibliography is attached.
Development Assistance Committee (DAC) Expert Group on Aid Evaluation, Organisation for Economic Co-operation and Development (OECD), 1997, Evaluation of Programs Promoting Participatory Development and Good Governance, Paris: OECD.
This report is the synthesis of a study carried out by the OECD / DAC Expert Group on Aid Evaluation for the purpose of increasing the understanding of what assistance programmes and strategies donor agencies can effectively use to promote good governance and participatory development in the emerging democracies of the Third World. Each chapter focuses on a specific area, summarizing donors' experiences and lessons learned in support of:
The chapter on participation examines:
Edgerton, J., K. McClean, C. Robb, P. Shah and S. Tikare, April 2000, "Participatory Processes in the Poverty Reduction Strategy" in Poverty Reduction Strategy Sourcebook, Draft for Discussion, Washington D.C.: The World Bank, 30 pp.
This draft chapter of the World Bank Poverty Reduction Strategy Sourcebook focuses on developing participatory processes while the poverty reduction strategy is being formulated or strengthened. It looks at types and dimensions of participation; the participation of local people in the diagnosis of poverty; consultation and information flows within government; civic engagement at the local and national levels; and information dissemination and feedback to stakeholders. The chapter also addresses possible pitfalls and constraints.
Estrella, Marisol and John Gaventa, no date, Who Counts Reality? Participatory Monitoring and Evaluation: A Literature Review, IDS Working Paper 70, Brighton: IDS, 70 pp.
This paper presents a literature review of experiences in participatory monitoring and evaluation (PM&E) from around the world, used in differing contexts and involving different kinds of stakeholders - NGOs, donors, research institutions, government, people's organizations, and communities. It introduces the key principles of PM&E, its application for differing purposes, a number of tools and methods used, including participatory leaning methodologies as well as more conventional approaches. Finally, it raises key issues and broad challenges from the literature and outlines potential areas for future research. An extensive list of resources and references is attached.
Food and Agriculture Organization (FAO), 1990, Participation in Practice: Lessons from the FAO People's Participation Programme, Rome: FAO, 44 pp.
Written for policy makers and development practitioners, this paper reviews the first decade of the work of FAO's People's Participation Programme (PPP) and summarizes the lessons learnt. After a brief overview of the reasons for promoting participation in rural development projects and programmes, it describes the PPP and its working hypothesis: that of realizing people's participation through small group formation. The paper provides guidelines for replication of the PPP approach. A section of the paper deals with the costs and benefits of participation. Although the cost-effectiveness of participation is difficult to determine and data is fragmentary, there is sufficient evidence to show that PPP's benefits are significant to both individual participants and to society in general. A list of sources is attached.
Grimble, Robin and Kate Wellard, 1997, "Stakeholder Methodologies in Natural Resource Management: A Review of Principles, Contexts, Experiences and Opportunities", in Agricultural Systems, Vol. 55, No. 2, pp. 173 - 193.
This paper reviews the underlying concepts and methods of stakeholder analysis and the links between economic efficiency, equity and environmental concerns. It discusses the particular relevance of stakeholder analysis to natural resource management and environmental concerns and the essential notions of conflict and trade-offs. It then reviews parallel developments in social science theory and practice and considers differing approaches in applying stakeholder analysis in natural resource management. Finally, it discusses the need for new analytical tools in natural resource policy and management, and assesses the role that stakeholder analysis can play in the process.
Hentschel, Jesko, 1994, Does Participation Cost the Bank More? Emerging Evidence, Human Resources Development and Operations Policy Working Papers 31, Washington D.C.: World Bank, 21 pp.
According to advocates of participation, integrating beneficiaries and stakeholders in preparing and implementing World Bank-supported projects can improve project performance and sustainability. However, participation in investment operations has cost implications for the World Bank. This paper reports on the results of interviews with World Bank staff of 21 participatory operations. It concludes that the cost category most affected is salaries of the World Bank staff and consultants in preparing and supervising participatory projects. Cost increases in this category range from 10 to 15 percent. Using the Management Information System, project statistics of an enlarged group of participatory projects are then compared to a Bank-wide control group. It also emerges that costs are partly shifted to trust funds and other funding sources. Overall, participation is not cost-free for the World Bank, but it does not cause insurmountable cost increases. A list of references is attached.
Hinchcliffe, Fiona, Irene Guijt, Jules N. Pretty and Parmesh Shah, 1995, New Horizons: The Economic, Social and Environmental Impacts of Participatory Watershed Development, Gatekeeper Series No. 50, London: International Institute for Environment and Development (IIED): London, 22 pp.
This paper summarizes the findings of case studies of the processes and impacts of 22 participatory watershed development projects. The study was a collaborative effort coordinated by the Sustainable Agricultural Programme of IIED and partner institutions in Asia, Africa, Latin America and Australia from 1992 to 1994. After a brief review of the technical, social and economic failure of many efforts in soil and water conservation, the paper presents the basic principles of participatory watershed development programmes. It gives a short description of the process of the impact analysis, the indicators selected and the key findings of the case studies. Finally, it lists the implications for national and international institutions that emerged from a workshop of the case study team members. The list of references includes papers presented at the workshop.
Huizer, Gerrit, 1983, People's Participation Projects: Guiding Principles, Rome: FAO, 100 pp.
Developed for national project coordinators, group promoters, action researchers and farmers in the People's Participation Programme (PPP), these guidelines emphasize the need for monitoring and ongoing evaluation of PPP. The areas covered are: the set-up of people's participation projects, the selection of project areas, the recruitment and training of group promoters and the identification of participants and group formation. One section is devoted to the tools of monitoring and ongoing evaluation.
Institute of Development Studies (IDS), 1998, Participatory Monitoring and Evaluation: Learning from Change, IDS Policy Briefing Paper Issue 12, Brighton: IDS, 9 pp.
This paper presents a summary of some key issues surrounding participatory monitoring and evaluation (PM&E), including the reasons for the growing interest in PM&E, practical applications and implications for development agencies, including the need to assess the quality of participation, and some of the costs in terms of time and human resources.
Inter-American Development Bank (IDB), no date, Resource Book on Participation, Washington D.C.: IDB, 178 pp.
This resource book is intended for Inter-American Bank personnel who seek to facilitate the involvement of non-traditional partners in the design and execution of Bank-financed projects. It is a reference tool designed to foster a deeper understanding of what participation in development means. It is divided into eight sections:
Kottak, Conrad Phillip, 1991, "When People Don't Come First: Some Sociological Lesson From Completed Projects", in Cernea, Michael M. (ed.), 1991, Putting People First: Sociological Variables in Rural Development, New York: Oxford University Press, pp. 421 - 464.
Drawing on an analysis of ex post evaluation findings and comparative studies of project reports by the World Bank and the U.S. Agency for International Development (USAID), this paper affirms the need for informed social strategies for economic development. The study shows that proper social designs for implementation and sensitivity to social issues have financial benefits as assessed, for example, by such measures as economic rate of return and long-term sustainability of benefit flows. It also points out the need for better collection and use of socio-cultural background information and for a socially sensitive perspective in evaluation. Finally, it stresses the need to tap the information pool of local participants in carrying out evaluations.
Marsden, David and Peter Oakley (eds.), 1990, Evaluating Social Development Projects, Oxford: Oxfam, 162 pp.
Bringing together the papers of an international conference, this book explores questions of how to evaluate social development projects that involve the promotion of participation, the awakening of consciousness, the encouragement of self-reliant strategies and the development of indigenous sustainable capacity. It covers four main themes:
Marsden, David, Peter Oakley and Brian Pratt, 1994, Measuring the Process: Guidelines for Evaluating Social Development, Oxford: INTRAC, 178 pp.
Based on an international workshop of both practitioners and academics, this book sets out guidelines for evaluating social development processes. It combines a theoretical overview of the concepts involved with insights into planning and implementation of evaluation. Three case studies of evaluations are provided from Colombia, India and Zimbabwe.
Martin, Adrienne and John Sherington, 1997, "Participatory Research Methods - Implementation, Effectiveness and Institutional Context", in Agricultural Systems, Vol. 55, No. 2, pp. 195-216.
This paper outlines some of the main issues and debates in participatory research and participatory technology development, concentrating on implementation. Arguments in favour of farmer participatory research are noted, as are a number of difficulties, including that of assessing the effectiveness of participatory methods. It also notes that the resources needed for collaborative participatory work are often underestimated and that methodologies for monitoring and evaluation have been particularly weak.
McAllister, Karen, 1999, Understanding Participation: Monitoring and Evaluating Process, Outputs and Outcomes, Ottawa: IDRC, 55 pp.
This paper examines the challenges and proposes an approach for monitoring and evaluating participatory research for community-based natural resource management projects. Intended for researchers and programme and project managers, it focuses on using monitoring and evaluation as a tool for adaptive learning and project improvement, for integrating social theory into participatory methods, and for understanding the links between participatory processes and outcomes. The paper presents a framework for monitoring and evaluating participatory research for use by donor agencies. The paper contains an extensive list of references.
McGee, Rosemary and Andy Norton, 2000, Participation in Poverty Reduction Strategies: A Synthesis of Experience with Participatory Approaches to Policy Design, Implementation and Monitoring, IDS Working Paper 109, Brighton: IDS, 84 pp.
This paper reviews experiences in applying participatory approaches to macro-level policy formulation, implementation and monitoring, with a view to supporting country-led facilitation of inclusive and high-quality participation in the Poverty Reduction Strategy process. Directed to governments, civil society organizations and bilateral and multilateral donor agencies, the document outlines the challenges in establishing participatory, sustainable and country-owned poverty reduction strategies. Some of these are also relevant to participatory projects. One section of the paper examines the need to ensure quality participation and standards and benchmarks for assessing the process and outputs of participation. An extensive list of references is attached.
Oakley, Peter and David Marsden, 1984, Approaches to Participation in Development, Geneva: International Labour Organisation (ILO), 91 pp.
This monograph explores different concepts of participation in rural development. The authors distinguish two broad interpretations:
After reviewing different experiences of participation, the authors outline some of the elements of a strategy for promoting people's participation in rural development.
Oakley, Peter, 1988, The Monitoring and Evaluation of Participation in Rural Development, Rome: FAO, 55 pp.
This paper presents a framework for monitoring and evaluation (M&E) of the participation of the rural poor in development projects. It gives an overview of the concept of people's participation and how participation has been incorporated into projects. After reviewing the difficulties of monitoring and evaluating participation using conventional techniques, it proposes an alternative M&E approach based on understanding qualitative processes. The paper presents guidelines for developing both quantitative and qualitative indicators of participation and for operationalizing the M&E of participation at the project level. Attention is also given to the interpretation of data and information collected at the project level. The paper contains a list of indicators of people's participation and an extensive list of references.
Oakley, Peter et al, 1991, Projects with People: The Practice of Participation in Rural Development, Geneva: International Labour Organisation (ILO), 284 pp.
Looking at a wide range of attempts to promote the participation of people within rural development projects, the study highlights the methodological approaches applied in promoting participation within a variety of sectoral, institutional and policy settings. Elements of a strategy and of a methodology for promoting people's participation are drawn from these case studies. The study also gives an overview of the concepts and challenges of participation and the principles of participatory practice. One section of the study is devoted to evaluating participation, including the appropriate forms of evaluation, indicators, information and data collection and its interpretation. An extensive bibliography is attached.
Rudqvist, Anders, 1992, "The Swedish International Development Authority: Experience with Popular Participation" in Bhatnagar, Bhuvan and Aubrey C. Williams (eds.), 1992, Participatory Development and the World Bank: Potential Directions for Change, World Bank Discussion Paper 183, Washington D.C.: The World Bank, pp. 46 - 58.
After presenting the development policies of the Swedish International Development Authority (SIDA), this paper discusses participatory experiences in SIDA-financed programmes. It reports on the response of selected SIDA officials to the World Bank survey on the capacity to support participation, including time and staff requirements.
Rudqvist, Anders and Prudence Woodford-Berger, 1996, Evaluation and Participation -Some Lessons, SIDA Studies in Evaluation 96/1, Stockholm: SIDA, 65 pp.
This paper reviews and synthesizes the experiences of eight bilateral donor agencies with regard to support to participation / participatory development in policy work and through programme and project funding. It also draws on other sources of information about participation and evaluation. The focus is on two main sets of issues:
The review shows that participation as operational practice has lagged far behind donor awareness, advocacy, policy declarations and general development rhetoric. Therefore, evaluations explicitly addressing participation are only now beginning to emerge. Evaluating participation requires methodological adjustments and the paper explores some of these, including the need for qualitative indicators. The paper also summarizes the World Bank findings on costs and benefits of participation. Finally, it draws some general conclusions on evaluating participation.
Skutsch, Margaret M., 1998, "First Analyse Your Conflict", in Proceedings of a Satellite Meeting to the XI World Forestry Congress, Forest, Trees and People Programme, Conflict Management Series, pp. 261-305, Rome: FAO.
In this paper, consideration is first given to how natural resource professionals and foresters in the field commonly understand the term "participation", and why the concept of "conflict" does not seems to overlap with notions of participation. The paper then presents a number of methods for analysing conflict. It asserts the importance of focusing first on conflict analysis, rather than on conflict management or resolution, as a pragmatic method to deal with natural resource-based conflict. Finally, the paper considers how conflict analysis could be introduced much more widely in forestry and natural resource management training, and how it could enter the normal procedure of project and programme planning.
Uphoff, Norman, 1992, "Monitoring and Evaluating Popular Participation in World Bank-Assisted Projects" in Bhatnagar, Bhuvan and Aubrey C. Williams (eds.), Participatory Development and the World Bank: Potential Directions for Change, World Bank Discussion Paper 183, Washington D.C.: The World Bank, pp. 135 - 153.
This paper considers methodologies for monitoring and evaluation (M&E) of participation that can contribute to longer-term participatory operation and maintenance. Borrowing two categories from anthropology, the paper considers "etic" and "emic" approaches to M&E of participation. Etic approaches are external ones and include standard monitoring and evaluation techniques with special attention to participation; participant observation and beneficiary assessment; rapid rural appraisal / participatory rural appraisal; and focus groups. Emic approaches are internal ones and include local planning, monitoring and evaluation; participatory self-evaluation; visitation and peer training; and monitoring and evaluation across levels. The paper also discusses the basic concepts of popular participation, the costs and benefits of participation, and some pitfalls in evaluating participation.
Uphoff, Norman, 1991, "Fitting Projects to People", in Cernea, Michael M. (ed.), 1991, Putting People First: Sociological Variables in Rural Development, New York: Oxford University Press, pp. 467 - 511.
This paper considers some project experiences and how development expenditures are likely to be more worthwhile to the extent that projects are planned in ways that involve the intended beneficiaries in decisionmaking, implementation, evaluation and benefits. Projects planned with little input from the intended beneficiaries were beset with problems and insufficient results deriving from the top-down approaches taken. When the projects were modified to become more open to people's participation, progress became more satisfactory. The paper presents some conclusions on what people can contribute to the planning and implementation of projects, and how projects can be better conceived and carried out to put people first.
Uphoff, Norman, 1989, Participatory Self-Evaluation of P.P.P. Group and Inter-Group Association Performance: a Field Methodology, Rome: FAO, 47 pp.
Written for the FAO People's Participation Programme (PPP), this paper proposes a methodology for members of groups and associations to evaluate their own performance. The goal is to place a flexible and practical evaluation methodology in the hands of PPP members and group promoters. The paper gives examples of how the methodology works, discusses the potential benefits, outlines a process for introducing the system and discusses various problems that may arise. The paper contains an extensive list of questions for participatory self-evaluation of the extent and quality of participation and of group performance.
Warren, Patrizio, 1998, Developing Participatory and Integrated Watershed Management, Community Forestry Case Study Series 13, Rome: FAO, 158 pp.
This case study of the FAO/Italy Inter-regional Project for Participatory Upland Conservation and Development (PUCD) presents the three major phases of the project:
Some of the major conclusions are:
World Bank, 1998, Assessing Aid: What Works. What Doesn't, and Why, New York: Oxford University Press, 150 pp.
This book looks at macro-level issues of the provision of financial aid to developing countries. It identifies two major needs for aid to be successful: a good policy environment and a good institutional environment. It also looks at how aid can contribute to promoting both good policy and good institutional environments. Mention is made of the importance of a participatory approach to service delivery in projects.
World Bank, 1996, The World Bank Participation Sourcebook, Washington D.C.: The World Bank, 259 pp.
This Sourcebook is intended to be an easy reference for people working with in participatory development projects and programmes. Chapter I, "Reflections on Participation", explores the meaning of participatory development and participatory processes in planning and implementation of World Bank-supported operations. Distinguishing between popular and stakeholder participation, it deals with the importance of stakeholder analysis. It also discusses the importance of using participatory approaches to reach the poor. Chapter II, "Shared Experiences", contains examples of participatory approaches in Bank-supported operations. Chapter III, "Practice Pointers in Participatory Planning and Decisionmaking" takes the reader through the various steps of participatory planning and decisionmaking. Chapter IV, "Practice Pointers in Enabling the Poor to Participate" looks at common barriers, strengthening the financial and organizational capacity of the poor, and creating an enabling environment for participation. The Sourcebook also contains a section on Methods and Tools, and summaries of working papers on sectoral and cross-cutting issues.