Michael J. Spilsbury and
David Kaimowitz are with the Center
for International Forestry Research
in Jakarta, Indonesia.
An analysis based on a survey of forestry experts
and a theoretical review suggests
that research influences policy
in an indirect way.
Funding agencies expect policy researchers to show that their ef-forts have a measurable impact. This poses a considerable challenge. It was traditionally thought that research influences policy directly and that specific policies can be traced back to particular research findings. However, many analysts have now come to regard the link between research and policy as more diffuse: research induces changes in "conventional wisdom" (the set of dominant paradigms at a given moment regarding the desired ends of policy and the means of achieving them) and "policy narratives" (simplifying assumptions about the problem to be addressed and the approach to be taken), which in turn influence policy outcomes.
Given the bewildering array of factors that influence policy-makers' decisions, it would be naïve to overestimate the role of knowledge acquisition in that process. A role exists nonetheless.
This article combines a theoretical discussion of how policy-makers utilize research with a pragmatic attempt to find out what research has been influential. Between December 1997 and March 1998, the authors asked forest policy experts by e-mail which publications influenced international and national debates on policies that affect forests. The survey elicited 162 replies. The article analyses those responses in the context of the broader debate on the link between research and policy.
"Dost thou not know, my son, with how little wisdom the world is governed"
Count Oxenstierna, letter to his son, 1648
In J.F. Lundblad, Svensk plutarik (1826)
Textbook accounts of the policy-making process have traditionally portrayed it as rational, sequential and functionally differentiated. It is rational in the sense that policy-makers promote policies that best meet a set of predefined objectives; sequential because policy-makers first identify problems, then assess alternatives, make decisions, implement those decisions, evaluate the results and modify their policies; and functionally differentiated because each activity is separate and clearly distinguishable (Nakamura, 1987).
Real life is less linear and far more iterative. Policy-makers have ambiguous and shifting goals and often decide what positions to take based on extraneous issues. Participants drift in and out of the policy process. Solutions may be proposed before problems are identified. The same issues are dealt with simultaneously in different policy arenas. Politicians respond to conflicting pressures with muddled compromises (Cohen, March and Olsen, 1972).
Thus, it is hardly surprising that research often affects policy through circuitous and diffuse paths. Weiss (1977) argues that policy-makers use research more to help them define problems, think about issues and provide new perspectives than to solve specific problems. Research findings are just one of policy-makers' many sources of information.
Policy-makers use research not only as an input into decision-making, but also as a political tool to justify decisions made for non-scientific reasons (Boehmer-Christiansen, 1995). They may use research to "further their own interests, delay decisions, mark and occupy turf, or to enhance organizational credibility" (Garrett and Islam, 1988).
"Wisdom denotes pursuing the best ends by the best means"
Francis Hutcheson, Enquiry into
the origin of our ideas of Beauty and Virtue,
Treatise 1, 1725
Instead of leading directly to new policies, successful research more often influences policy by modifying conventional wisdom and policy narratives. Not all conventional wisdom constitutes policy narratives, but policy narratives form a central component of conventional wisdom.
Kuhn (1962) fundamentally changed scientists' thinking about how science and, by extension, conventional wisdom and policy narratives evolve. Science was previously viewed as an entirely incremental process through which humanity accumulated knowledge, with each new scientific finding building on those that preceded it. Kuhn emphasized that science sometimes advances through incremental learning but that, on other occasions, the basic tenets of conventional wisdom undergo fundamental changes. These "paradigm shifts" often only become apparent in retrospect, when it becomes evident that certain key events and findings have changed the way people think about a topic. Scientists begin to see problems in a new light and adopt a new world view or paradigm. Policy changes can be conceptualized similarly.
Effective myth-busting or paradigm-shifting research successfully questions existing policy narratives. In the case of forestry policy, the tendency of international organizations and funding agencies to follow fads in policy seems to favour such paradigm shifts. Even research that does not significantly depart from conventional wisdom may be influential, however, if it incrementally alters the opinions and actions of powerful organizations and individuals.
Since the impact of research per se is abstract and difficult to measure, a survey was carried out to explore the influence of publications, which are loosely interpreted as a reflection of research. The survey, initiated in December 1997, sought to determine which publications forestry policy experts have considered influential in international and national debates regarding forest policies. The survey was distributed through the Forest Policy Experts (POLEX) electronic mailing list, managed by the Center for International Forestry Research (CIFOR). The POLEX list consists of individuals considered to be opinion leaders in forest policy issues. Recipients were asked to list:
The role of publications in influencing debates was stressed rather than their role in modifying specific policies because the intention was to identify publications that had an impact on conventional wisdom and policy narratives in a broad sense.
The 162 respondents represented 28.6 percent of the population sampled. About one third were from Europe, one third from North America, 15 percent from Latin America and the remainder from Asia or Africa or of undetermined origin.
The results have an inherent bias because international organizations, universities in developed countries, environmental non-governmental organizations (NGOs) and donor agencies are better represented on the POLEX list than developing country policy-makers or researchers. They also have a "supply side" bias since respondents tended to be researchers and forestry advisers rather than policy-makers: 37 percent of all responses came from universities and research centres in developed countries. North American and European environmental NGOs accounted for 10 percent and bilateral funding agencies 8 percent. FAO and multilateral bank staff each contributed 7 percent of the responses and CIFOR scientists provided 10 percent. Personnel from developing country universities, governments and projects only made up 10 percent of respondents (see Figure 1).
FIGURE 1: Affiliation of survey respondents
Respondents did not cite exactly six publications each; many cited fewer, and a few individuals cited as many as 15 or 19. The opinions of respondents who cited more publications weigh more heavily in the results than those who cited only a few. This factor probably did not affect the general nature of the results, but it may have affected whether or not certain publications were included in the final list of influential documents. In total, respondents mentioned 370 publications as being influential at either the international or national levels.
The most influential publications were mostly semi-popular books, general articles in prestigious non-disciplinary journals and institutional documents. Among the 64 documents cited most frequently, 31 percent were commercial books, 17 percent came out in academic journals (Science [five articles], World Development, Nature, BioScience, Scientific American, Ambio and Forest Ecology and Management), 13 percent came from the World Bank, 11 percent came from the World Resources Institute (WRI), 9 percent were FAO documents and 8 percent were documents from United Nations conferences (see Figure 2). The Table on p. 7 lists the 31 most-mentioned publications.
FIGURE 2: Frequency of citation, by document type, for documents cited three or more times
With regard to publications influencing national debates, each country had a separate list of publications; the general results were not that specific documents were mentioned many times, but rather that certain types of documents were perceived as influential in multiple countries. These included World Bank sector reports, Tropical Forest Action Plans and reports by government commissions.
The responses make it clear that three institutions have dominated the debates regarding policies affecting forests over the past 30 years: FAO, the World Bank and WRI (see Figure 3). Approximately one third of all respondents mentioned at least one document associated with FAO or the World Bank. An even higher number (64 respondents) mentioned at least one publication associated with WRI. In the case of WRI, the results were greatly affected by the large number of people (34) who mentioned Repetto and Gillis' book, Public policy and the misuse of forest resources (1988). Even so, 34 respondents (21 percent) mentioned at least one WRI-related publication besides that book.
FIGURE 3: Publications cited two or more times, aggregated by institution
The major role of the World Bank and FAO in defining conventional wisdom can be explained in part by the critical mass of intellectual resources these institutions command and in part by the fact that they can promote their ideas by funding initiatives supported by their perspectives.
The survey results showed a troublesome dominance of authors from the United States and Europe. Of the 39 authors and co-authors of documents cited by five respondents or more, not one was from Africa, Asia or Latin America, even though the majority of the literature mentioned focused on tropical forests. The results may partially reflect the low representation from those continents among the individuals sampled, but that is probably only part of the story.
The survey responses provide little evidence that the documents that respondents considered influential directly affected policies. In most instances, it is likely that the publications have influenced general conventional wisdom and policy narratives in international policy, academic and funding circles, and that the influence has eventually filtered down to policy-makers in specific countries.
Nevertheless, at the national level, many respondents did claim that World Bank and government reports and Tropical Forest Action Plans directly influenced policies. This is logical since these documents are associated with groups directly involved in bringing about policy change. It was probably not the documents per se that had the impact, but rather the processes leading up to the documents or following their creation, which resulted in agreements on what needed to be done.
Survey respondents also mentioned that some issues became prominent as a result of the actions of popular movements, specific events or the efforts of journalists. Research on these issues sometimes reflects "jumping on the bandwagon" to gain research funding and command policy-makers' attention.
More generally, the publications identified in the survey tend to follow broader social and academic trends with respect to topics and approach. Thus they tend to favour less government intervention, greater attention to environmental services and improved access of disadvantaged groups to natural resources. To what extent research actually forms public opinion on these issues or merely mirrors shifts in conventional wisdom caused by other forces remains uncertain.
Nevertheless, the survey responses suggest that the main policy narratives related to forests have tended to become associated with a handful of publications that crystallized public interest in a topic or gave greater legitimacy to a particular policy perspective. Thus respondents tended to associate:
It is not possible from the survey results to make a distinction between issues and arguments that become prominent because of certain publications and those that gained momentum for other reasons but later became associated in experts' minds with a given set of publications.
Many respondents stressed that the fact that a publication was influential did not necessarily imply that it was good. Several commented that certain influential pieces were much weaker than others available on the same topic. The influential pieces were apparently marketed better, i.e. given a journalistic treatment, placed in widely read outlets, promoted by public figures or movements or connected with prominent institutions or policy change processes.
A few respondents even claimed that influential publications often gave incorrect or misleading messages, either by oversimplifying issues and exaggerating threats and opportunities to reach a wider audience or by bending the facts to support their particular agendas. Seven respondents mentioned the article by Peters, Gentry and Mendelsohn on NTFPs in Peru as an illustration of this defect. While they acknowledged that the article put NTFPs "on the map", they criticized its methods and conclusions. This example points to the fact that work that is later criticized or discredited can nevertheless be extremely influential in raising issues, shifting scientific debate and shaping policy outcomes. Presumably, when policy narratives are misguided, overstated or incorrect, flawed development policies and practices follow.
Conventional wisdom takes time to become established and popularized. Once a policy narrative becomes accepted, however, it is likely to exert influence for some time (even when later research questions its validity), as continued citation of earlier publications reinforces the narrative and individuals and institutions develop vested interests in its maintenance.
Documents cited as being influential by five or more survey respondents | ||
Authors and year |
Document |
Number of citations |
Repetto, R. & Gillis, M. (1988) |
Public policy and the misuse of forest resources. New York, Cambridge University Press |
34 |
Peters, C.M., Gentry, A.H. & Mendelsohn, R.O. (1989) |
Valuation of an Amazonian rainforest. Nature, 339(29): 655-656 |
22 |
Poore, D., Burgess, P., Palmer, J., Rietbergen, S. & Synnott, T. (1989) |
No timber without trees: sustainability in the tropical forests. London, Earthscan Publications |
22 |
UN Conference on Environment and Development (UNCED) (1992) |
Agenda 21: Programme of action for sustainable development. New York, UN |
17 |
FAO (1985) |
Tropical Forest Action Plan (TFAP): a call for action. Rome |
15 |
World Commission on Environment and Development (Brundtland Commission) (1987) |
Our common future. Oxford, UK, Oxford University Press |
13 |
UNCED (1992) |
Non-legally binding authoritative statement of principles for a global consensus on the management of all types of forests (the "Forest Principles"). New York, UN |
11 |
Westoby, J. (1978, 1987) |
World Forestry Congress presentation, The purpose of forests (1978); The purpose of forests - follies of development. Oxford, UK, Blackwell (1987) |
11 |
Johnson, N. & Cabarle, B. (1993) |
Surviving the cut. Sustainable forest management in the humid tropics. Washington, DC, World Resources Institute (WRI) |
9 |
Anderson, A.B., ed. (1990) |
Alternatives to deforestation: steps towards sustainable uses of the Amazon rain forest. New York, Oxford University Press |
8 |
Binswanger, H.P. (1989) |
Brazilian policies that encourage deforestation in the Amazon. Environment Department Working Paper No. 16. Washington, DC, World Bank |
8 |
Mahar, D.J. (1989) |
Government policies and deforestation in Brazil's Amazon region. Technical Report. Washington, DC, World Bank |
8 |
UNCED (1992) |
Convention on Biological Diversity. New York, UN |
8 |
World Bank (1991) |
The forest sector: a World Bank Policy Paper. Washington, DC |
8 |
Repetto, R. (1988) |
The forest for the trees? government policies and the misuse of forest resources. Washington, DC, WRI |
7 |
Eckholm, E. (1975) |
The other energy crisis: firewood. World Watch Report No. 1. Washington, DC |
6 |
FAO |
National Forestry Programmes/plans |
6 |
FAO (1982) |
Tropical forest resources. FAO Forestry Paper No. 30. By J.-P. Lanly. Rome. |
6 |
Hardin, G. (1968) |
The tragedy of the commons. Science, 162: 1243-1248 |
6 |
Myers, N. (1984) |
The primary source: tropical forests and our future. New York, W.W. Norton |
6 |
World Bank (various years) |
National sector reports (general 2, Bolivia 1, Zimbabwe 1, Malaysia 1, Costa Rica 1) |
6 |
Wilson, E.O., ed. (1988) |
Biodiversity. Washington, DC, National Academy Press |
5 |
FAO (1978) |
Forestry for local community development. By J. Westoby. FAO Forestry Paper No. 7. Rome |
5 |
de Beer, J.H. & McDermott, M.J. (1989) |
The economic value of non timber forest products in South East Asia. Amsterdam, the Netherlands, Committee for IUCN |
5 |
Myers, N. (1981) |
The hamburger connection: how Central America's forests became North America's hamburgers. Ambio, 10: 3-8 |
5 |
Peluso, N. (1992) |
Rich forests, poor people: resource control and resistance in Java. Berkeley, California, USA, University of California Press |
5 |
Chambers, R. (1983) |
Rural development: putting the last first, New York, NY, USA, John Wiley and Sons |
5 |
Schneider, R.R. (1994) |
Government and the economy on the Amazonian frontier. Latin America and the Caribbean Technical Department, Regional Studies Program, Report No. 34. Washington, DC, World Bank |
5 |
Skole, D.L. & Tucker, D.J. (1993) |
Tropical deforestation and habitat fragmentation in the Amazon: satellite data from 1978 to 1988. Science, 260: 1905-1910 |
5 |
Vincent, J. (1992) |
The tropical timber trade and sustainable development. Science, 256: 1651-1655 |
5 |
WRI (1986) |
World resources. Washington, DC |
5 |
"It is the customary fate of new truths to begin as heresies and end as superstitions"
T.H. Huxley, "The coming of age of the Origin of species"
In Science and culture and other essays (1881)
A comparison of the chronology of influential publications cited in the survey with key events and social trends mentioned in histories of forest policy debates (Humphreys, 1996; Kolk, 1996; Shepherd et al., 1998) revealed suggestive links in the evolution of policy narratives related to forests.
For example, Eckholm's book The other energy crisis: firewood put the fuelwood crisis on the map in 1975. FAO picked this up in 1978 with its map of the fuelwood situation in developing countries. In 1980, FAO's assessment of global forest cover helped convince donors and others of the importance of tropical deforestation and the fuelwood crisis. Fuelwood remained a prominent issue until several critical publications in the late 1980s effectively removed it from the international debate.
Another illustration: from the late 1970s to the mid-1980s, several works by Myers, including "The hamburger connection: how Central America's forests became North America's hamburgers" (in Ambio, 1981) and The primary source (1984), highlighted the gravity of the tropical forest crisis and the importance of biodiversity. In the late 1980s, a flurry of publications and activities related to biodiversity, including a 1988 United States National Academy of Sciences book edited by Wilson, preceded the signing of the Convention on Biological Diversity in 1992.
Over the past 20 years, policies and projects in many countries - particularly the smaller and poorer countries that depend on foreign support - have reflected similar shifts in conventional wisdom. The spread of ideas provoked and mirrored changes in priorities and positions in the World Bank, FAO and the main bilateral aid agencies. These agencies, in turn, provided developing country policy-makers with new ideas and financial incentives to accept them. In addition, a relatively small cadre of consultants went from country to country broadcasting the conventional wisdom of the moment and designing and implementing projects based on it.
In most cases, there was probably a lag of several years between the "launching" of new policy narratives in well-publicized events or prominent publications and the filtering down of these ideas into changes in conventional wisdom, policy and funding.
Most publications cited as influential by three survey respondents or more were published between 1989 and 1994 (see Figure 4). This result undoubtedly reflects the fact that concern about tropical forests was high during that period, but it also suggests that it takes three to ten years for a publication to become acknowledged and disseminated widely enough to gain major prominence.
FIGURE 4: Frequency distribution of date of publication for the 64 most cited documents
It was difficult to identify publications that directly influenced policies towards forests solely by the force of their arguments. Although certain publications have been influential, specific policies cannot usually be attributed to them. Still, policy research does seem to enhance policy actors' awareness and to shape conventional wisdom.
Some documents were found to have directly influenced policies at the national level; however, it was probably not the documents per se that had the impact, but rather the processes accompanying their creation. Research that targets or associates itself with major policy processes or powerful organizations has a better chance of having an impact and being recognized.
Being "right" does not seem to be either a necessary or sufficient condition for having an impact. Some documents have been both influential and wrong. Work that is later criticized or discredited by scientific peers can nevertheless be extremely important in raising issues, shifting scientific debate and shaping policy outcomes.
Credibility is at least as important to the impact of policy research as "being right". Credibility seems to be closely linked to the reputations and track records of the authors, the prestige of the publishers and the influence of the organizations that sponsor the research and/or promote the findings. Unfortunately, the process by which credibility is acquired has given a rather small group of Northern policy analysts and a few large organizations an inordinate amount of influence, potentially stifling the effective input of analysts and institutions in developing countries.
Research that tells policy-makers and opinion leaders what they want to hear has a better chance of being influential than work that goes against the tide. Conventional wisdom and policy narratives can be successfully challenged and debunked, but this is easier when the prevailing political, social, economic and scientific winds are blowing in the same direction.
Policy researchers can increase their impact not only by providing good answers to the right questions, but also by supplying these messages to the right (most influential) people at the right time and in an appropriate format. The most influential researchers and institutions will be those who effectively build "coalitions" to support their viewpoints in the policy arena and succeed in associating their work with well-funded initiatives.
Impact-oriented researchers and institutions must pay attention, not only to the development of the "research product", but also to the "market" in which that product must compete. The findings from this survey highlight some important aspects of the research market. Researchers and institutions wishing to enhance their influence must constantly appraise the demand for their research products and identify opportunities for their work to gain prominence. Surveys such as the one presented here have an important role in that appraisal.
Bibliography
Boehmer-Christiansen, S.A. 1995. Britain and the International Panel on Climate Change: the impacts of scientific advice on global warming, Part I: integrated policy analysis and the global dimension. Environ. Pol., 4(1): 1-18.
Cohen, M.D., March, J.D. & Olsen, J.P. 1972. A garbage can model of organizational choice. Admin. Sci. Qtly, 17: 1-25.
Garrett, J.L. & Islam Y. 1988. Policy research and the policy process: do the twain ever meet. IIED Gatekeeper Series No. 74. London, IIED.
Humphreys, D. 1996. Forest politics: the evolution of international cooperation. London, Earthscan Publications.
Kolk, A. 1996. Forests in international environmental politics: international organisations, NGOs and the Brazilian Amazon. Utrecht, the Netherlands, International Books.
Kuhn, T.S. 1962. The structure of scientific revolutions. Chicago, Illinois, USA, University of Chicago Press.
Nakamura, R.T. 1987. The textbook policy process and implementation research. Policy Stud. Rev., 7(1): 142-154.
Shepherd, G., Brown, D., Richards, M. & Schreckenberg, K. 1998. EU forestry sourcebook. London, Overseas Develop-ment Institute.
Weiss, C. 1977. Research for policy's sake: the enlightenment function of social research. Policy Anal., 3: 531-545.
