Essentially all FAO member governments are engaged, with different levels of intensity, in promoting sustainable use of their tree and forest resources. Yet, most of these governments have limited information about the effects of their recent changes in public policy concerning forestry and rural development in general. Have there been any substantial changes in forest use patterns and resource conditions as a result of the new policies? The question is important for both policy makers and practitioners. Without an understanding of the effects of current and past policies, it is impossible to know how to adjust current policies in a way that benefits future outcomes. Without a continuous feedback of policy results to decision makers, the effect of the newly introduced policy instruments will not only remain unknown, but may even lead to unexpected and undesirable results (Ribot, 2002; Piciotto, 1998; Wiesner, 1993). At the international level, the effective implementation of the UN conventions on biological diversity, climate change and desertification "require[s] monitoring of forests including the production of goods and services" (Saket et al, 2002: p 2). To make sustainable forest use happen, national governments need monitoring programs that study the effects of natural resource management policy on resource users' decisions, but few non-industrial countries have such programs in place (Burki et. al., 1999; FAO, 2002; Saket et al, 2002; Andersson, 2002).
Having recognized the importance of monitoring forestry policy effects, FAO's Forest Resources Assessment (FRA) Program recently launched a series of initiatives to work together with interested member governments to support their systematic forest assessments. These initiatives are grouped into a program for supporting National Forest Assessments. This program aims to strengthen the participating countries' technical and institutional capacity in monitoring, and raise awareness for the inclusion of forest resources monitoring in the national policies. In doing so, the new FAO approach emphasizes the need to adapt the assessment program design to each country's technical, human, as well as financial resources. Unlike the traditional approaches to forest inventories, which tend to limit themselves to measurement of bio-physical characteristics of trees and forest resources, the new FAO program supports the development of analytical methods for documenting the stock and flow of wood products, non-wood forest products (NWFPs) and services, as well as human uses thereof. The human uses of goods and services derived from trees and forests are documented through systematic interviewing of forest users nationwide.
The purpose of this report is to provide technical advice to FAO with regards to the interview component of the national forest assessments. The advice offered here is based upon a 10-day visit to Guatemala's national forest assessment pilot project1, a literature review of sampling design and measurement issues in the social sciences well as interviews with a group of researchers specialized in human interactions with forests resources. To make the contribution of this report as down-to-earth and practical as possible, I will try to relate my observations and recommendations to the context of Guatemala's forest assessment experience.
The report is divided into six main sections. After the introduction and a brief background history of the NFA, I discuss the NFA data content and format; method assessment; field observations, and finally recommendations. In the Data Content section, I begin with an overview of the variables included in the interview component of the new FAO approach to National Forest Assessments. Using the data collection format used in Guatemala, I then carry out an analytical exercise to determine what type of analysis one can do with the data that are currently being collected. I relate the suggested variables for the Global Forest Resource Assessment 2005 to the measured variables in the pilot cases and assess to what extent the current approach meets the mark of FRA 2005. In the subsequent Methods section, I introduce and discuss three key aspects of all social science-related field measurements: reliability, validity, and uncertainty estimate. In the Field Observations section, I then apply these concepts as evaluative criteria when I examine the existing methods used for interviews in Guatemala. This leads to a series of observations about some of the main strengths and weaknesses in the current organization of the interview component. Finally, in the Recommendation section, I suggest possible ways of modifying the current approach that would allow the FAO to capitalize on the strengths of the program while addressing some of its most serious limitations.
In the report, I will make repeated use of several terms that may need further clarification. I have defined some of the most important ones below.
Forest resources |
Forest resources explicitly include forests, other wooded lands and trees outside of the forest, as defined by FAO. |
Forest Resources Assessment Program |
The FAO Program tasked to lead the implementation of the Global FRA, and to support National Forest Assessments |
Global Forest Resources Assessment |
The global process led by FAO that collects, analyzes, manages and makes available information about forest resources, their management and uses for all countries. |
Measurement |
The assigning of numbers to objects or events according to rules (Stevens, 1951) |
Measurement error |
Degree to which instruments yield data that are incorrect due to the measurement process (Litwin, 1995) |
National Forest Assessment |
A national process to collect, manage, make available and analyze information on forest resources, their management and use covering the entire country, including also analyses, evaluations and scenario development for use, e.g., in policy processes. |
National Forest Inventory |
The principal activity to collect data within a National Forest Assessment. An NFI may be based on different field sampling strategies that are either random or systematic and may or may not involve remote sensing components. |
Random error |
The degree to which instruments yield data that are incorrect not due to the measurement process, but due to uncontrollable fluctuations in responses (Litwin, 1995). |
Reliability |
The extent to which measurements are repeatable, either by the same individual using different measures or by another group of individuals using the same measures (Nunnally, 1967). |
Systematic error |
The degree to which instruments yield data that are incorrect due to a bias in the measurement procedure or research design |
Uncertainty estimate |
Defining the width of the confidence interval of a measurement or an inference of a relationship between variables. |
Validity |
To measure what we think we are measuring (King et al, 1994). A validity assessment examines how well an instrument is able to measure what it is intended to measure (Litwin, 1995). Hence, validity requires reliability (Zeller and Carmines, 1980). |
1 Guatemala is one of the three countries that are implementing a national forest assessment pilot project. The other two are Cameroon and The Philippines.