3. DATA CONTENT AND FORMAT

In this section, I analyze the data that is being collected through personal interviews according to the reporting format in Guatemala. I start out at identifying which interview variables are most appropriate for measuring expected NFA output concepts such as what products and services benefit what users, perceptions in trends of supply and demand for different products, etc. I then discuss whether an analysis of these variables would be adequate for policy makers in the national and international policy arenas, or what possible improvements are conceivable in terms of what is measured by the NFA interview component.

I find that although the NFA has some remaining issues to resolve, the information that the NFA makes available is of potentially great value both to FAO and to national policy makers, and it is much richer and more complete than the current forest assessments of most FAO member countries, whether developing countries or not. Ultimately, the usefulness of this data is related to how the FRA end users perceive its overall quality. The issue of data quality is critical for the future success of the NFA, and it is addressed in Sections 4, 5, and 6.

According to the Global FRA Advisory Committee, there are about twenty different variables that member countries will be asked to report on in the next global FRA in 2005. The FAO-NFA includes all of these variables as core measurements in each country. In addition, country governments that participate in the FAO-NFA process may add variables that are of particular interest to the country's policy process. In the case of Guatemala, for example, the Government's Forest Service wants to assess the physical availability and the harvesting practices of six different non-timber forest products. It is also a pronounced goal of the FAO-NFA to assess the role of trees and forests in sustaining rural people's livelihood, including their food security (Saket et al, 2002). The interview component of the NFA should be able to provide national level measures on several of these parameters. Table 1 provides the summary list of the expected outputs of the FAO-NFA, which include the 20 indicators to be included in the GFRA 2005. Highlighted are the parameters for which NFA interview information will be a critical input.

Table 1: Attributes and outputs of the FAO-NFA

The highlighted NFA outputs are the most relevant to the variables that are measured through field interviews with forest users. The first part of the analysis considers how the interview variables might contribute to produce these outputs.

3.1. The Contribution of Interview Variables to NFA Outputs

This section examines the scope and limitations of the NFA interview variables. The interviews with forest users have the potential to provide very valuable information that would be very difficult, if not impossible, to acquire through direct observations of the forest resources. For instance, what are the most important tree and forest products for smallholders in rural Guatemala? Do these smallholders perceive the supply of these products to be rising or declining? To know the answers to these questions can be critical for designing appropriate forestry policy responses. Yet the ultimate value of the interview information is determined in part by how well the estimated variables fit the target concepts, or outputs that the NFA aims to measure. First, I try to link interview variables to NFA outputs and second I explore the potential limitations of such linkages and suggest possible remedies.

The analysis in Table 2 highlights some common features of interview variables that may complicate the analysis at the national level. Among these, we note that

• Most of the variables on products and services are likely to be affected by the "small number"- problem, that is, the incidence of some products and services is so low that the total number of observations is not sufficient to reach statistically significant results for national estimates. This is particularly true for estimates of NWFPs and endangered species. If one considers such information important, however, one might consider carrying out a complementary study in purposely-selected sites.
  
• A similar limitation deals with the fact that the results of the interviews are representative only at the national level and for that reason it is not always possible to differentiate findings according to sub-national regions. This is likely one of the most critical limitations for the viability of the NFA for national policy. Effective policymaking requires more fine-scaled, sub-national information as the basis for public investment decisions (spend where it will have most effect) so as to optimize outcomes. However, complementary studies that detail the necessary information for policy makers may be carried out as a follow-up, if deemed necessary.
  
• Many of the interview variables are qualitative measures that do not quantify the uses and benefits to the users. While such qualitative measures are helpful for cross-temporal comparison, it is hard to use such data to make inferences for the population at large. For instance, what does it mean that 38 percent of the forest users in Guatemala perceive a slight decline in the supply NWFPs? It may be possible to estimate the number of users and direct beneficiaries from products and services through interviews, but this would require a modified field protocol, something that is further elaborated on in Chapter 6.
  
• There is an opportunity to simplify the reporting format. For instance, eliminating the distinction between children and adult users, and using access to health clinics as a proxy measure for infrastructure access is likely to improve reliability.

Table 2: Links Between NFA Outputs and Interview Variables

If the expected NFA outputs in the table represent the minimum requirement for what information is relevant and there is a desire to simplify the reporting format as much as possible, it would make sense to consider whether there are any variables that are currently measured but are not used as proxy measures for NFA output concepts. Unless, national governments insist to measure all listed variables it seems like the following are currently not used as primary sources of information:

• Season extracted
• Last extracted
• Future trees expected
• Future trees desired

If these would be superfluous, are there any variables that are missing? Obviously, this depends on the information needs of each member country. Judging from the 20 core variables that the FRA advisory group has recommended be included in the national report for the next global FRA in 2005, there are at least two variables that are missing:

• Employment through primary activities in forests. In the current NFA format, information is not gathered about the value of formal and informal employment related to productive, protective and recreational functions in the sector. If such measures are to reflect the contribution of the informal (not necessarily illegal) as well as formal forestry sector activities, interviews can be used to solicit information about how much of the interviewed users' total family income is related to forestry activities.
• Value of NWFPs is not specified at the national level in the current NFA format. This can be a complicated issue for many products, but it should be possible to estimate the value of the products that are harvested for commercial purposes. However, such estimates require quantifications of amounts harvested from the sampled areas. It should be possible to include such information in future interview formats.

3.2. Usefulness of Interview Variables for Policy Makers

One of the rationales of the NFA process is to provide reliable and accurate information to policy makers about the changing conditions of a country's tree and forest resources and these resources' contribution to human development. Is the NFA delivering its promise?

The FAO-NFA team has decided to limit the NFA outputs to national level relevance at this point. A major objective of the NFA is to hold down the operational costs so as to increase the likelihood of more countries deciding to carry out the inventories. A national inventory that could deliver high-resolution information about regionally differential parameters would cost significantly more than the current NFA budgets allow for. Although policy makers at the national level often require sub-national level information to make specific policy decisions, it seems wise for the NFA to leave such fine-scaled studies up to each national government to undertake.

Table 3: Current Interview Reporting Format

This is not to say that the end products of the NFA information that is currently gathered would not be useful for national policy making - on the contrary. Getting relevant national level data is a necessary first step in the national policy making process. Many problems in the forestry sector never get the attention they deserve precisely because the severity of the problem at the national level is unknown. Without national level evidence about the importance of some specific resource problems, such as a possible undersupply of certain NWFPs or proportion of forest users without legal user rights, the problems may not appear as relevant to policymakers. No political action is likely unless politicians are convinced that the problem is real and relevant. In that sense, the NFA analysis can be an extremely powerful tool to raise awareness about the important role of forestry in the social and economic development of any nation. The next section explores some ideas on how the content and format of the current NFA approach could be modified so as to increase the policy relevance of the NFA products.

3.2.1. Ideas on How to Make the NFA Format More Policy Relevant

Public policy making is the process that defines "who gets what when and how" (Lasswell, 1952). The forestry policy process defines the formal rules that govern citizens' access to and use of tree and forest resources. Since the careers of politicians and policy makers depend in part on how they are able to meet the public's demands for public policy interventions in the forestry sector, they need constant feedback on whether the interventions are effective. The NFA has the potential to provide such feedback to policy makers, but the format used for gathering information limits the usefulness of information for public policy.

The NFA Product and Services Form, in Table 3 above, which is currently used to systematize interview information about local forest use, is structured according to products and services; not categories of users. The thrust of the current interview data format seems not to be "who gets what, where and how" but rather "what is gotten when, how and by whom". This implies that in sites with multiple users groups harvesting the same forest product or service, information about specific user groups may be lost. Because of the public policy relevance of user-specific information, it would make sense to restructure the current interview format so that this information is gathered in a way that makes it possible to produce national aggregates of the main beneficiary groups of different kinds of forest use. The way to do this may not be as burdensome as it may sound. Once the categories of users have been defined, representatives from each of these groups are interviewed.

One possible way of defining user groups is to group them according to their legal rights to harvest the product or service in question. The suggested format in Table 4 below offers one possible typology for such a classification, but each country needs to agree on how to classify user groups, whether they want to differentiate according to legal rights, ethnic groups, gender, age, or perhaps family income. Once the user group has been defined, information about each group's use of products and services in a particular land use section is recorded. according to the suggested new format in Table 4. Drawing on interview information, field personnel are asked to estimate the number of harvesting households and the quantities of the harvested products and services from each land use section (LUS). New in this suggested format is also the elimination of the categories that seem superfluous to the goals of the FRA 2005, such as the as the date of the last extraction, future trees expected and future trees desired.

Table 4: Suggested Modified Format of Coding Interview Information

Despite the noted limitations of parts of the NFA data, the fact that national assessments are underway in several developing countries should be recognized as a major step forward for forestry policy making both at the national and international levels. Thanks to this FAO-FRA initiative, more countries will have much more information about their tree and forest resources than they currently possess. The ultimate success of the NFAs, however, is likely to depend on the ability of the FAO-NFA team to make sure that the results meet a high standard of quality. Regardless of the final composition of the variables discussed in this section, there is a need to carry out quality control regarding the measurement and analysis of these variables. Defining data quality as the degree to which measurements and analysis are reliable and valid, I illustrate a method for how actions of quality control can be carried out by FAO-NFA.

4 A qualitative measure of whether the current supply for a certain NWFP is either "high, significant, insignificant, negligible, or unknown." When the measurement is repeated overtime, a trend in supply may become apparent.