Chapter IV
Results and Findings (continued)

4.3 STATISTICAL RESULTS BY DEFINITION OF FOREST (mean estimates and associated error of forest cover and deforestation rates)

The results for the area surveyed, presented hereafter, have been based on the interpretation of the entire recent image in each sampling unit. Sampling unit area in this case does not correspond to that of the change matrices discussed in the previous sections. By referring to the entire recent image the land area observed amounts to 324.05 million hectares, representing an actual sampling intensity of 10.6 percent, versus the 8.2 percent intensity represented by the change matrices.

The resulting forest cover of each SU has been subsequently adjusted to the years 1980 and 1990 by applying the rate of change determined in the area common to both historical and recent images for each specific forest definition. Definitions of forest and their implications are discussed in Section 2.2.1.

In Table (4.3) 1, below, the figures at years 1980 and 1990 represent percentages of land area, while the deforestation rate represents the total area lost in ten years as a percentage of the forest cover in 1980.

These results refer to the land area from which the SUs were selected, which excluded satellite scenes with less than one million hectares of land or estimated, according to the project's pan-tropical vegetation map, to have less than ten percent forest cover (see Section 2.1.1). The area surveyed represents 62 percent of the entire tropical land area (see Annex 4), comprising some 87 percent of total tropical forests, according to most recent FORIS estimates. As a consequence, the forest cover estimates presented here cannot be directly compared to other estimates which refer to total and regional tropical countries' land area¹. In this context, for instance, to produce a comparable global estimate, the estimated “(F2) forest cover 1990” mean of 51.73 percent, given below, would correspond to some 32 percent when referred to the total land area, and should be increased to some 36 percent on account of the 13 percent of forest excluded from the area surveyed.

More than the mean value alone, one should consider the range of values within confidence limits, which are determined by the sampling error associated to such mean.

The statistical error associated with the global level estimates of forest cover at year 1990, ranging between ±3.5 and 4.5 percent of the mean, is quite small and very close to the ±3.9 percent value estimated during the design phase of the survey. A higher margin of error is associated with mean deforestation rates, ranging between ±12.3 and ±13.7 percent of the mean rates. This is not too high, however, considering the high variance deriving from the ‘event’ character of deforestation.

Future sample rounds will progressively reduce the sampling error associated to forest cover and deforestation estimates, thus allowing the production of results at more detailed reporting levels.

The graph in Figure (4.3) 1 illustrates the relationship between forest definitions and deforestation rates, highlighting their inverse correlation.

FOREST DEFINITIONS
		Fc 19901
F-1 = Closed Forest		Def + se2
F-2 = Closed + Open + 2/3 Fragmented Forest		Def Rate3
F-3 = Closed + Open + Fragmented Forest + Long Fallow		Def - se4

¹ Estimated forest cover at year 1990, referred to Y axis on the left.
² Deforestation rate + standard error, referred to Y axis on the right.
³ Mean deforestation rate, referred to Y axis on the right.
⁴ Deforestation rate - standard error, referred to Y axis on the right.

There is an apparent decrease in deforestation rate with the broadening of forest definition. This is determined by the fact that the changes within the forest, when a broad forest concept is used, are classified as degradation and fragmentation and not as deforestation. The forest definitions applied in this survey and the definition of forest area changes (deforestation, degradation, fragmentation, amelioration, afforestation, etc.) are discussed in Section 2.1.1.

¹ For a specific comparison with Phase I estimates for tropical countries, published in Forestry Papers 112 and 124 [FAO 1993 and 1995], see Section 5.1 “Relation between the Remote Sensing Survey and FORIS (FAO Forest Resources Information System) databases”.

4.4 EVALUATION OF ERRORS

How reliable are the results of the present survey?

This question cannot be answered by a single statement. There are elements with high reliability and others with low reliability. There are errors that can be quantified and errors that cannot be quantified. It is important, therefore, to discuss all these aspects and to try to understand both the strengths and weaknesses of the survey. This would help not only the user of the data presented here but also those who intend to carry out similar studies in the future. A thorough understanding of the error budget is in fact an essential prerequisite for the development of survey procedures.

The features of the survey that determine the errors, or that can be considered a source of error, can be defined as follows:

1. it is a statistical sample survey;
2. it is based on the interdependent observation of multi-date satellite data with the main focus on land cover changes; and
3. it is based on visual interpretation of satellite data, covering all ecological conditions of the tropical belt.

The first feature is a source of sampling error while the other two are sources of measurement error.

4.4.1 Statistical errors

The sampling errors related to the first feature, viz., the statistical sampling design, are the easiest to estimate and have been reported along with results of forest cover mean estimates and deforestation rates in Section 4.3.

Depending on the definition of forest, the standard errors of forest cover estimates at pan-tropical level range between ±3.5 and ±4.5 percent. It is important to highlight that the result for the intermediate definition of forest, which correspond more closely to the preliminary data set used during the design phase of the survey, has a standard error of ±3.6 percent, which exceeds in precision the originally defined goal of ± 3.9 percent. This shows that both the survey procedure and the sampling intensity were chosen correctly.

The standard error of the estimated rate of deforestation for the same definition of forest is ± 12.4 percent, much higher than that of forest cover. This can be expected, however, as deforestation has a very high coefficient of variation.

An additional source of statistical error relates to the use of dot grids for area measurement (see Section 2.4). The error in area estimates caused by dot grid measurement cannot be quantified exactly but there are empirical formulae which have been developed to obtain a good approximation of such error. Error estimation can be achieved by applying the following formula (Chevrou, 1971), reported in the Manual of Forest Inventory (Lanly, 1973), and modified slightly by Baltaxe in 1991:

e % = 52 (┘k / n^3/4)

As the land area studied in the present survey is quite large (324.05 million hectares, corresponding to 810 112 dots), and all class measurements have been based on many dots, errors can be considered negligible even when choosing a high k value of 8, as shown in Table (4.4.1) 1:

4.4.2 Change estimate errors

The measurement error related to the second feature, viz., change assessment, can be of paramount importance. In a comparison of two independent observations, the error associated with the change is higher than the error associated with the two original observations, as the law of propagation of error postulates. In the case of changes of small magnitude the resulting error could be too high to be acceptable.

In consideration of this problem the methodology developed for the assessment of class transitions was based on an interdependent interpretation procedure (see Section 2.3). This procedure ensured the highest level of consistency between the classification of recent and historical images. As a result of this approach the measurement error associated with class transitions can be minimized and remains comparable to that of the recent and historical classifications, as shown in the formulation below:

Errors in the estimation of change

INDEPENDENT versus INTERDEPENDENT interpretation approach

(ε₁ + μ₁) and (ε₂ + μ₂) are the errors ‘e’.
ε and μ may be positively correlated; i.e., much noise in the image makes the interpretation more uncertain.

VAR (state estimation where there is no change; i.e., Y₁=Y₂)
                      classification according to: X = (X₁+X₂)/2 = Y₁ + (μ₁ + μ₂)/2 + (ε₁+ε₂)/2
Var (X)= σ_ε²/2 + σ_μ²/2 + (2Kov (ε, μ))/2 = σ_e/²/2

VAR (state estimation where the class has changed)
                      classification according to: X₂ = Y₂ + ε₂ + (μ₁+μ₂)/2
Var = σ_ε² + σ_μ²/2 + Kov (ε₂, μ₂) = σ_e²/2 + σ_ε²/2

VAR (change estimation) = Var (ε₂ - ε₁) = 2σ_ε²

From the above formulation it would appear that the interdependent approach, where both images are used throughout the interpretation process, is beneficial not only in terms of change assessment but also in state assessment as the consultation of both images provides additional evidence in support of the delineation of the classes in both historical and recent images, as discussed in detail in Section 2.3.

4.4.3 Classification errors

The measurement error related to the interpretation accuracy of all 117 sampling units is very difficult to quantify. In fact, the accuracy of each interpretation could have been estimated exclusively on the basis of “true” reference (i.e. ground truth) collected in each study area at a very high cost; moreover, it is virtually impossible to estimate the interpretation accuracy of historical images.

Nevertheless, it is possible and important to associate the results with a critical evaluation of their reliability based on experience and on “educated guesses”. Following an approach similar to the fuzzy set theory (Zadeh, 1965; Gopal and Woodcock, 1994) it is possible to rank the results at various levels of reliability on the basis of some specific assumptions. Given an adequate knowledge of the study area, a good and coherent perception of the classification used and an acceptable image quality, it can be assumed that:

• a class is delineated with higher reliability where the contrast with the surrounding classes is sharper. Similarly, a change is more reliable when there is a sharp contrast between the original land cover class and the final one;
• the tonal intensity of vegetation cover classes is directly related to biomass density. Therefore, the reliability of class delineation (or class change) is higher where the biomass gradient with the surrounding classes (or with the original class) is greater; and
• the biomass density of vegetation cover classes is a function of the ecological character of the specific area studied.

Following the above assumptions an estimated (indicative) biomass value has been allocated to each land cover class for each of the three broad ecological zones defined in the previous chapter. This value represents aboveground woody vegetation biomass in tonnes per hectare. These values have been defined on the basis of the best available information on average biomass values of closed and open forest classes by ecological zone; the values for the remaining classes in each zone were derived from these two “biomass indicators”, as explained in Section 2.8.

The Tables in Annex 8 give, in the form of transition matrices for each ecological zone, the estimated biomass difference (in absolute value) for all possible class-to-class combinations. This difference is here termed as biomass gradient. The values thus determined can be used in two different contexts:

• Spatially - the biomass gradient indicates the contrast between two adjacent classes and the probable “sharpness” or reliability of such a boundary.
• Temporally - when comparing the same area at two points in time (on historical and recent satellite images) the biomass gradient indicates the difference between the two situations and, therefore, the “sharpness” or reliability of a given class-to-class transition.

Table (4.4.3) 1 provides a reliability ranking for the land cover classes in three ecological zones by stratifying the biomass gradients into three groups on the basis of their magnitude. Arbitrarily, the class relations (in space and in time) with a biomass gradient above 100 tonnes have been ranked as 1 (high reliability), those with biomass gradient between 50 and 100 tonnes as 2 (medium reliability) and those with a biomass gradient below 50 tonnes as 3 (low reliability). In view of its distinct spectral signature the class water and all combinations involving it are considered of high reliability.

The estimation of reliability applies equally to the spatial aspect (state assessment) and to the temporal aspect (change assessment); in the former, it represents the reliability in class delineation as a function of the contrast among adjoining classes, while in the latter it represents the reliability of class transition as a function of the contrast between the land cover class at date one and the class at date two.

Ecological zone: WET and VERY MOIST

Ecological zone: MOIST with short and long dry season

Ecological zone: SUB-DRY to VERY DRY

Frequency of class transition (Percentage of total change in each ecological zone)

No box = less than 1 percent

1 to 5 percent

5 to 10 percent

Over 10 percent

However, the reliability of classification (spatial aspect) is more difficult to relate to the final results discussed here since it depends mainly on the spatial relation existing among classes, which can only be seen on the spatial outputs of each individual SU. On the contrary, this estimation of class transitions reliability relates perfectly to the results since the change matrices report all class-to-class transitions originally observed in each SU.

In order to concentrate attention on the most frequent changes, Table (4.4.3) 1 indicates the class transitions that account for less than one percent of the total change observed in that particular zone, those representing between one and five percent, those representing between five and ten percent and the most important ones, representing more than ten percent of the total change.

Some comments:

• It appears that there is an overall decline in reliability in drier ecological conditions where the range of biomass values is narrower.
• The changes in forest area, and in particular in closed forest, are prevalently of high reliability in wet and moist conditions, and of medium reliability in dry conditions.
• The changes frequently observed, but considered of low reliability, are those from shrubs and short fallow to other land cover and vice versa.

Figure (4.4.3) 1 shows the indicative reliability of class transitions, of the pan-tropical change matrix, in relation to the number of changes observed. The pan-tropical reliability ranking has been estimated as the average of the three ecological zones weighted on the transition areas observed.

Although not an estimation of accuracy, this figure can serve as a useful guide when using the results presented or considering specific class transitions.