The “ink was barely dry” on the recent global forest assessment by the Food and Agriculture Organization (FAO) of United Nations (FAO 2001), when the World Resources Institute disputed FAO estimates of tropical deforestation (Stokstad 2001). However, a fundamental criticism used by the World Resources Institute is based on a flawed conclusion by NASA scientist Compton Tucker.
FAO used imagery from the Landsat satellite to estimate areal extent of forest cover and deforestation in tropical forests at broad, multi-national scales. They utilized internationally consistent definitions, protocols and baseline years. However, resources were insufficient to measure all 1203 Landsat scenes in the forested tropics1. Moreover, 100% coverage was unnecessary to meet FAO objectives. Therefore, FAO used a 10% stratified random sample of all scenes. Controversy ensued when Tucker asserted in Science that “a small and random sample – such as the 10% sample used by FAO ---‘will give you grossly inaccurate numbers.’ ” (Stokstad 2001).
Tucker based his criticism of FAO estimates upon his analysis of 41 Landsat scenes from lowland Bolivia, where 75% of all deforestation was concentrated into 5 Landsat scenes in 1985 (Tucker & Townshend 2000). Through simulations, Tucker & Townshend concluded: “Because tropical deforestation is spatially concentrated, it is very improbable that an accurate estimate of deforestation by random sampling of Landsat scenes will be achieved.”
The figure below gives empirical sampling distribution for all possible survey estimates with Tucker’s Bolivia dataset. This distribution assumes a simple 10% random sample with replacement (n=4, N=41), and it was estimated using a Monte Carlo simulation with 100,000 iterations. The resulting sampling distribution for Bolivia is strongly bimodal, which is apparent from the figure for “National Scale.” For most simulated samples of n=4, deforestation was significantly underestimated. Less frequently, a simulated sample contained one or more “hot spots” of deforestation, and the estimated extent of deforestation was grossly exaggerated. Therefore, Tucker & Townshend (2000) conclusions are very reasonable for Bolivia. However, Tucker’s extrapolation of these results to the larger geographic domains assessed by FAO is not scientifically defensible.
FAO never used its remote sensing survey to make estimates at the national scale, such as Bolivia. Rather, FAO used detailed forest inventory data supplied by each of 217 nations. Unfortunately, these national data include inherent limitations for multi-national and global assessments (FAO 2001, Stokstad 2001). Each nation optimises their own national forest inventory within their own funding constraints to address their own national issues, and the importance of international compatibility is rarely an important design criterion. Few tropical nations regularly conduct national forest inventories, and many are incomplete and out of date. There is no universally accepted definition of forest cover, which has a major affect on estimates for marginal forests in arid or cold regions. Funding disparities among nations cause differences in methods and data quality. Definitions and methods change over time. Expert opinion, assumptions and models must be used to adjust for these shortcomings, and there is no basis to evaluate the accuracy of those adjustments. Therefore, FAO supplemented these national data with its remote sensing survey, which uses globally consistent methods, data and definitions. Tucker criticised this latter survey.
Ideally, FAO would have measured each of the 1203 Landsat path/rows (3609 multi-date scenes1) that cover the forested tropics of the world. However, FAO did not have sufficient funding or staffing to accomplish this immense task. Moreover, a complete measurement of all 3609 scenes was not necessary to achieve the FAO objectives to assess pan-tropical forest cover and change. Instead, FAO opted to draw a 10% stratified random sample of these Landsat scenes, and use established statistical techniques to scientifically estimate the areal extent of forest cover and change for very large geographic areas.
The smallest sampling domain considered by FAO in the remote sensing survey was a multi-national sub-continental region, such as Central Africa. To test Tucker’s conclusion at this broader scale, I replicated each scene from Bolivia 4 times, which produced a hypothetical simulation population of N=124 scenes. Then, I drew a 10% random sample (n=12) without replacement, and made one estimate. I repeated this simulation 100,000 times, each with an independent random sample. The figure displays the resulting distribution of simulated estimates at the ”Sub-continental scale.”
Empirical sampling distributions for deforestation estimates at different scales from a 10% sample of Landsat satelite scenes. Distributions are estimated with simulation populations built using data from Tucker & Townshend (2000) for Bolivia, and using the same Monte Carlo techniques employed by Tucker and Townshend. The horizontal x-axis is the estimated deforestation rate, and the vertical y-axis is the expected frequency of that estimated rate over all possible 10% samples. At the “National Scale,” a 10% sample performs poorly in this example: very frequently, deforestation is underestimated; many times, deforestation is overestimated; and the estimates rarely agree with the true value. These results agree with conclusions by Tucker & Townshend for Bolivia. However, a 10% sample performs well at the broader geographic domains used by FAO. Therefore, Tucker and Townshend’s concerns over 10% sampling are not valid at the scales used by FAO, and Tucker’s widely publicized criticisms of FAO estimates are not scientifically credible.
FAO more often makes interpretations at even broader scales, such as the entire tropical zone of Asia, or a pan-tropical ecofloristic zone, such as the moist tropics of the world (FAO 1996). To evaluate estimates for these larger domains, I replicated Tucker’s data 13 times and simulated a 10% sample (N = 403, n = 40). FAO also provides important estimates at the global scale, which is simulated by replicating Tucker’s data 40 times (N = 1240, n = 124). The figure demonstrates that a 10% sample is capable of accurate estimates at appropriate scales. As the geographic area increases, so does the accuracy of estimates based on a 10% sample. This well-known phenomenon in sample surveys was ignored by Tucker in his criticism of FAO estimates of global deforestation.
These hypothetical simulations do not correctly estimate the sampling error in FAO estimates. Most likely, FAO estimates are more precise. Deforestation in lowland Bolivia is very concentrated, which is not true in all tropical regions. Deforestation in many other tropical areas is more spatially defuse (e.g., shifting agriculture). This latter type of spatial pattern can reduce sampling error with very large primary sampling units, such as the Landsat scenes1 used by FAO. Furthermore, FAO actually used stratified random sampling, where strata were formed based on expected rates of deforestation, and those strata with higher expected deforestation rates were proportionally sampled more intensively. Therefore, Tucker’s assertion is based on a worst-case scenario at an irrelevant scale. The simulations presented here merely use Tucker and Townshend’s own data to demonstrate the flaws in their conclusions when inappropriately extrapolated to the scales used by FAO.
1 Each Landsat scene covers 3.4 million hectares. 2376 Landsat path/rows cover the tropics, but only 1203 have significant forest cover (FAO 1996). 3609 scenes are needed to cover three dates (1980, 1990 and 2000).
