methodology

The major challenge for anyone attempting to collect economic information about the forestry sector is that, unlike other quantitative information such as standing volumes or productivity rates, the data collector often can not collect the data themselves, but has to rely on other people to supply the information required. This can be a problem if the respondent to an inquiry doesn't understand what sort of information is required or why it is required. Another problem is that they may not keep the data in the format required by the data collector and sometimes not keep such information at all. Perhaps the greatest challenge is that economic data is often quite sensitive and respondents may be reluctant to supply the information or have an incentive to give artificially low or high values.

Because of these challenges, economic data collection is often given a low priority within government forestry institutions. However, the importance of reliable economic data can not be stressed enough. Accurate and timely economic data is often essential for forestry policymakers to make sensible decisions regarding the setting of government charges and tariffs, the allocation of resources and the planned development of the forestry sector. Reliable information on the forestry sector also helps the forest administration (i.e. LBB) promote its policies and programmes within government and gain political support for its actions. In nearly all countries, forestry is seen as a relatively minor, low-value economic activity and reliable data to support policy decisions is an important tool to help forestry policymakers compete with other government departments for scarce public resources.

Accurate economic information about the forestry sector is also very important for the private sector. Good information is important to analyse potential investment in new infrastructure and machinery and to help develop markets. It is also important for the daily management of forest operations, in order to minimise costs of production and maximise the value of output. In a small developing economy such as Suriname, good information provided by a reliable and independent source is also crucial to attract foreign investment in the sector. A major concern to any domestic or foreign investor considering investing in the forestry sector is the risk associated with operating in the sector. To some extent, this can be reduced if reliable information about some key aspects of the forest economy is publicly available and trusted to be reasonably reliable.

Given the importance of economic information about the forestry sector and the difficulties associated with collecting it, this short report will attempt to identify the key aspects of successful data collection and recommend a strategy for collecting and improving the economic data currently held by LBB about the forestry sector in Suriname. The remainder of this section of the report will discuss the general principles behind successful data collection using, as examples, the methodology used during this study, before going on to present the data collected as part of this study in later sections.

Quantitative and qualitative information

Economic data on the forestry sector can be broken down into broadly two types: quantitative information and qualitative information. Quantitative information includes data such as: costs; prices; rates of productivity; profit levels; production, capacity and trade statistics. This type of information can be collected using: censuses and surveys; interviews with local experts; field studies; and mandatory reporting. Qualitative information includes variables such as: market expectations; investment and marketing plans; and opinions about topics such as: current policy measures; development options; and barriers to trade. Such information is usually collected through interviews or round-table discussions with key experts or stakeholders in the sector. However, in some countries, surveys have also been developed to collect this type of data.

Quantitative data is often the easiest type of information to interpret, because it can be analysed using standard statistical techniques, so that the precision of the data can be examined and useful measures such as averages and trends can be constructed. However, although qualitative information is often more difficult to analyse, it is often very valuable because it can give a more complete picture of the condition of the sector than the statistics alone would give.

Level of detail

A second important distinction between different types of information or data is the level of detail present in the information and the scale or area covered by the information. At a broad scale, data can be collected for a whole industry or the whole of the country. This is usually called macroeconomic data. This type of data includes information such as: the total size of markets; total production; general price levels; inflation and interest rates, and is usually collected using censuses, large representative surveys or mandatory reporting. Such data is most useful for monitoring broad trends and developments in the economy, but is often difficult to interpret (i.e. it is often difficult to analyse what lies behind any trends identified). In the forestry sector, the most common use of macroeconomic data is for projecting or forecasting future forest product supply and demand.

Alternatively, information can be collected and analysed at the level of individual companies or communities. This is usually called microeconomic data. Typical examples would include: production levels of individual firms; cost and price structures; productivity rates; and sawnwood recovery rates. Such information is usually collected using: field studies; interviews; and small surveys. This type of information is often more useful to get a better idea of the variation within a sector, measure the impact of policy changes and examine policy and investment options (option appraisal). Of course, if such information is regularly collected from a large enough representative sample, it can also be used to monitor broad trends and developments (as above).

A final distinction it is important to make is whether information is collected is in an aggregate form or as unit figures. For example: total sawnwood consumption; total sales income; total production and total fuel cost for a company or industry are broad aggregates. Corresponding examples of unit figures for each of these variables might be: per capita sawnwood consumption; product prices; labour productivity; and fuel prices. If unit figures are available and a measure of physical input or output (e.g. wage rates and number of employees), the corresponding aggregate measure can still easily be constructed, but the analyst has more flexibility in how they analyse the data.

Generally, unit figures are more useful for the forestry policy analyst, because they enable the analyst to examine a wide range of factors that might have an effect on the sector. For example, labour costs could change due to a change in the wage rate, changes in productivity, or a combination of both factors. Unit figures also make it easier to construct pieces of information that it would not be easy to collect directly. For example, it would probably be difficult to collect information directly on forest harvesting costs in Suriname, because few companies seem to keep such information. Most of them do, however, have a slightly better knowledge of productivity rates, numbers of employees and harvesting volumes (i.e. m³/ha). Such information can be used along with information collected elsewhere, to calculate the total production cost and examine the effects of changes in each of these variables on total harvesting cost (sensitivity analysis). It also generally makes it easier to update aggregated statistics in the future, because each of these components are likely to change at different rates over time (e.g. labour productivity may change only slowly and need to be remeasured every three years, but wage rates might change rapidly and need updating every year).

Objective, need and availability

A first priority before embarking on any data collection exercise is to be clear about what the objective of the exercise is. This will determine what sort of information is needed and start to clarify the tasks that must be performed. For example, the objective of the data collection exercise for this study was:

to collect cost, price and production data necessary to calculate the economic rent from roundwood production and broadly estimate the contribution of the forestry sector to the economy of Suriname.

The objective of the data collection exercise was, therefore, to provide data for a fairly complicated and large-scale analysis. This suggested that quite a large amount of detail about a wide range of variables was likely to be needed. However, due to time constraints, it also suggested that it would only be possible to collect a relatively small amount of information about each variable.

Other data collection exercises are likely to require different levels of detail or information about a wider or narrower set of variables. For example, in the proposal to create the Foundation for Forest Management and Production Control (Stichting voor Bosbeheer en Bostoezicht or SBB), three separate data collection exercises (exploration inventory, annual stock survey and the forest production control system) are proposed to meet the following objective:

to identify the appropriate level of harvesting in each forest concession and ensure that these harvest volumes are not exceeded and that all roundwood levies are paid to the government.

Each of these exercises require a large volume of data to be collected, but only require information to be collected about a relatively small number of variables (e.g. for the forest production control system, information is only required about: roundwood source; volume; species and type of product). Thus, with these needs in mind, a completely different system for collecting such data should be designed.

Another important consideration, which should be taken into account when designing any data collection system, is the availability of existing information. As noted above, the collection of economic information usually requires the co-operation of the private-sector to help to supply the information. There is likely to be considerable resistance to take part in any survey, if others have already asked respondents for similar information. Therefore, care should be taken to ensure that the required data has not already been collected elsewhere, either by other parts of LBB, other government departments, or the private-sector industrial associations (e.g. the Chamber of Commerce). The best way to check this is to speak with these organisations directly or even involve them in the design and implementation of the data collection exercise (see below). Other published sources of information may also be available either in Suriname or from international agencies. These should also be examined in order to reduce the amount of information that might have to be collected during the survey.

There is always a temptation when designing a data collection system, to try to collect more information than is actually needed. It is true that, if a staff member of LBB has to visit a sawmill to collect some information, then the cost of collecting more than is immediately required is quite small. However, any additional data requested should not be excessive. This is particularly the case if respondents are expected to complete questionnaires themselves. A short questionnaire has a reasonable chance of being completed, but a large questionnaire may simply be ignored (and thus provide LBB with no information at all). An excessively large questionnaire may also cause resentment and undermine any future attempts to collect information. Thus, any additional information requested should only be limited to information that may prove useful in the near future.

Censuses and samples

A key question, which must be asked when designing a data collection exercise, is whether a complete census of the sector is required or whether a sample of companies or individuals can be used. Broadly the main differences between a census and a sample are as follows:

A census involves collecting information about the total population under investigation (e.g. all companies or individuals in the forestry sector or all trees in a forest). Because of the amount of information collected, a census is therefore, likely to cost more to implement than a sample. It should however, give results that are precise and representative of the whole sector (as long as the whole population is accurately measured).

A sample involves collecting information from a proportion of the population (e.g. 10 companies or 15 x 1 ha plots in a forest). Less information is collected than in a census, so data collection and analysis is generally less expensive. However, the design of a sample survey is likely to be more complicated and, therefore, more expensive. Sample results are less precise than census results, in that there is always a risk that the sample may include a number of individuals that are untypical of the sector as a whole (sampling error). Precision increases with the size of the sample taken and the skill in designing a sample is to try to ensure that it is representative of the sector as a whole and that it collects sufficient information to give a reasonably precise estimate.

Generally, censuses are used to collect information that is simple and easy to collect, while samples are used to collect more detailed information. These and other factors that should be considered when choosing between using a census and a sample to collect information are shown in Table 1.

Table 1 Factors to be considered when choosing between using a census or a sample to collect information

It is rarely necessary to collect information using a census, because a well designed sample survey can usually produce sufficiently precise information at a much lower cost. Censuses are usually only considered where the information that will be collected is relatively simple and considered to be important. For example, LBB may wish to consider implementing a forest industry census in the future to monitor broad trends in the forest sector (see: Section 5.2 Improving and updating the information). However, for most purposes, a sample is likely to be the most efficient way of collecting economic information on the forestry sector in Suriname.

Given the time constraints for this assignment, it was not possible to consider undertaking a census to collect any of the information required for this study. Therefore, for the purposes of this study, information was collected from a small sample of companies in the logging, sawmilling and forest concession sectors, as well as local communities, industry associations and other key stakeholders in the sector.

Basic principles of sampling

As noted above, most of the economic info rmation about the forestry sector which LBB is likely to need in the future, can probably be most efficiently collected using samples of individuals or stakeholders in the sector. This section therefore, briefly explains the basic principles of sample selection. It does not discuss the detailed statistical calculations necessary to optimise sample selection, which can be obtained from any basic textbook (see, for example, Philip (1994) or FAO (1981), for a good discussion of forest sampling methodologies or Wonnacot and Wonnacot (1985) for an introduction to collecting and analysing socio-economic statistics). It focuses more on the broad principles and practical considerations when designing samples.

The overall aim of any sample is to collect information that is representative of the whole population without having to incur the expense of taking a complete census of the population. With this in mind, three considerations are most important when designing the sampling strategy: precision; accuracy; and robustness. The different meanings of each of these terms are explained in Box 1. Accuracy is by far the most important consideration and the elimination of bias should be given the highest priority in the design of any sample.

Box 1 The difference between precision, accuracy and robustness

There are two general principles that are commonly followed to try to ensure that a sample is accurate. The first is to try to divide the population into groups or strata, based on existing information about key variables which are thought to affect the information which the sample is trying to collect. This ensures that examples of different types of individual within the population are included in the sample. It can also be used to explain some of the variation recorded in the sample and thus increase the precision of the results. For example, in the sample surveyed for this study, information was deliberately collected from different locations (e.g. Paramaribo; two smaller towns; and in the forest), different scales of operation (e.g. small; medium-sized and large sawmillers) and different types of operator (e.g. sawmillers; independent loggers; and local villagers). The other way in which bias can be minimised is to collect information from a random group of individuals in the sample or each stratum and to try to ensure that each respondent gives all of the information requested.

Two concerns about accuracy were encountered with the sample selected for this study. Firstly, many of the individuals visited and interviewed tended to be people who already had a good relationship with LBB or the FAO Project (and who may not, therefore, be representative of the population as a whole). Secondly, some of those individuals could not or would not provide all of the information requested. There may, therefore be some bias in the information collected. A major concern about accuracy of measurement was that everybody interviewed assumed that the volume of an average log is 2 m³. This may not be true.

Precision can only be calculated if several measurements of the same piece of information are taken from different individuals. Thus, it is only generally possible to calculate precision if a formal surveying technique such as a questionnaire is used with a large sample. The survey techniques used for this analysis were less formal and used interviews and discussions to collect information about costs and prices. It is not possible therefore, to estimate how precise they are in the statistical sense. Other checks can be made however, to examine the reliability of the data collected (see: Section 2.3 Evaluating the information).

Given the fact that economic data about the forestry sector is often difficult to collect and that all economic analysis is subject to a certain amount of imprecision anyway, acceptable standards of precision for the collection of economic data are generally lower than might be required in other areas of forestry.

With respect to the robustness of the data collected from a sample, it is generally the case that many variables do not change rapidly over time. Thus, for example, industrial capacity, product recovery rates, labour and capital productivity rates and possibly even some cost data probably do not change by very much from year to year. The variables that probably do change a lot over short periods of time are: production levels; export levels; and product prices. Therefore, information about these variables may have to be collected fairly frequently.

The export markets for Surinam's forest products are currently depressed and this appears to have resulted in a large increase in supply to the domestic market. International product prices have declined and this shift has also pushed down domestic product prices. Thus, assuming that markets will eventually return to more normal levels, the price information collected as part of this study may not be very robust.

Basic types of survey

Once the type of information that needs to be collected has been identified and the census or sample has been designed, the next stage in the data collection process is to decide how the information should actually be collected. There are four basic ways in which economic information can be collected and these are briefly described below.

Direct measurement is probably the most time-consuming and expensive way of collecting economic information, but is likely to yield the best results. In the same way that a forest inventory crew can go to a forest and measure trees, it is possible for the forest economist or other suitably qualified data collector, to go to a forest or a sawmill and collect economic information directly. Accounts, invoices and receipts (if they are available) can be examined and used to produce valuable cost and price information in the format that is required for later analysis. Similarly, good estimates of productivity can be obtained if they are measured directly by the data collector (who knows what he or she is looking for) rather than having to rely on the estimate of the sawmill or forest manager.

The main advantage of direct measurement is the quality of the results usually obtained from such an approach. The main disadvantages of direct measurement are that it is generally slow and expensive and that it requires a tremendous amount of co-operation on the part of the sawmill or forest manager to agree to such an intrusion. However, this can sometimes be overcome if the data is not sensitive and the managers can see some use in having the information themselves. Alternatively, a questionnaire can sometimes be left with a manager and they can be asked (or paid) to collect the information themselves.

Questionnaires are probably the most common way in which data is collected. Questionnaires can be sent by post to respondents (for them to complete and return), or they can be completed over the telephone or face-to-face by the data collector.

The main advantages of using questionnaires are: that they help to clarify the information being requested; they ensure that all respondents are asked the same set of questions; they get to the point and can, therefore, collect a lot of information over a short period of time (although they may require some time to organise and administer); and they don't need a skilled person to ask the questions (see Box 2 for a basic outline of good questionnaire design). Postal questionnaires also have the advantage that they are very cheap to administer. The main disadvantages of questionnaires are that they rely on the respondent to provide the information, they are rigid and often require the respondent to have the information available in a particular format, they don't offer much scope for exploring issues which may be of interest and they often suffer from a poor response rate (which can lead to a lot of bias).

Interviews can be used to complete questionnaires (see above) or to collect information in a less formal way. Informal interviews may not be as rigid as questionnaire interviews, but they should still have an overall structure and the data collector should have in his or her mind a list of major subjects which should be examined or pieces of information which should be collected (see, for example, Appendix 2, for a list of the subjects covered in the interviews conducted as part of this assignment).

The main advantages of using informal interviews are that they allow the data collector to collect more detail in areas where the respondent has greater knowledge or has more enthusiasm to provide answers. They can also provide important information that the analyst might not have thought of and are particularly useful for collecting qualitative information. Perhaps the main advantage of this technique however, is that it is less threatening than direct measurement or a formal questionnaire and is, therefore, more likely to gain the co-operation of the respondent. The main disadvantages are that progress is generally slow with such an approach and that it usually requires a quite skilled interviewer (preferably the data analyst) to keep the discussion on the subjects that are important and collect the information required.

Box 2 Basic principles of good questionnaire design

Group discussions are similar to interviews, but with groups of people rather than individuals. They can be either formal (such as consultations with industry representatives) or informal (such as meetings with groups of local villagers). The same general principles as were described for interviews above also apply to group discussions. The only additional disadvantage with group discussions is that there is a risk that a few members of the group may dominate the discussions.

Formal discussions are unlikely to be useful for the collection of quantitative economic information, because members of the group may be competitors and unwilling to divulge such information to other members of the group. They are often, however, very useful sources of qualitative information and can be used to examine broad economic issues which are affecting the industry as a whole.

Group discussions can also be useful to collect information in situations where respondents may not be familiar with all the information being requested or are not used to answering such questions. For this reason, group discussions are a popular way of trying to collect socio-economic information about communities living in and around forests (see Box 3 for a brief explanation of the types of information which are often collected in such ways by socio-economists working in the forestry sector). The socio-economic impact of forestry development in Suriname is currently unknown and much of the discussion of its potential impacts is based either on experience in other countries or on very little information at all. Given the scale of expansion in the forestry sector in Suriname, this may become more of an issue in the future and skills in using group discussions to collect such information may have to be developed.

Box 3 Participatory rural appraisal - a technique to collect socio-economic information at the village level

Because of the wide range of data required for this study and the lack of existing information about what information might or might not be available in Suriname, it was decided to use an interview approach to collect economic information on the forestry sector. A complete record of discussions held with stakeholders is presented in Appendix 2.

Who collects the data?

The last point that should be considered in any data collection exercise is the question of who should collect the data. Two issues are of major importance: the level of skill required to collect the information and the need for confidentiality.

Generally, it requires a high level of skills to collect information using informal interviews or group discussion techniques. Firstly, the data collector should be technically competent in the subject. If they are, they can ask the right questions and follow-up interesting answers with other questions. They may also have to explain some of the questions to the respondent and should be able to spot inconsistencies in answers and help the respondent clarify what they mean (but not, of course, try to answer the question for them). The second skill required, is an ability to keep the respondent on the subject and supplying information that is of interest. This is generally the most difficult skill to acquire and can only be acquired with practice.

Collecting information with questionnaires or direct measurement generally requires less skill. The skill required for such work can usually be obtained with a little training and some testing with a supervisor present or checking the results later.

The issue of confidentiality can be a major obstacle to collecting sensitive information such as cost and price data. Respondents may understandably be reluctant to supply such information, particularly to government officials or consultants (who they may see as future potential competitors). The only way around this is to use data collectors to collect sensitive information who can present a credible guarantee that the data collected will be passed on anonymously to the government. Many countries use universities or independent research organisations to fulfil this role and this may be an option for Suriname. Another way in which such fears may be addressed is by involving the industry's associations in the process. For example, in many countries the private-sector forest manager's and processor's organisations are used to collect sensitive information and pass this on to the government. The scope for doing this may, however, be limited in Suriname, because none of the main forest industry associations have an independent secretariat.

As has already been noted, there is a range of statistical techniques that can be used to evaluate the quality of data collected in any sample or census survey. Indicators commonly used include: the standard errors of each estimate; the response rate to the survey; or just simply the sample size achieved in the survey. However, surveys of economic information often score poorly in terms of these measures (see earlier comments on precision), such that it is often very difficult to calculate whether the information collected is comparatively good or bad. Due to such problems, a number of other measures have been designed to evaluate the quality of economic data and these are briefly discussed below.

Internal consistency

The simplest way to evaluate the quality of a response to an economic survey is to check for internal consistency in the answers given. Extra zeroes or decimal points in the wrong place can often be spotted fairly easily, but other types of mistakes may be more difficult to explain (see Table 2, for examples of inconsistencies which might be encountered and possible explanations - all of these examples were encountered during this study). Internally inconsistent answers can often be overcome by contacting the respondent again and checking the answers with them. If this is not possible it may be necessary to discard the inconsistent answers or sometimes even the whole set of responses from that person.

Convergent validity

After any inconsistencies in the data have been corrected or eliminated, a slightly more rigorous test of the quality of the data collected is to examine whether each respondent has given answers to questions which are either similar to the answers from other respondents in the sample (or any major differences can be explained) or similar to data collected in other studies. If all the answers to a particular question in the survey are about the same, then they pass the test of convergent validity. This test only works, of course, if it would be expected that all the answers should be roughly the same. Thus, for example, wage rates paid for a particular type of forest worker might all be expected to be roughly similar, but product recovery rates might be expected to vary considerably (in which case theoretical validity is a better test - see below).

In the interviews carried-out as part of this study, one piece of information, which showed strong convergent validity, was the amount paid to village captains for permission to cut timber on HKVs. All respondents quoted figures of around Sf 3,000/m³ and this figure is similar to one collected in a separate study (Flaming, 1996). On the other hand, the sawmill export prices quoted by companies for 1997 varied considerably. Some of this variation may be due to differences in markets but, with the incentive to understate export values to avoid paying export levies, the validity of some of this information is questionable.

Table 2 Examples of inconsistencies that may be encountered in economic data and possible explanations

Theoretical validity

Theoretical validity is a test to examine the quality of answers to questions about variables that might be expected to differ. Thus, for example, transport cost information collected in a survey might be expected to differ, depending on the distance each respondent has to transport their roundwood from the forest to the sawmill. Similarly, expenditure on repairs and maintenance might be expected to be higher for a company with old machinery compared to one with relatively new machinery.

To test for theoretical validity, the answers given in a survey can be compared to variables which might explain differences (also collected in the survey or from elsewhere), to see if any trends or patterns emerge. This can be done either simply by plotting them on a chart, or by using more complicated tools such as regression analysis. Thus, for example, machinery repair and maintenance costs collected in a survey might be plotted against age of machinery, or transport costs might be plotted against distance from forest to mill. Any responses that appear to be well outside the trend (i.e. outliers) should be queried to see if there is any other explanation for why they should be so different.

Given the time available and the amount of information collected locally, not enough data was collected in Suriname to perform such a test on the responses given during the interviews with local sawmillers and loggers. However, a greater amount of information was collected from international sources for some variables and regression analysis was used to refine this information and identify underlying trends that were useful for the analysis (e.g. the calculation of depreciation rates from second-hand machinery prices collected from international sources).

With modern microcomputers is should be possible to store, for an indefinite period, all the information needed to carry-out economic and policy analyses of the forestry sector in Suriname. It is recommended therefore, that some of the information, which is currently held on paper files, should be put into simple spreadsheet databases and updated at regular intervals. In addition to putting all this information onto computer files, it is also important to be able to retrieve it again easily and at short notice. For this reason, it is useful to have some sort of classification or filing system that records or catalogues the location of different types of information. It is equally important to store and catalogue all of LBB's other information, such as books, reports and journals. More detailed recommendations on data storage will be given in: Section 5.3.1 Information storage and dissemination.

Box 4 Checklist for surveys of economic information