# 4. GENERAL SAMPLING CONSIDERATIONS

4.1 CENSUS COSTS AND OBJECTIVES OF SAMPLING
4.2 ACCURACY AND PRECISION IN SAMPLING
4.3 ACCURACY AS A FUNCTION OF SAMPLE SIZE
4.4 A PRIORI ACCURACY INDICATORS
4.5 SAFE SAMPLE SIZE FOR LANDINGS AND EFFORT
4.6 VARIABILITY INDICATORS
4.7 STRATIFICATION AND ITS IMPACT ON SURVEY COST
4.8 THE PROBLEM OF BIASED ESTIMATES
4.9 NEED FOR REPRESENTATIVE SAMPLES
4.10 THE “BOAT” AND “GEAR” APPROACHES

Choosing to undertake sample-based surveys is based primarily on the recognition that complete enumeration through census-based surveys imposes huge costs that are both unsustainable and unnecessary if the nature and methods of statistical sampling are properly considered. Such considerations include understanding of:

• the reasons for and objectives of sampling.
• the relationship between accuracy and precision.
• the reliability of estimates with varying sample size.
• the determination of safe sample sizes for surveys.
• the variability of data.
• the nature of stratification and its impact on survey cost.
• the risks posed by biased estimates.
• the differences between “boat” and “gear” statistical approaches.

Census-based techniques are generally impractical in small-scale fisheries due to the large number of fishing operations that would have to be monitored over a reference period. The following example outlines the logistics problems and costs involved in census-based surveys.

## 4.1 CENSUS COSTS AND OBJECTIVES OF SAMPLING

Assume a fishery of moderate size comprising 1,000 fishing canoes, each fishing 24 times during a month on a one-day-per-trip basis. This would mean that:

1) There would be about 24,000 landings during the month and all landings would have to be recorded, each with its complete set of basic fishery data (species composition, weight, etc) (Note that there will be no need for a separate survey for fishing effort, since all trips will be recorded.)

2) Assuming that a single recording of a landing would take a minimum of ten minutes (experience shows that this is the case in many data collection systems), a minimum of 240,000 minutes (4,000 work hours) will be needed.

3) If a data collector works 8 hours per day for 25 days in a month, then collection of data would require 4,000/8 × 25 = 20 data collectors just to monitor this relatively small fishery. This assumes that such a level of data collection is feasible and that landings and hence fisher availability is spread evenly over the day.

4) In addition to the costs of data collectors there would also be the costs of a) supervision, b) data editing, checking and inputting for 24,000 landings per month, and c) computer data storage for 12 x 24,000 = 288,000 landings per year.

On the other hand a well-defined sampling scheme would most likely need only one or two recorders for data collection and only a fraction of the computer storage and processing resources, due to the much lower volume of incoming data.

Thus there are three objectives of a sampling programme:

• to examine representative sub-sets of the data with the purpose of producing estimates of parameters, such as CPUE, prices, etc, that are as close as possible to the “true’ values that would be obtained through complete enumeration.
• to reduce operational costs.
• to reduce analytical and computing requirements.

## 4.2 ACCURACY AND PRECISION IN SAMPLING

In sampling procedures accuracy and precision are two different statistical indicators and it is perhaps worth clarifying their meaning at this point, as frequent reference will be made to these two terms in the coming sections.

4.2.1 Sampling Accuracy

• Sampling accuracy is usually expressed as a relative index in percentage form (i.e. between 0 and 100%) and indicates the closeness of a sample-based parameter estimator to the true data population value.
• When expressed as a relative index, sampling accuracy is independent of the variability of the data population, i.e. data population parameters of high variability can still be estimated with good accuracy.
• When sample size increases and samples are representative, sampling accuracy also increases. Its rate of growth, very sharp in the region of small samples, becomes slower beyond a certain sample size.

4.2.2 Sampling Precision

Sampling precision is related to the variability of the samples used. It is measured, in reverse sense, by the coefficient of variation (CV), a relative index of variability that utilizes the sample variance and the sample mean.

The CV index also determines the confidence limits of the estimates, that is the range of values that are expected to contain the true data population values at a given probability.

Estimates can be of high precision (that is with narrow confidence limits), but of low accuracy. This occurs when samples are not representative and the resulting estimates are lower or higher than the true data population value.

When sample size increases precision also increases as a result of decreasing variability. Its growth, very sharp in the region of small samples, becomes slower and steadier beyond a certain sample size.

The figure below illustrates the meaning of accuracy and precision. They are both important statistical indicators and regularly used for assessing the effectiveness of sampling operations. Their correct interpretation can greatly assist in identifying problem areas and applying appropriate corrective actions as necessary.

## 4.3 ACCURACY AS A FUNCTION OF SAMPLE SIZE

The following diagram illustrates the pattern of accuracy growth when sample size increases (see also table 4.5).

To be noted that:

• Accuracy is 100% when the entire population has been examined (as in the case of a census).
• The pattern of accuracy growth is not linear. The accuracy of a sample equal to half the data population size is not 50% but very near to 100%.
• Good accuracy levels can be achieved at relatively small sample sizes, provided that the samples are representative.
• The result of this relationship is that beyond a certain sample size the gains in accuracy are negligible, while sampling costs increase significantly.

## 4.4 A PRIORI ACCURACY INDICATORS

A frequent concern of fishery administrations is the limited budgetary and human resources for data collection. Such constraints have direct impacts on the frequency and extent of field operations for data collection and demand the development of cost-effective sampling schemes. Therefore, during survey design it is better to establish accuracy indicators so that sample sizes can guarantee an acceptable level of reliability for the estimated data population parameters. This is at times difficult, since at the outset little may be known about the distribution and variability of the target data populations. Until some guiding statistical indicators become available statistical developers will tend to require large samples which increase the size and complexity of field operations and data management procedures.

Formulation of a priori indicators for sampling accuracy during the design phase is feasible and may be achieved by:

• Guessing the general shape of the distribution of the target data populations.
• Setting-up accuracy indicators that are only a function of the data population size.

4.4.1 Target data populations

In the estimation of total catch and fishing effort (Sections 2 and 3), the two target data populations in sample-based catch/effort surveys are:

• The set of landings by all boats over a month from which an overall CPUE can be estimated.
• The set of 0-1 values (equivalent to “boat not fishing”, “boat fishing”) describing the fishing activity status of all boats over a month.

The target data population of fishing activity is used to formulate the probability (BAC) that any one boat would be fishing on any one day. The BAC will then be combined with the number of boats from a frame survey and a time raising factor to formulate an estimate for fishing effort.

The above two data populations have different sampling requirements for achieving the same level of accuracy. The next paragraph provides more detail on how sample size is determined in each case and in accordance with the level of accuracy desired.

## 4.5 SAFE SAMPLE SIZE FOR LANDINGS AND EFFORT

The desired accuracy level for a sampling and estimation process depends on the subsequent use of the statistics and the amount of error that users are willing to tolerate. In general, experience indicates that the accuracy of basic fishery estimates should be in the range 90% - 95%.

The table below illustrates safe sample sizes required for achieving a given accuracy level for two target data populations, boat activities and landings.

 Accuracyin % Sample size for boat activities(boats sampled) Sample size forLanding surveys(landings sampled) 90 96 32 91 119 40 92 150 50 93 196 65 94 267 89 95 384 128 96 600 200 97 1,067 356 98 2,401 800 99 9,602 3,201

From the table above the following conclusions can be made:

• Sample requirements for boat activities are about three times higher than those for landings.
• For a general sampling survey accuracy level of 90%, 32 landing records and 96 boats’ state of fishing activity records are required.

The above sampling requirements refer only to a given estimating context, that is a geographical stratum, a reference period (i.e. calendar month), and a specific boat/gear category. The process of determining safe sample size at a given level of accuracy must be repeated for all estimating contexts with the view to determining overall sampling requirements.

## 4.6 VARIABILITY INDICATORS

As already mentioned earlier, the second important statistical indicator is related to precision or, in reverse terms, to variability. The Coefficient of Variation (CV) is the most commonly used relative index of variability, usually expressed in percentage (i.e. 10%, 15%, etc). Experience indicates that CVs below 15% are indicators of acceptable variability in data samples. When very low variability (e.g. 0.1%, 0.5%) is repeatedly reported these results may be suspicious. Although this may indicate a very homogeneous data population, it may also be an indication of biased samples.

There are standard methods for explaining the overall variability in space and time. This is useful when it is feasible to increase sampling operations with the view to decreasing the variability of estimates. In such cases the availability of separate variability indicators in space and time would direct sampling operations to collect data from more locations or on more days. Reducing variability in estimates can also be addressed through the stratification of sampling (see below and section 5).

## 4.7 STRATIFICATION AND ITS IMPACT ON SURVEY COST

4.7.1 Definition

Stratification is the process of partitioning a target data population (e.g. all fishing vessels) into a number of more homogeneous sub-sets based on their characteristics (e.g. trawl, gillnet, purse seine; or large, medium, small; or commercial, artisanal, subsistence). Stratification is normally undertaken for the following reasons:

• For statistical purposes (e.g. to show the difference in catch by vessel type) and when there is a need to reduce the overall variability of the estimates. For example, catch rates will differ greatly between vessels of a similar type but of a different size, therefore sampling of each size class separately will enable the preparation of meaningful statistics. If all vessel size classes are ‘lumped’ together - i.e. not stratified - then, say, average catch is not meaningful for any one size class.
• For non-statistical purposes (e.g. different geographic regions) and when current estimates are not meaningful to users of the statistics unless estimates are shown separately.
• At times stratification is “forced” due to administrative needs such as limits to data collection and reporting.

4.7.2 Impact on costs

The implementation of sampling stratification can be an expensive exercise and should always be applied with caution because all new strata need to be covered by the sampling programme. Introducing a large number of strata may have serious cost implications because the overall accuracy of the estimates will not be increased if data collection effort is kept at the original level, even though the results from strata will be more homogeneous than the original data population. In general, more strata means greater sampling costs, although obtaining better value (= statistical accuracy) for money.

To fully benefit from a stratified population, safe sample sizes must be determined for each new stratum. In very large populations this would mean that a new sampling scheme with three strata would need three times more samples for achieving the desired accuracy, hence greater costs.

## 4.8 THE PROBLEM OF BIASED ESTIMATES

4.8.1 An illustrated example

The figure above illustrates in basic terms the problem of bias. Biased estimates may be found systematically above or below the true (but unknown) population value (here all estimates are shown higher than the true value). Bias is independent of the precision (= variability) of the estimates. In this example accuracy is bad but precision is misleadingly good and this is indicated by the narrow confidence limits.

4.8.2 Bias as a major risk in sampling programmes

Biased estimates are systematically lower or higher than the true population value, generally because they are derived from samples that are not representative of the data population. Bias is not easily detectable and at times not detectable at all. Consequently users may be unaware of the problem since they also do not know the true population value.

Precision (or the relative variability indicator CV) cannot be used to detect bias. However, repeated cases of extremely small variability (e.g. CV<1%) may be indications of a biased estimate.

Although attempts to increase the representativeness of samples are often compromised due to operational constraints, the best approach to the reduction of bias is through the application of appropriate stratification.

## 4.9 NEED FOR REPRESENTATIVE SAMPLES

The risks of biased data are considerably reduced if sampling operations collect data that are as representative as possible.

4.9.1 Data collection at sampling sites

Collection of representative samples at a sampling site is not a difficult task provided that data collectors are adequately trained and briefed. For the collection of effort data, sampling should always be undertaken from a random selection of fishers without prior knowledge of whether they have been fishing or not.

When boats land within a short period, recorders at times tend to sample those with a small catch in order to cover as many landings as possible. Also, if landings occur over longer periods and recorders must visit other sites during the day, only the first landings at the first site will be sampled. These selections may introduce negative bias in CPUEs, species composition and prices. Therefore, care should always be taken to sample from a random selection of landings at random times.

4.9.2 Selection of sampling sites

In medium and large-scale fishery surveys the major task in obtaining representative samples is at the first sampling stage through selection of the locations where data will be collected. Often, a good approach is to select sampling sites on a rotational basis as part of an overall sampling strategy. Field teams would then cover the chosen sampling locations by visiting all of them at appropriate times, say once a month. Such an a priori selection of sampling sites enables planning for sufficient and mobile human resources.

When there are other operational constraints (accessibility, availability of data collectors, limited mobility, etc.), a planned rotational approach may not be feasible and data collection may be performed from sampling sites at fixed locations for long periods. The problem is that pre-selection of sampling sites runs the risk of biased samples if the landing sites are not representative of the entire statistical area.

4.9.3 Criteria for selecting sampling sites

Frame surveys and existing geographical information are used to make a priori selection of fixed sampling sites. The main criteria in selecting sampling sites are:

• Sampling sites should provide a satisfactory geographical coverage of the statistical area. Limited human resources or transport will usually be the major operational constraint to this coverage.
• Original frame surveys of the numbers of boats (fishing units) by site and boat/gear type will indicate the relative importance of sites in terms of potential fishing effort (i.e. very important, important, less important, etc.). Sampling sites should represent all boat/gear types involved in the survey, and sampling should focus on sites with larger numbers of fishing units.

4.9.4 Example

Rather than examining sites on an individual basis, planners may look at groups of sites that offer a better statistical coverage because of their proximity. Criteria for grouping several sites together are:

• whether a recorder can visit all grouped locations within the periodic (e.g. daily) sampling schedules.
• whether the group of sites contains sufficient fishing units from most or all boat/gear types.

The figure above illustrates a minor geographic stratum with 19 fishing sites. Table 4.9.5 contains the results of a frame survey for gillnets, beach seines and castnets.

Table 4.9.5 Frame survey data

 Site Gillnets Beach seines Castnets 1 4 0 7 2 11 0 0 3 1 8 2 4 5 0 9 Group 2, 3, 4 17 8 11 5 12 4 5 6 3 0 0 7 2 1 3 8 2 2 0 9 4 1 0 10 5 3 6 11 4 3 0 12 3 2 4 13 1 0 9 14 0 0 7 15 8 3 6 16 7 4 3 Group 13, 14, 15, 16 16 7 25 17 6 0 0 18 14 5 9 19 5 0 7

On an individual basis, sites 5 and 18 are the most important sites since they contain the largest numbers of all boat/gear types. However, if secondary sites are grouped they can offer better statistical coverage. Thus, if planners consider the two sampling options from:

• Sites 5 and 18, or
• Groups 1 and 2

the second option offers more statistical advantages both for in-space coverage and boat/gear representativeness.

## 4.10 THE “BOAT” AND “GEAR” APPROACHES

Determining the fishing unit (boat or gear) that will be the subject of sampling operations is a major decision in planning sample-based fishery surveys.

4.10.1 The “boat” approach

The fishing boat as statistical unit is the commonest approach because:

• Frame surveys usually provide numbers of boats by boat/gear type, which can then be used for in-space raising factors for estimating fishing effort.
• Fishing activity level is most often measured by the Boat Activity Coefficient (BAC), which expresses the probability that any boat will be active on any day.
• Catch Per Unit of Effort (CPUE) is often expressed as the average catch per day of a boat of a certain boat/gear type.

4.10.2 The “gear” approach

Alternatively the fishing gear type can be used as the statistical unit, e.g. 100-metre gillnet units, 500-hook line units, 100-metre beach seine units or traps, etc. This approach can be used when:

• Frame surveys provide numbers of gears by boat/gear type as in-space raising factors for estimating fishing effort.
• Fishing activity levels are measured by the Gear Activity Coefficient (GAC), which expresses the probability that any gear will be active on any day.
• Catch Per Unit of Effort (CPUE) is expressed as the average catch per day of a gear of a certain boat/gear type.

4.10.3 Comparison of the two approaches

Overall, the “boat-specific” approach is more advantageous than the “gear-specific” because:

• Frame surveys of gear numbers are more complex, more demanding in staff time and become less accurate over time since fishing gears change more frequently than fishing boats.
• Gear activity level is far more difficult to measure, and the relative variability of “gear-specific” estimated parameters is not lower than “boat-specific” ones.
• Estimates produced from the “gear-specific” approach cannot be easily integrated.

The major advantage of the “gear” approach is that it can better handle cases of multiple gears (whether in sequential or concurrent use).

 SUMMARYIn this section general aspects of sampling methods have been discussed, including:(a) The reason for and objectives of sampling: sampling techniques can provide estimates of good reliability and are more economical than census approaches.(b) The appropriate context of the terms accuracy and precision: Accuracy is a measure of closeness of an estimate to the true data population value, while precision is related to its variability. Both are functions of sample size.(c) Safe sample sizes can be determined on an a priori basis, and separately for landings and effort surveys.(d) The interpretation of variability indicators in space and time.(e) Stratification and its impact on survey cost.(f) The problem of biased estimates.(g) The problem of selecting representative sampling sites.(h) Comparison of “boat-specific” and “gear-specific” statistical approaches.