4 MONITORING AND SAMPLING STRATEGIES

Key outputs of observer programmes are data and summary information for fisheries management. Of course, data should not be just a random collection of whatever is available to observers placed indiscriminately on any vessel steaming off to the fishing grounds. Their collection instead needs to be based on an overall strategy. Conditions vary from fishery to fishery, and each observer programme will to some extent be unique. However, any collection strategy should specify:

a) which vessels are sampled in the fleet;

b) which events on the vessels are sampled; and

c) what catch is sampled from a fishing event.

Furthermore the formulation of any strategy involves the following steps:

• evaluation of the baseline information available on the fishery in relation to the programme objectives;

• assessment of whether complete monitoring or sampling is appropriate for the fishery;

• assessment of operational considerations for the fishery and the programme;

• strategy design in relation to vessels, events and catch to be monitored;

• implementation of a pilot phase to validate the design strategy and methods for data collection; and finally

• establishment of a feedback mechanism between the observer programme and the fisheries authority to ensure that data satisfy the objectives and current environment.

The rest of this chapter explores each of these areas in more detail.

4.1 BASELINE INFORMATION

Baseline information describes the overall fishery and is essential to establish a good data collection scheme. Information can be gathered from many sources such as fishing logs, vessel registers, research cruises, maps and charts. It should include a description of the following features:

• The fishing grounds and associated water body.

• Ports: their locations, patterns of distribution and accessibility.

• Fishing fleets: the numbers of vessels by type (e.g. in relation to fishing gears), their size distribution and ability to take observers, and their geographical distribution in relation to home or landing ports.

• Fishers, companies and fishery associations in relation to the fleets (above) and related to the activities (below).

• Fishing activities and patterns related to the home and landing ports, noting any seasonal movement between fisheries.

When no information is available for any of the above parameters, this in itself is an important observation, which would stimulate efforts to obtain it. After a pilot phase has been completed the available information will have increased and the baseline picture of the fishery can be re-assessed.

4.2 COMPLETE MONITORING AND SAMPLING

Complete monitoring (also known as complete enumeration or census) and sampling are the two options available for data collection in an observer programme. Although opportunities for complete monitoring are improving (see Box 10), some sampling will inevitably occur. Even in a very small fishery, where observers can monitor every vessel (complete monitoring of vessels) and possibly even monitor every fishing event (complete monitoring of fishing events), they cannot monitor every fish.

Box 10

There are three main stages in preparing a sampling strategy for a given fishery:

a) which vessels are sampled in the fleet;

b) which events on the vessels are sampled; and

c) what catch is sampled from a fishing event.

The aim of either complete monitoring or sampling is usually to derive a fishery statistic from raw data, e.g. catch per unit effort. To find out the true value of any statistic complete monitoring is required. For example, if observers are placed on every vessel in a fishery, statistics on the catch, the gear used and the number and duration of fishing events (effort) could be calculated exactly.

When should a variable be completely monitored and when should it be sampled? The choice is usually controlled by financial and human constraints. Complete monitoring rarely makes operational sense, particularly when a sampled statistic is sufficient to meet information requirements. There are three significant exceptions where complete monitoring is the normal case.

· Compliance controls, regulatory conditions, and conservation and management measures: when the monitoring of violations is only possible by observers, e.g. when strict gear controls are required; when discard of undersized or by-catch species is prohibited; or when expensive fines are imposed as a deterrent.

· Pilot schemes are used for baseline data collection, especially if the data collected is required to make statistical assessments for determining long-term sampling strategy.

· Data variability is high and the sample is small: e.g. in a fishery with very few but very different vessels (e.g. in their size or crew characteristics). In a case like this, little effort would be saved in creating a viable sampling scheme.

When sampling is used, an estimate is made of the true value of a statistic, and the aim is to make that estimate as accurate as possible. For example, if an observer samples 10 baskets of fish out of every 100 to determine the overall species composition of the catch, the results of the 10 samples will be multiplied by 10 (thus raising the figure) to produce the sampling estimate.

The key question is, how many samples should be collected, with what frequency? The more samples collected, the more likely they are to be representative of the underlying data population (i.e. their accuracy in relation to the 'true value'). The question of sampling frequency is dependent on the variance (difference) between the samples. However, increasing the number of samples does not proportionally increase the precision of the value. This means that the cost of decreasing the variance (getting closer to the 'true value') gets disproportionately higher the closer you get to the 'true value'. This is a clear case of diminishing returns, as demonstrated in Figure 5.

Figure 5 The relationship between the number of samples and the deviation from the true value

The above example shows that halving variance from 5% to about 2.5% (which may, of course, be desirable) will increase sampling frequency (and hence effort and cost) by 10 times from about 20 to 200 samples.

Decision-making on sampling frequency - the sampling methodology - needs to be based on sound statistical analysis. However, at the start of an observer programme, perhaps even after a pilot phase, there will be very little data to allow for such analysis. Simulated data (generated by computers) can be used as an alternative but these data are rarely adequate substitutes for the real thing. What can be done? The answer is to design a sampling strategy based on the knowledge gained during the development of baseline information, common sense, and a few simple guidelines given in the sections below.

4.3 OPERATIONAL CONSIDERATIONS

A sampling strategy should be based on long-term planning for the financial and human resources that it will involve. This planning may also need to include training (Chapter 3) and information systems (Chapter 5).

Factors and information to assess operational constraints to be taken into account when designing sampling strategies include:

• Number and distribution of ports (long distances apart may make the placement of observers difficult and costly).

• Length of fishing trips and possible movement of vessels out of the EEZ to fish or to off-load fish (or observers!).

• Conditions at-sea and facilities for sampling on the vessels.

• The average number of fishing events per 24-hour period (hauls or longline/basket sets).

• The level of safety awareness, equipment and certification on the vessels.

• National labour and employment laws relating to overtime, days off in lieu of weekends and public holidays worked, etc.

• Financial requirements, including human, capital and operational costs.

4.4 SAMPLING STRATEGY DESIGN

The aim of strategy design is to sample the catch in a way that best represents the whole catch. The challenge is to avoid sample bias. Bias is different from variance; it is a measure of the difference with the true value because the samples are not representative of the data population. This is commonly caused by, for example: observers picking out the biggest, brightest or most beautiful fish, rather than from all fish; or by observers persuading coordinators to let them go on to the more comfortable vessels. In other words, selection of the sample is not random.

The most common solution to the problem of bias is to stratify the data population and to sample randomly within the strata.

4.4.1 Stratification

Most sampling strategies are stratified. This means that the vessels or the fish are divided into groups related to how similar they are. This stratification makes it easier to obtain a better result from the applied sampling effort. This works because the strata systematically remove some of the data variability, e.g. the difference between samples that may be taken on small or large vessels, or by different fishing gear types. After the data are collected they can be raised according to the different strata to produce a more accurate estimate of the true value than if all the data were raised together by a common raising factor.

A multispecies catch will require careful random sampling (P.E. Bergh).

To reduce the bias and variance of the estimate a good sampling strategy is needed.

To reduce mistakes good training and a positive attitude are needed.

The aim is to make the variability between the strata greater than the variability within the strata (Figure 6).

Figure 6 The variability between the strata is greater than the variability within the strata

It is important to differentiate between the catch and the fish stock. Observers are performing sampling on commercial catch, while research sampling will often aim to sample the entire fish stock.

Usually, the allocation of strata is initially made through operational considerations and common sense, for example by homeports, fleet details, fishing gear or vessel type (see Table 9). The data collected can be statistically analysed by setting up a null hypothesis that there is no difference between the strata and then statistical testing to see if this is correct. Depending on the results, the sampling design may need to be adjusted by increasing the sample sizes, number of samples or by further stratification. Allocating sampling effort between strata should also be managed and not just done by chance. There are three basic rules known as the Von Neumann allocation that optimises the allocation of sampling effort between strata (see Box 11).

Box 11

Box 12 gives examples of the types of sampling strategy analysis that can be used and provides details of further information on the subject.

Table 9 Possible strata for observer programmes

4.4.2 Random, non-random, systematic and adaptive sampling

Random sampling (i.e. non-selective sampling) is required by most statistical analyses. In theory, random sampling seems simple, but in practice it may be quite complicated to achieve.

For example, when choosing which vessels to place observers on, random sampling is difficult to achieve. A random number generator can be used to generate random numbers within a specified range (e.g. 1 to 5 or 1 to 20) and then observers are placed on the vessels due to leave port in relation to these numbers. The difficulty with this approach is to cover each stratum (the vessels) adequately, with neither too few nor too many observers. It is often easier to use systematic placement of observers on vessels, by choosing, for instance, to place an observer on say every 5th or 20th vessel leaving port.

It is sometimes possible to use adaptive sampling for the placement of observers. Adaptive sampling is when a random or systematic system is overruled by circumstances. The opportunities for this have increased with the introduction of new technology such as vessel monitoring systems (VMS). For example if VMS reveals that all the vessels with observers on board are in the northern fishing grounds it may be appropriate to place an observer on a vessel that is headed south.

When choosing which fishing event to sample, random sampling is rarely successful. Vessels and observers have routines and generally observers prefer to sample fishing events systematically. The main problem here is when observers follow a pattern that introduces bias, such as by only sampling in the morning or only in daylight. Procedures are required to ensure that such bias is avoided; to specify this to observers and to check that they are not 'stuck in a routine.'

Random sampling, nevertheless, can be applied in most cases on vessels to sample fish for size frequency, sex ratio, and maturity data (see Chapter 5, on collecting the sample).

An alternative method is systematic sampling when, say, every fifth basket or one-hundredth fish is sampled. This can be quite complicated and time consuming, and is usually not feasible.

It is important that observers get a representative sample of the fish that are caught and not those that are processed. Emphasis must be placed on collecting the sample before any sorting has taken place. Non-random sampling can be used for collecting fish that will be used for length weight or conversion factor purposes as well as for certain collection schemes such as for otoliths, statoliths, stomachs etc.

4.5 PILOT PHASE

It is important that observers get a representative sample of the fish caught and not those that are processed.

Before launching a full observer programme for the first time, or before starting a collection scheme on a new fishery, a pilot phase should be carried out. Its primary purpose of which is to gain familiarity with the fishery conditions and data types available, rather than to collect data for direct use. A pilot phase is limited in time and space, but often very inclusive in the types of variables collected and the procedures used. This enables 'testing' of the suitability of different variables and different procedures for collecting it (sampling methods). Box 13 gives an example of a simple strategy that could be used for a pilot phase.

4.6 FEEDBACK MECHANISM

Proper programme maintenance requires continuing 'feedback checks' since situations change very quickly, even for well-designed strategies, and a flexible system is more likely to succeed.

The primary feedback check is to analyse the data to ensure that the data, and estimates derived from them, are statistically valid. This will provide the opportunity to evaluate the success of the stratification, the appropriateness of the number of samples and sample sizes, and to make any necessary changes.

The primary purpose of the pilot scheme is to gain familiarity with the fishery conditions and data types rather than to collect data for direct use.

Other feedback mechanisms could include checks on how well the programme is following the sampling strategy. For example, if the strategy proposes to cover 10% of all fishing trips this can be easily checked on a regular basis. This kind of simple check allows for constant re-adjustment of the strategy, especially in the early days when operational constraints are starting to be more fully appreciated.

It is statistically better to take many small samples from many different vessels than fewer larger samples from fewer vessels.

Box 12

Box 13