PART I
TECHNICAL AND REVIEW PAPERS (Contd.)

THE DELPHI TECHNIQUE: A POTENTIAL METHODOLOGY FOR EVALUATING RECREATIONAL FISHERIES¹

James R. Zuboy

Department of Commerce, National Oceanic and Atmospheric Administration, National Marine Fisheries Service, Southeast Fisheries Center, 75 Virginia Beach Drive, Miami, Florida 33149, USA

ABSTRACT

The Delphi technique, a methodology for systematically developing expert opinion consensus, is suggested as an approach for generating recreational fishery data. This paper describes the Delphi technique, cites an example of how it has been applied in water resource management, and discusses the results of a Delphi experiment which was designed to provide an estimate of the recreational (diver) catch of spiny lobsters from Florida waters.

RÉSUMÉ

Les travaux statistiques permettant d'estimer les captures de la pêche récréative sont coûteux et leurs résultats sont souvent l'objet de graves critiques. L'auteur propose comme solution de rechange pour élaborer des données sur la pêche récréative de Delphi, une méthodologie qui permet de mettre au point systématiquement un consensus d'opinions parmi les experts. Le document décrit la technique de Delphi, cite un exemple de son application dans l'aménagement des ressources aquatiques, et discute les résultats d'une expérience réalisée à l'aide de cette méthode en vue de fournir une estimation des captures de la pêche récréative (sous-marine) de la langouste dans les eaux de la Floride.

¹ Southeast Fisheries Center Contribution Number 80-14M.

INTRODUCTION

Effective fishery management requires data on the recreational as well as on the commercial component of a fishery. The usual means of collecting recreational fishery data is by statistical sampling survey. The Delphi technique is an alternative to statistical surveys which should provide sufficient information for management in many cases. Delphi is not offered as a panacea which will replace traditional methods of obtaining fishery data, however, it can provide useful and timely data at less cost than standard survey techniques.

This paper provides a brief description of the Delphi technique and an example of how the technique has been applied in water resource management in Michigan. The emphasis here is on the application of the technique to estimate the recreational (diver) catch of spiny lobsters (Panulirus argus) from Florida waters. For a detailed discussion of the technique and its applications, the compendium entitled The Delphi Method (Linstone and Turoff 1975) is an excellent source.

THE DELPHI TECHNIQUE²

The origin of the Delphi technique can be traced to the “cold war” era of the early 1950's. The purpose of “Project Delphi,” a U.S. Air Force sponsored study, was to apply “expert opinion to the selection, from the point of view of a Soviet strategic planner, of an optimal U.S. industrial target system and to the estimation of the number of A-bombs required to reduce the munitions output by a prescribed amount.” Since this inauspicious beginning, Delphi has evolved as a basic tool in the area of technological forecasting and is used today by many technological-oriented corporations. The usefulness of Delphi has also been recognized by those attempting to solve the complex problems of society, and in recent years the technique has been applied in areas such as the environment, health care, and transportation.

Delphi is a methodology for systematically developing expert opinion consensus.³ The concept is based on the reasonable premises that 1) the opinions of experts are justified as inputs to decision-making in inexact areas (i.e., where absolute answers are unknown or impossible) and 2) a consensus of experts will provide a more accurate response to a question than a single expert (Fusfeld and Foster 1971).

The primary characteristics of Delphi are: 1) anonymity of the experts, 2) controlled feedback, and 3) an estimator of group opinion. The anonymity feature is important because it helps to eliminate bias. In any group interaction (e.g., brainstorming) an individual may be influenced by what another individual says or simply the manner in which he says it. A Delphi inquiry is an iterative process and at each iteration there is an assessment of group judgement and controlled feedback to all participants in succeeding rounds. This presents the experts with an opportunity to revise their opinion based on new data as the experiment progresses. An estimator of group opinion, usually the median and interquartile range of the estimates, is calculated and provides the basis to extrapolate a forecast.

Here's how a typical Delphi would operate. The investigator identifies a group of experts on the subject. The experts are then polled individually, usually by questionnaire. The results are summarized by the investigator, which generally entails determining the median and range of responses to a given question. This information is then given to each respondent and he is asked to reanswer the questions in light of the new data generated by the aggregate responses. If his new response is outside the interquartile range from the previous round he must write a brief explanation in support of his estimate. These explanations are then provided to all the respondents in the next round. Cycling through this procedure usually results in consensus by the fourth or fifth round.

There have been extensions to the Delphi technique over the years, but they have not affected the basic design. The extensions to Delphi include: 1) cross-impact analysis, developed to solve the problem of interrelationships among forecasted events; 2) SEER (System for Event Evaluation and Review), which helps to make Delphi forecasting more efficient by developing initial lists of events through interviews, and thereby reducing the amount of time required by the Delphi participant; and 3) trend extrapolation, which forces the experts to relate their intuitive models of the future to actual rather than perceived historical trends by providing historical time series data for analysis and prediction (Fusfeld and Foster 1971).

As with any survey tool, Delphi has both positive and negative aspects. In the Michigan Sea Grant Delphi, which will be discussed in the next section, Ludlow (1972a) had his panelists evaluate the methodology. Some of the positive aspects identified were: 1) that it has the advantages of a committee approach without the necessity of meeting, 2) the introduction of ideas not likely to be considered by any one individual, 3) the systematic refinement of issues as the exercise progresses, 4) participants are less susceptible to being influenced by a strong personality than in other procedures, and 5) participation can be at one's convenience rather than at a specific time. The negative aspects identified were basically related to the desirable aspects of conferences that are not realized with Delphi. The participants felt that the Delphi technique did not provide any opportunity to: 1) change and exchange opinions, 2) hear reasons for various opinions, 3) benefit from the cross-fertilization of the others' (sometimes better informed) ideas, and 4) clear up any ambiguities in terms and questions. Linstone (Linstone and Turoff 1975) identified a number of pitfalls of which a Delphi investigator should be cognizant, and then concluded with a statement (p. 586 which sums it up well: “The Delphi designer who understands the philosophy of his approach and the resulting boundaries of validity is engaged in the practice of a potent communication process. The designer who applies the technique without this insight or without clarifying these boundaries for his clients or observes is engaged in the practice of mythology.”

² Delphi was the most revered oracular site in ancient Greece. Thus, Delphi is an appropriate, and rather imaginative, name for a technique which purports to foresee (in a sense) the future.

³ Delphi can also be used to obtain a consensus of “informed” opinions where experts are not available.

UNIVERSITY OF MICHIGAN SEA GRANT DELPHI

The University of Michigan's Sea Grant Delphi inquiry (Ludlow 1972a, b) is a landmark application of the Delphi technique to natural resource management. Prior to this study the Delphi technique had been used primarily to forecast long-range technological developments.

The basic objectives of the study were:

1. to obtain the judgements of a multidisciplinary team of researchers regarding:

   1. potential technical, social, economic, and political developments that could influence the management of water resources in a region similar to the Grand Traverse Bay area;

   2. assessment of the relative importance of future sources of pollution of a body of water similar to the Grand Traverse Bay;

   3. recommended waste-water treatment and disposal systems;

   4. regional opportunities, problems, and planning strategies;

2. to communicate the researchers' judgements to people in the Grand Traverse Bay area;

3. to evaluate the effectiveness of the Delphi technique in solving the problem of integrating the judgments of the researchers and conveying their informed insights to decision makers (Ludlow 1972b).

The magnitude of this first natural resource oriented Delphi was impressive. Sixty-nine individuals took part in the study. They were assigned to different panels depending on their background and experience. One panel was composed of individuals whose expertise was primarily in the physical sciences, predominantly engineering. Another panel was composed of individuals who were more oriented to the behavioral sciences. The third panel was made up of concerned citizens who were residents of the Grand Traverse Bay area and believed to be influential in the following fields: civics, business, planning, politics, natural resources, government, and education.

The Sea Grant Delphi followed the general procedure of a standard Delphi inquiry with minor modifications. The details need not be discussed here. The important point is that the Delphi exercise was felt by the participants to be successful in meeting specified objectives. Especially important, a critical evaluation of the method demonstrated the potential of a Delphi inquiry for improving the dialogue between researchers and regional problem solvers. One should then be able to infer that Delphi holds the same potential, by analogy, for fishery researchers and managers.

FLORIDA SPINY LOBSTER DELPHI

Background

I first suggested that Delphi had potential use in fishery management in a paper given at meetings of the Gulf of Mexico and Caribbean Fishery Management Councils (Zuboy and Jones 1979). At that time the only natural resource oriented example I could cite was the previously mentioned Michigan Sea Grant Delphi. I knew that at some point I would have to try a Delphi experiment in a fishery situation to support my claims about Delphi's potential: hence the spiny lobster Delphi.

I selected the spiny lobster fishery, in particular the recreational harvest thereof, as a target for the Delphi experiment for several reasons:

1. it is a fishery with which I am familiar;

2. no conclusive evidence for any given recreational catch estimate exists;

3. the experiment could be kept simple; and

4. a supportable and acceptable estimate of the recreational harvest was needed.

One of my first projects in the Southeast Fisheries Center was to develop a data base management system for spiny lobster. Consequently, I became intimate with the spiny lobster literature, attended various lobster research conferences, and got to know many of the people involved in lobster research. I was part of the team which wrote the spiny lobster management plan for Puerto Rico and the U.S. Virgin Islands, and I also provided a formal review of the Gulf of Mexico/South Atlantic spiny lobster plan. Thus it was rather natural for me to target the spiny lobster fishery for the Delphi.

Many estimates (opinions) of the recreational harvest of lobsters have been presented. These estimates were usually vague, in terms such as “quite small,” “equal to the commercial catch,” or “greater than the commercial catch.” Based on these statements the implied range of estimates could be taken as, perhaps, 45 000-2 700 000 kg. Even the most current estimates, based on extrapolations of field survey and tagging data, vary by an order of magnitude: 27 000-270 000 kg. The point is that there is simply no conclusive evidence to support any given estimate, and hence the Delphi approach to obtaining a consensus as to the “best” estimate seemed appropriate in this case.

The experiment had to be kept simple because I was interested in obtaining not only an estimate of the recreational catch, but also in knowing how the method itself would work in this particular type of application, i.e., I didn't want the basic methodology to appear to be a failure if it was not at fault, simply because I made the experiment too complex. Therefore, I restricted the inquiry to answering only one question, for this was my first attempt at a Delphi survey.

In the past, speculative estimates of the recreational harvest were acceptable, even if not supportable, because management of the resource was not formalized. Since FCMA, and the requirement that commercial and recreational fisheries be treated equally in fishery management plans, the possibility of allocation among user groups exists. In the context of allocation, the need for a supportable (perhaps in court) and acceptable (to all groups) estimate of the recreational harvest is obvious.

Procedure

The first task was to identify a group of spiny lobster “experts” to participate in the experiment. I began by listing the names of people who I knew were engaged in spiny lobster research currently or who had done lobster research in the past. I also solicited the names of candidates from some of my colleagues. The initial list of 25–30 names also included a number of recreational and commercial lobster fishermen. After considerable deliberation I decided to limit participation in the “expert group” to persons currently engaged in research related directly to lobster management (as opposed to research on behaviour, for example) or who, by nature of their current jobs, have a fairly broad perspective of the lobster fishery. I wanted to have at least 8 participants in the experiment (the minimum number suggested by the literature). It was very difficult to select 8 individuals based on the criteria noted above. Part of the problem was that many of the individuals work together on the same projects or in the same office and thus would probably share the same data and opinions. This was a negative aspect in the sense of limiting the number of individuals available for the Delphi panel, but it was a positive aspect once the experiment was in progress because the selected experts could consult with their colleagues and obtain feedback on their estimate before submitting it in the Delphi. After selecting seven people for the panel I was having much difficulty selecting the eighth. Finally, again after considerable deliberation, I decided to participate on the panel myself. There are two ways of viewing my participation as a panel member. Since I was going to administer the Delphi there might be a legitimate question raised about my ability to maintain objectivity if I were also a panel member. On the other hand, as a panel member my personal opinion would be in full view (after the fact), and hence subject to scrutiny. Any tendency toward a “manipulated” consensus would probably be rather obvious.

Next, I contacted each of the 7 potential panel members on the telephone and explained what I wanted to do and how the Delphi would operate. Fortunately, they all agreed to participate in the study.

To initiate the first round I wrote a letter (Appendix A) to the panel members which reiterated how the Delphi would be conducted. With the letter was a separate sheet asking for their estimate of the recreational lobster harvest (Appendix B). The respondents were not given any indication of what a reasonable estimate might be— they were given a blank slate. The value of this approach is obvious. Respondents were asked to return their estimates to me within 5 days of receipt. A self-addressed, stamped envelope was included for their convenience and to encourage timely replies.

In the second round, participants were provided with the median and interquartile range (IR) of the Round 1 estimates and some additional information (Appendix B). Each panel member was asked to reanswer the question from Round 1, and if his new response was outside the IR about the median he had to provide a supporting statement.

In Round 3 the estimates from Round 2 were summarized as the median and range of responses and the supporting statements for estimates outside the IR were included. Panel members were asked to consider this “new data” from two viewpoints: 1) Did the new data affect their estimates, perhaps calling for revision? 2) Could they provide feedback that may cause the other panel members to reevaluate and perhaps revise their estimates? Rounds 4 and 5 proceeded similarly.

At the conclusion of Round 5, I determined that the estimates had more or less stabilized and that the experiment was over. I summarized the information from Round 5 and sent it to the panelists in a “final round” statement (Appendix B). At this time I asked the participants to send me their general comments on the Delphi technique and also to describe how their thought processes operated in regard to changes in their estimates during the course of the experiment. These comments are in Appendix C.

Results

The estimates of all participants for all rounds are summarized in Table 1. The range of the estimates decreased by a factor of four over the course of the experiment, i.e., the Round 1 range was about 900 000 kg while the Round 5 range was about 225 000 kg. The reduction in range is illustrated by the decline in the mean over the 5 rounds. The consensus produced by the Delphi inquiry is that the recreational harvest is between 235 000–455 000 kg, with the mean of the 8 experts' opinions being 345 000 kg.

Discussion

From my viewpoint, the spiny lobster Delphi was a success. It produced a “best” estimate of the recreational lobster harvest and also illustrated that the Delphi technique can be applied in a data analysis type situation as well as to pure speculative forecasting. I'd like to note a few points concerning the characteristics of Delphi, which should be taken into consideration by the potential Delphi user.

Although Delphi produces a “best” estimate, the user should recognize that the answer may not be unbiased (true). In fact, the estimate may be off by a relative order of magnitude in some cases. This is more likely to occur in a forecasting-type Delphi, however, than in a Delphi which uses the present as a time frame or one based on real data to some extent. The key to success in the latter case is in the selection of experts. Insofar as the experience of the experts is appropriate (to the problem at hand) and broad, the resulting estimate should tend to be unbiased.

Total anonymity of the participants may not always be possible. When applying Delphi in a specialized area, where the number of experts is limited, it is rather likely that each will have a good idea of who else is participating in the inquiry. Also, during the course of the inquiry, when supporting statements are provided, they will often reveal by inference the identity of the originator. This did not seem to pose a real problem in the lobster Delphi, however. On a couple of occasions a particular expert's affiliation, but not his name, was revealed in the supporting statements. Still, I had the distinct impression that most of the other experts were not sure who was providing the statement. In any case, I don't think anonymity (or lack of it) had any real effect on the outcome of the experiment. I think rather that the strength of the supporting statements had the most impact on narrowing the range of the estimates toward a consensus.

Table 1. Summary of Delphi estimates of spiny lobster harvest by divers in Florida for all rounds.^a

^a Estimates in 1 000s of kilograms rounded to the nearest 5 000.
^b My estimates
^c Estimate was given as a range of 455–910. I assigned a value of 680+ for convenience in the first round and asked for a point estimate in subsequent rounds.
^d Carried over estimate from Round 2 because no estimate was received for Round 3. Found out later that Round 3 estimate was 400.
^e This particular change from the previous rounds was a result of the expert completing analysis of all of his data, i.e., not a function of the Delphi process per se.

The usual procedure for narrowing the range of the estimates in a Delphi inquiry is to provide the median and interquartile range each round and require the panelists whose estimates are outside the IR to provide a supporting statement as to why they feel their estimate is correct. One can see the value of this procedure in forcing the estimates in the direction of consensus. The procedure apparently works quite well in the case of a Delphi which is based largely on expert opinion only, i.e., where there are no real data available. In the case of spiny lobster, I knew there were some data available from commercial landings, tagging studies, and surveys. Since the participants would generally be basing their initial estimates on real data to some degree, I felt it would create too artificial a situation to ask them to revise their estimates each time they were outside the IR. Hence I modified the basic procedure somewhat. I summarized the initial estimates as the median and IR and fed this information back to the panelists with some additional information (Round 2—Appendix B). I asked them to reevaluate their estimates based on the new data and provide another estimate. If their new estimate fell outside the original IR they had to provide a supporting statement. In the next round (Round 3—Appendix B) the range had been cut in half, and all the estimates on the high end fell within the original IR. Two estimates on the low end remained outside the IR. I decided to provide the median and new range (not the IR), along with the supporting statements, and to see how things developed from there. I felt that the range had already been significantly reduced, and that, if by the end of the experiment all the estimates were within the original IR, we would have a sound consensus. As it turned out, one of the original low estimates was revised several times and finally ended up within the original IR, the other original low estimate never changed, and one estimate which was within the original IR changed and fell outside the IR on the low end.⁴ Thus, in the final round, while two estimates were still outside the original IR, the original range had been reduced by a factor of four. Based on my knowledge of the individual experts and observing their estimates each round a general tendency was obvious. The experts who felt they had the best real data on which to base an estimate tended to stick with their own estimate or modify it only slightly, while the experts who apparently felt they had less real data tended to revise their estimates based on the new data revealed in each round. By Round 5 it was apparent that the estimates would not change significantly thereafter.

Maintaining objectivity was a problem throughout the experiment and would have been a problem even had I not participated as a panel member. I attempted to overcome or minimize this problem by always providing my personal estimate for each round prior to receiving any of the other experts' estimates. Also, in providing feedback, I attempted to use neutral language at all times. When supporting statements were to be fed back I provided them as written or with only minor changes to preserve anonymity. I only provided comments on the supporting statements (as an anonymous panel member) in one instance, and this was largely to address erroneous data or assumptions. My own original estimate of 340 000 kg, which is embarrassingly close to the final mean estimate, remained unchanged throughout the experiment. I felt I had a good estimate which was very close to the median from the beginning, and the cases presented by the other experts were not strong enough to induce me to revise my original estimate. While the potential for misusing Delphi to produce a manipulated consensus exists, I feel the danger is no more prevalent than in any other experiment. In the final analysis, it is the basic integrity of the investigator that will determine the results of an experiment, not the inherent characteristics of a particular methodology.

None of the panelists received any compensation for participating in this experiment. I had no budget and designed the experiment accordingly to demand as little of their time as possible. This Delphi asked only one question and became fairly complicated. No one should consider a more sophisticated Delphi inquiry without adequate time and funding, including consultant fees for the experts.

⁴ This latter change was a result of the expert completing analysis of all of his data, i.e., not a function of the Delphi process per se.

CLOSING

The Delphi technique has been demonstrated to be a useful tool and has been applied in the area of technological forecasting for approaching 20 years. Only more recently has mention of Delphi begun to appear in the renewable resources management literature (Regier 1978; Tomlinson and Brown 1979). The Florida spiny lobster Delphi is but a very limited example of one potential use of Delphi in fishery management. The greatest potential for Delphi in resource management lies in that very nebulous area dealing with “socioeconomic” aspects. Regier (1978) speculates that Delphi processes involving public participation in renewable resource management decisions may be operational by 1990. If this paper provides a small impetus in that direction it will have served a useful purpose.

ACKNOWLEDGMENTS

Sincere thanks go to “my experts” whose names appear below. Without their interest, cooperation, and expertise this experiment would have, obviously, been impossible.

Dr. C. Bruce Austin, Associate Professor, Economics Department and RSMAS, University of Miami;

Dean G. Barber, Marine Biologist, Florida Department of Natural Resources, Marine Research Laboratory;

Gary E. Davis, Marine Research Biologist, U.S. National Park Service, Everglades National Park;

Susan M. Foster, Marine Biologist, Florida Department of Natural Resources, Marine Research Laboratory;

Douglas R. Gregory, Jr., Biologist, School of Forest Resources and Conservation, University of Florida;

Peter Maley, Fishery Reporting Specialist, National Marine Fisheries Service, Southeast Fisheries Center;

James Tilmant, Management Biologist, U.S. National Park Service, Biscayne National Monument;

Captain Ralph Tingley, Marine Patrol, Florida Department of Natural Resources;

Gregg T. Waugh, Research Assistant, Rosenstiel School of Marine and Atmospheric Sciences, University of Miami.

(Austin-Waugh and Barber-Foster were collaborators.)

LITERATURE CITED

Fusfeld, A.R. and R.N. Foster. 1971 The Delphi technique: survey and comment. Business Horizons, 14:63–74.

Linstone, H.A. and M. Turoff. 1975 The Delphi method. Reading (Mass.), Addison-Wesley. 620p.

Ludlow, J.D. 1972a Evaluation of methodology in the University of Michigan's Sea Grant Delphi inquiry. Ann Arbor, Univ. of Mich. Sea Grant Tech. Rept. No. 22, 90p.

Ludlow, J.D. 1972b Substantive results of the University of Michigan's Sea Grant Delphi inquiry. Ann Arbor, Univ. of Mich. Sea Grant Tech. Rept. No. 23, 102p.

Regier, H. A. 1978 A balanced science of renewable resources/with particular reference to fisheries. Seattle, Univ. of Washington Sea Grant Pub. WSG 78-1. 108p.

Tomlinson, J.W.C. and P.S. Brown. 1979 Decision analysis in fish hatchery management. Trans. Am. Fish. Soc., 108:121–129.

Zuboy, J.R. and A.C. Jones 1980 Everything you always wanted to know about MSY and OY (but were afraid to ask). Miami, NOAA Tech. Memo. NMFS-SEFC-17. 19p.

APPENDIX A: INSTRUCTIONAL LETTER

Dear

You have agreed to participate in an experiment which will employ the Delphi Technique to estimate the recreational (diver) harvest of spiny lobsters from Florida waters. To refresh your memory, the Delphi Technique is a methodology for systematically developing expert opinion consensus. There are eight experts, including yourself, participating in this experiment. Throughout the exercise, individual responses will be held confidential. Thus, while you may know or assume who some of the other participants are, you will not know who provided a specific estimate. This anonymity feature of Delphi is important in that it tends to minimize the direct influence of one expert's opinion on another's and thus reduce bias.

The experiment will be conducted in the following manner. For the initial round, you will provide your best estimate of the recreational harvest of spiny lobster for the 1978–79 season. I will summarize (median and interquartile range of the responses) the results of the first round and feed them back to all of the participants. This will be your first opportunity to revise your estimate, if appropriate, based on the “new data” generated. On subsequent rounds, those individuals whose estimates deviate markedly from the median will be asked to justify their estimates. I will summarize the justifications and feed them back to all of the participants, once again eliciting reappraisal of individual estimates based on “new data.” By 3–5 rounds of this procedure the estimates should converge toward a consensus.

To begin the experiment then, please write your estimate on the enclosed form and return to me within five days. Please be prompt.

APPENDIX B: SPINY LOBSTER DELPHI⁵

Round 1

What is your best estimate of the total harvest of legal spiny lobsters taken by recreational divers in Florida waters during the July 1978-March 1979 season?

                         pounds

Comments:

Round 2

The results of Round 1 are summarized below as the median and interquartile range of the estimates.

One half of the estimates fell between 750 000–1 500 000 pounds. A quarter of the estimates fell between 112 500–750 000 on the low end of the range, and a quarter fell between 1 500 000–2 250 000 on the high end of the range.

Most of the estimates were based on a knowledge that the reported commercial catch for the same season was between 5,0–5,6 million pounds (depending on who you talk to!).

Please reanswer the question from Round 1 based on this “new data”. (Provide a point estimate, not a range.) If your new response is outside the interquartile range about the median (i.e., less than 750 000 or greater than 1 500 000) please write a short explanation of why you feel your answer is correct.

For your convenience, a copy of your Round 1 response is enclosed.

Please return within five days.

Round 3

The results of Round 2 are summarized below as the median and range of the revised estimates.

The median did not change, however, the range narrowed considerably. All of the estimates that were at the high end of the range were revised and now fall within the original interquartile range about the median. Two estimates at the low end of the range were outside the original interquartile range. The respondents have provided supporting statements (enclosed) for these estimates.

For Round 3 please examine the supporting statements and consider them from two viewpoints.

1. Do they provide information which affects your estimate, perhaps calling for revision?

2. Can you provide feedback which may cause the respondents to reevaluate and perhaps revise their estimates?

Please provide your Round 3 estimate and comments on the supporting statements within 5 days.

Round 4

The results of Round 3 are summarized below as the median and range of the revised estimates.

The median remained the same. The range narrowed somewhat because the lower end estimate increased.

Supporting statements provided in Round 3 drew comments from four respondents. For this round, all participants are requested to objectively evaluate the comments on Cases 1 and 2 (enclosed), and to reevaluate their estimates in light of this “new data.”

Please provide your Round 4 estimate and any additional comments within 5 days.

Round 5

The results of Round 4 are summarized below as the median and interquartile range of the revised estimates.

The range of the estimates has been reduced by about a factor of 4 as compared to the original estimates.

Based on comments received in round 4, the “Case 1” estimate was revised upward to 766 000.

The “Case 2” estimates remained the same for this round and a supporting statement in response to the comments and questions of Round 3 is included.

The estimate at the high end of the range was revised downward from 1 350 000 to 875 000. (This should have been reflected on the Round 4 handout, however, I did not receive the participant's estimate soon enough and simply carried his Round 3 estimate forward for Round 4.)

One participant, whose original estimate was within the interquartile range about the median, revised his estimate downward to 530,000 pounds. This revision was not precipitated by the Delphi responses, but rather was due to his completing a study on commercial tag return rate and subsequent analysis of all his tag return data. A supporting statement (Case 3) is enclosed.

One additional comment on Case 2, which supports an earlier “Comment on Case 2”, is enclosed.

The estimates appear to be stabilizing and we seem to be approaching the end of the experiment. Just hang in there a little bit longer!

Please evaluate all of the “data” that has been provided and send me your Round 5 estimate within 5 days.

Final Round

The estimates have basically stabilized and I have determined that additional rounds would not likely alter the present estimates significantly. The results of Round 5 are summarized below as the mean, median, and range of the revised estimates.

The median was used throughout the experiment in order to narrow the range of the estimates. It served that purpose well, as the range was reduced by a factor of 4. Now, however, the mean provides a more meaningful (no pun intended!) descriptor, as it incorporates all the estimates into an average value, rather than simply dividing them into half above and half below a certain value. The consensus, then, is that the true recreational harvest lies somewhere between 520 000–1 000 000 pounds, with the mean value of the eight experts opinions being 757 000 pounds.

As you'll recall, I was interested not only in obtaining an estimate of the recreational lobster harvest, but also in how the Delphi technique would perform in this type of application. To that end, I would appreciate your taking a few minutes to reply to the questions on the enclosed sheet.

I will send you a copy of the paper that will result from this study by December.

Many thanks for your interest, cooperation, and sticking it out till the end!

⁵ Editor's note: Weights given in Appendix B are in pounds rather than kilograms as this is the way the study was conducted.

APPENDIX C: COMMENTS OF PARTICIPANTS⁶

⁶ Comments were received from only four of the participants.