10.4 Technical feasibility

Contents - Previous - Next

10.4.1 Assessing New Equipment Prototypes

Quantitative and qualitative techniques are used to evaluate technical handling and other aspects of equipment alternatives. Typically, open-ended qualitative survey techniques are used to evaluate how well the equipment handles, to identify issues, and to discover thoughts about changes that might be made. In some cases, these assessments are combined with physical measurements (e.g., efficiency of work indicators such as dynometer readings of pulling power and work rates for a draft animal harness, or precision of work indicators such as seed placement or seed damage rates for a planter) to complete this technical evaluation,

During assessment of the operational handling aspect of alternative equipment, strategies, etc., varied reasons for poor operational handling will be encountered, such as:

• Poor quality manufacturing,
• Inappropriate designs of prototype equipment,
• Lack of training in using unfamiliar equipment and strategies, and so forth,

When handling problems can be foreseen clearly by FSD staff or farmer participants, it is usually advisable to make corrections or necessary adjustments before undertaking statistical testing. Assessments of operational handling may not always require a control for comparison, Controls would not be very meaningful if the experimental operation was qualitatively different from other conventional operations in the farming system.

Informal one-on-one exchanges or group discussions involving farmer participants, FSD staff, and relevant experts are highly effective and informative methods of assessing the handling characteristics of alternative practices. PRA techniques, such as matrix scoring (see Section 8.4.4), potentially are ideally suited to quantify these informal assessments.

Even when the initial assessments of handling performance are negative, significant progress still can be made if insights and suggestions lead to better designs and pinpoint needs for instruction and farmer training.

10.4.2 Assessing the Technical Responses of Alternative Technology Options

Standard statistical techniques listed in Table 10,1, as indicated earlier (see Section 10,2), are the same as those used in station-based trial work (e.g., t-tests, analysis of variance, etc.) and are well covered in standard statistical texts, Therefore, this section simply covers some of the issues involved in choosing which statistical procedures to use in on-farm research analysis,

A number of helpful observations and suggestions have been made by various authors about quantitative evaluation techniques that pertain to FSD in particular, though no single set of standards has emerged yet, The following represent some of these observations:

• FSD generally will not involve itself in a technology fishing expedition. Statistically speaking, this means that FSD technical assessments will place much greater emphasis on estimating levels of technical responses as opposed to testing hypotheses of significant difference.
• The technical merits of technology tested in FSD should have been identified already during the ex ante screening of technology options (see Section 5,8.2). The relevance of estimating the magnitude of a difference as opposed to its statistical significance relates back to the fact that these estimates serve as the foundation for economic analyses and contribute to the formulation of judgements and opinions by farmers and experts. In consulting statistical references, the FSD worker will discover that estimating responses or the magnitude of a difference requires more observations than do simple tests of significance of differences.
• In most cases, multiple-mean comparison procedures, such as least significant difference (LSD) and Duncan's, should be avoided in FSD. These are procedures to explore for -- some might say to fish for -- differences when there are no firm hypotheses about which of several treatments might be superior for the measurements evaluated, In principle, FSD works with ideas obtained from farmers or from station-based research. Invariably, information exists about the technical merit of these alternatives but these are still hypotheses as far as these farming conditions are concerned. (e.g., screening a large number of varietal introductions for adaptation to a new area).
• Questions can be asked in research trials that will provide powerful answers, even in simple trial designs. A common interest in FSD is to evaluate two or more alternatives that can be combined and progressively form a 'technology ladder' (see Section 6.4.2). Meaningful comparisons can be incorporated into the trial design and the analysis for this purpose. It is always appropriate for a researcher to discuss his or her ideas, early on, with statisticians to see whether they might be improved; for example, making a trial simpler or in some other way less demanding on participating farmers.
• A common attitude is that analysis of variance and regression tests are robust enough to handle some departure from the test assumptions about the data (e.g., homogeneous variances, normal distributions, etc.) and, as a result, the possibility of serious problems is ignored. When working in a highly variable farming environment, this attitude might be dangerous. For example, large differences in variance -- the spread between treatments on different farms -- could significantly upset a stability analysis result (see Section 10.4.3). Data can be transformed to correct this problem.
• A grossly under-utilized tool in FSD research is covariate analysis, As FSD work increases its emphasis on situation-specific assessments, aspects of the situation can be determined, measured, and incorporated into the analysis of the response, These technical variables, or covariates, might be physical, such as values relating to soil or weather situations, or socio-economic, such as the likelihood of a resource being available. Covariates are quantitative and continuous in nature. Some of the apparently qualitative criteria used to classify farms into recommendation domains could be represented quantitatively and used in covariate analysis.
• The principal objective in FSD analyses is to predict responses to farming alternatives on specific farms, at specific sites, and under specific circumstances across the target area. If FSD analyses have a predictive function, then they would tend to be characterized as follows.

• Predictive technical analysis will most often be based on statistical regression and related statistical techniques. However, other options exist that may become more prominent in the future (see Section 10.4.4).
• Two important requirements for obtaining accurate predictions of technical responses are:

-- That the data represent a sufficiently broad spread of environments and of the factors that affect the response. This spread must cover the full range of circumstances for which conclusions are sought. It would be highly inappropriate, for example, to make general conclusions based on responses measured only in years with below average rainfall. To achieve this spread of circumstances, numerous authors have written that in FSD work, it will be more important to include a larger number of farms rather than devote FSD and farmer resources to replications within the farm or field (see also Sections 9.5.1 and 9.5.2).
-- That trial designs be streamlined, straightforward, and convenient, such that farmers are able to accurately implement and manage trials in a manner consistent with their own management style. In FSD, this is usually given as another argument for single replications per field or farm.

• FSD researchers need to be vigilant in looking for variation in responses during technical analysis and for the possibility of unanticipated causal factors, These circumstantial responses might enable FSD to fine-tune recommendations for farmers. At the same time, a warning is in order, An initial set of responses should not be used to draw anything but a preliminary hypothesis about non-treatment factors causing a response. Doing otherwise will lead readily to spurious conclusions, if the response is related to chance or to some factor not identified properly.

If FSD researchers discover interesting responses that were not considered when the original trial was designed, these results need to be confirmed. Confirmation might include some or all of the following procedures.

• Review results with participants. Participants with their managerial expertise often are able to verity or reject initial hypotheses,
• Evaluate the scientific logic of the hypothesis (i.e., possibly involving outside experts).
• Re-do the trial designed in a manner that the preliminary hypothesis can be tested.

10.4.3 Modified Stability Analysis

In FSD, modified stability analysis (MSA) or adaptability analysis, as suggested recently by Hildebrand and Russell [1994], is being used increasingly to assess the biological performance of treatments (i.e., technologies) in different technical environments. When used judiciously, this analysis also can help assess technical feasibility under different farmers' management systems.

• Basis for Modified Stability Analysis. To better understand how these analyses are used, the basis for stability analysis is briefly examined. The performance of all treatments fall, or rise, depending on the harshness of the environment. The performance of stable treatments, however, is less susceptible to the effect of environment. The prediction of stability is achieved through the use of a statistical procedure. Estimates for statistics obtained in this procedure then are used as indicators of stability. Regression techniques are used in modified stability analysis. To date, such techniques have been underutilized in FSD work. In many respects, because of the difficulty of maintaining ceteris paribus conditions in experimental work undertaken at the farm level, regression analysis is intrinsically more suitable than analysis of variance in such situations [Stroup et al, 1991]

Stability analysis needs data from a multi-environment test. The same set of treatments is tested in each environment. Treatments could be varieties, tillage systems, and so on. Because the testing required for this procedure is expensive, having treatments without proven potential in at least one environment is not cost effective. A range of environments is obtained by conducting the test at different locations and seasons within a country or target area. A range of environments also can be created within one site by imposing another performance factor. For example, if several irrigation regimes are implemented at one site, the set of treatments can be tested in each regime. The test needs to cover the entire range of environments envisioned by the researcher as targets for these treatments.

In on-farm research, multi-environment tests are common, The testing of a technology to improve grain yield on a range of farms, in different villages, and over several seasons gives excellent data for stability analysis.

Stability is measured from a simple linear ( i.e., straight line) regression of the performance for one treatment on the index values of the environments. The environmental index is a value that is supposed to define the standard for performance in each environment. To visualize the analysis, consider a scatter diagram, The indices for environments are put on the horizontal axis and the measure of performance on the vertical axis, The environmental index points are not spaced equally. Two environments might have nearly the same index. The straight line regression shows how much change in performance of the treatment takes place for each change of the environmental index unit,

Performance of crop production treatments usually is measured as grain yield, But, performance also could be measured by other factors, such as crop establishment, plant height, weed levels following tillage treatments, etc. Criteria important to the farmer also could be used (e.g., net return per hectare, etc.), Therefore, in a sense, modified stability analysis potentially could be used to assess more than technical performance,

The regression analysis is easy; obtaining good measures of environmental indices is not, Measurements of rainfall, soil depth, and soil pH are examples of indices that have been used in stability analyses. As an example, stability analysis could show how much the yield of a tillage system changes with each 25 millimetre increase in seasonal rainfall. More often, indices are calculated from the performance of treatments themselves. Suppose yields for double-plough planting are regressed on yields of single-plough planting in data covering 120 farmer-implemented comparisons of these systems. The 120 single-ploughing yields become 120 environmental indices. The regression can explain how much change in double-ploughing yield takes place with each 10 kilogram increase in single-ploughing yield. In this ploughing example, yield of single-ploughing is a good measure of environment when this system is the standard for that area.

Usually, in practice, the standard performance is measured as the average of several or all the treatments. Statistically, this is a problem, because Y. the dependent variable in regression, is part of X, the independent variable. Their values are automatically related. Nonetheless, the average of treatments often is used as a measure of environmental index. Take a test of four cowpea varieties conducted on three farms in each of three villages with the test repeated for three seasons. For each of the 27 environments, the index can be calculated as the average yield of the four varieties. Stability is assessed separately for each variety by regressing variety yield on average yield.

For stability analysis to be of use, the range of environments must be adequate. Even though regression can give a prediction of outcomes for environmental indices beyond those in the data, the prediction is not likely to be accurate for environments that are far outside those of the test. Data drawn only from testing in wet seasons with high yields cannot be used to assess stability over dry and wet seasons. Data for stability analysis need to include a minimum of one environment from each end of a typical range of the indices.

In stability analysis, three regression statistics are used as indicators of stability: the coefficient of regression ('b'), the standard deviation of this coefficient (s_b), and the treatment mean.

Because 'b' indicates the slope of the regression, it explains whether the performance of a treatment improves faster, slower, or the same as the environmental index, If the environmental index is expressed in the same units as treatments (e.g., yield), 'b' equals I when treatment and index increase at the same rate. The s_b indicates how much the researcher can rely on the regression relationship. A high s_b means that the performance of the treatment is erratic in good and poor environments. The mean, combined over environments, indicates the general ability of a treatment to perform well.

The stability or adaptation of treatments (i.e., technology) is defined as follows:

• With good stability or general adaptation, 'b' equals 1, mean performance is high, and s_b is low. Other researchers feel that stability is highest when 'b' is less than 1 (i.e., the performance does not change much between poor and good environments), mean performance is high, and s_b is low,
• With good specific adaptation, 'b' is significantly different from 1 (e.g., less if adapted to poor environments and more for better environments). To be considered good specific adaptation, the performance must be relatively high in the environments for which the treatment is adapted and the s_b low.
• Poor adaptation is indicated by low performance means, regardless of the regression coefficient ('b') or s_b.
• Erratic performance is indicated by a high ski, regardless of performance mean or regression 'b'. Note that the performance of a treatment sometimes appears erratic because the treatment is interacting with factors that are not parts of the environmental index (e.g., a new sorghum hybrid does poorly in an otherwise good environment because of an adverse reaction to a soil factor).

• Adaptability Analysis. Adaptability analysis can be used in FSD to help in dividing farms into recommendation domains (see Section 4.5 and Box 10.1 ). When a technology is tested under farm conditions, a number of characteristics of the farm combine to make a unique environment: soil type, farm management, and so forth. Adaptability analysis capitalizes on each environment's capability to cause a particular yield response to each treatment and then provides the data platform on which the researcher can seek 'cause and effect' explanations for these responses.

The steps in adaptability analysis are the following:

• Conduct a trial with a common set of treatments in each environment, Usually in FSD, each environment is a different farm. It is strongly advised that the set of treatments include the farmer's conventional practice as a basis for making comparisons with farm environments outside of the trial.
• Regress treatment yields on the environmental index of each farm.
• Differentiate treatments that are favoured in the high-yielding environments from those that are superior in the medium- and in the low-yielding environments. It should be noted that these differences, which emerge as differences in regression slopes, may not be real if statistical tests for difference are not significant.
• Characterize high- and low-yielding environments further by looking for a non-experimental factor (e.g., soil factor, climatic factor, etc.) that is linked to the responses. The practical expertise of FSD staff and farmers is needed to identify factors that logically should link to different responses of trial treatments. To evaluate the factor-response relationship in an observational manner, the factor values can be plotted on the same regression graph used in the adaptability analysis. It is advisable to discuss with a statistician other tests (i.e., analysis of covariance) that might be used to assess how strong the relationship is between a factor of the environment and the trial responses. The factor that is chosen then is used to define the appropriate recommendation domains for the technologies that were tested in the trial.

This method of sorting farm environments works quite efficiently, if the causes of high or low performance are constant from year to year. However, in some environments, variation in weather or some other factors exerts a large influence on the relative performance of many treatments, This means that the relative performances of treatments might place the farm in one recommendation domain in one year and in another domain in the next season, Therefore, the use of adaptability analysis would need more years of data in areas with highly variable weather than in growing environments with more equable weather.

BOX 10.1: MSA CAN HELP DEFINE RECOMMENDATION DOMAINS

Farmers in the Manaus District of Brazil clear forested land to plant crops such as cowpeas. Soil fertility declines quickly and, after a few seasons, the land is abandoned. Research was conducted to define appropriate soil fertilization recommendations for this zone and different circumstances in this zone [Hildebrand and Russell, 1994]. MSA was used for data collected on 13 farms that each tested four potential fertilizer packages. This included one package that was the farmer's own practice of not applying fertilizer. For these farmers, cash is scarce and the major constraint to development, so the focus when assessing response was on the criterion of kilogrammes of cowpea per dollar of cash cost.

Following a step-by-step application of MSA to this problem, two recommendation domains were delineated and factors within the environments that characterize these two domains were identified. The message for extension of this MSA is that:

• The farmers' own practice would be appropriate in the first and second seasons after clearing land of primary forest and in the first season after clearing land of secondary forest.
• Chicken manure plus one-half dose of triple superphosphate fertilizer would be appropriate in later seasons and on waste land,

Because of its high cost, a full dose of triple superphosphate fertilizer would be too risky for most farmers. If it is used, it should be used only for the first or second seasons of planting.

10.4.4 Computer-Based Simulation

As of today, the use of simulation tools is not common in FSD technical analyses. However, the use of such knowledge-engineering methods has proved helpful in evaluating alternatives in other fields. These simulations permit the users to evaluate multitudinous alternatives, at least on a preliminary basis, without investing in the implementation of real-life trials. Simulation scenarios could involve technology or management alternatives, changes in the future circumstances of farming, and so forth. Simulation programmes in the future might prove to be a very effective means of organizing FSD data. These programmes might strengthen the linkage between knowledge systems (e.g., allowing participants to effectively interface with scientific knowledge frameworks).

Simulation is a developing area, however, and good tools are not presently available for most FSD analytical work. Criticisms of contemporary simulation models for use in FSD focus on their inability to:

• Accurately predict even biological responses over the range of circumstances encountered in real farming systems.
• Address aspects of farming related to human behaviour.

Solutions to some of these problems likely will be found. It is anticipated that simulation programmes, probably supporting an iteration between machine and human participants, will become part of the arsenal of FSD analytical tools of the future, particularly with respect to FSD with a 'natural resource systems focus' and with a 'livelihood systems focus' (Section 3.3).

Contents - Previous - Next