4. Discussion

4.1 Immediate impact

The majority of case studies concentrated on measuring the immediate impact of training on pesticide use and yield. Three studies used a combination of a longitudinal and latitudinal comparison, while four studies used a longitudinal comparison, and another six studies a latitudinal comparison. Remarkably, the first-mentioned group of studies had small sample sizes, which may have been a consequence of the combination of methods.

Generally, the case studies reported reductions, sometimes drastic reductions, in pesticide use attributable to the effect of training (Table 2). There was also a general increase in yield due to the effect of training. The range of results in latitudinal studies did not obviously deviate from that in the longitudinal studies.

Results for rice indicated that economic benefits were mostly determined by yield, not by pesticide expenditures which were small by comparison². Therefore, future studies should look more closely at yield effects. For vegetables and cotton, the contribution of pesticide expenditure are higher.

The four studies from Indonesia showed a considerable variation in results. This can partly be ascribed to sampling. For example, case 5 concentrated on sub-districts known for their strong IPM programs, whilst other studies used different selection criteria. Also, the contrast between treatment groups may have varied among studies due to differential levels of error (e.g. in identifying treatment groups; in over- or under-reporting of training effects) or bias (likely to increase with post-training period), or due to contemporary socio-economic conditions. One study reported an increase in pesticide expenditure (but not necessarily in actual use) in both comparison groups, which may be a reflection of the extraordinary inflation rate in the year of study. This variation among studies demonstrates the weakness of relying on single or incomplete data sources, even when simple parameters such as pesticide use or yield are considered.

Fewer studies have been completed for non-rice crops (e.g. vegetables, potato and cotton) than for rice. Nevertheless, Table 2 indicates that percentage pesticide reduction in non-rice crops was within the range of that observed for rice. When considering absolute pesticide reduction, however, a distinction has to be made between crops. In rice, pesticides were typically reduced from 1-3 to 0-2 applications per season, but in vegetables and cotton the reduction was from 3-7 to 1-3 applications per season³, whilst the market value of produce was higher for non-rice crops. Hence, absolute pesticide reductions and farm-level returns were higher in non-rice crops than in rice. Ongoing evaluations of vegetable and cotton IPM programs are expected to confirm this pattern, in particular in areas with gross pesticide over-use (baseline spray frequencies in excess of 20 per season are found in cotton and certain vegetables).

4.2 Developmental impact

Nine studies reported on developmental impacts of the IPM Farmer Field School, mostly using qualitative and open-ended methods, and in several cases by involving farmers in identifying and describing the impacts (Table 3). These studies emphasized social and political impacts. Only one study addressed impact on occupational health. No rigorous effort has been made to measure impacts on the environment or marketing.

The scope of studies ranged from the evaluation of learning processes, description of empowerment processes, to comparison of different approaches of extension. The studies reported the perspectives of farmers, project staff and external researchers.

It was found that complex knowledge on IPM acquired through experiential learning was retained, or even increased, in the years after training. Detailed case studies from Indonesia described that farmers were empowered by the training, in terms of increased self-regard, social skills and their active interaction with service providers, resulting in spontaneous activities, new structures and policy change. It must be noted that the Indonesian program had made considerable investments in follow-up activities (i.e. after Farmer Field School training) to encourage experimentation, community-based planning, farmer-to-farmer extension and networking. The resulting spontaneous actions and local programs were not isolated events but were widespread thoughout project provinces. In participatory evaluations, farmers identified what they most valued as impacts of training: an increase in creativity, independence, and collaboration, and lowered costs and improved incomes.

The described processes of empowerment and community action suggest that the Farmer Field School had an important trigger function, by introducing farmers to ecological and experiential learning methods, whilst enhancing group building and social skills. The exercise of agroecosystem analysis, for example, stimulated skills of thinking and communicating which could subsequently be applied to broader areas of people’s lives.

4.3 The added value of multiple perspectives

Each study had its own strengths and weaknesses. Some aimed to be statistically rigorous but were narrow in scope; others aimed to be comprehensive in scope but lacked geographic coverage and thus representation. Some aimed for aggregation; others aimed to capture diversity (in field conditions, in people’s responses, in impacts). Some were driven by donor interests, others by the values of local people. The complexity of impact evaluation of the IPM Farmer Field School implies that no single study can provide a complete picture of the reality of impact. However, when data sources from different angles with different objectives are combined, this improves our comprehension of processes that took place in the field, and provides triangulation by looking at patterns.

Eight data sources from Indonesia (Case 1-8) illustrate this point by using a variety of methods from photo reportage to aggregated data. The data show a large-scale impact on input use; spontaneous activities on a broad scale; declined pesticide sales in selected districts; increased farm-level profits; the indirect consequences of training in the lives of local people and farmer testimonies. This construction of perspectives improves the comprehensiveness and rigor of the overall evaluation. Also, the benefits are verified through different viewpoints. For example, pesticide reduction was confirmed by large scale assessments and case studies alike, and was evinced by lower pesticide sales and by organizational and policy changes. This cross-verification provides compelling evidence attributable to the effect of training. In contrast, the apparent lack of training effects in one study was less convincing because it was not verified through other perspectives and because of its small sample size.

The above underscores the importance of pursuing diverse approaches to evaluation and to combine the multiple perspectives, in order to construct a more rigorous and more comprehensive picture of impact as defined by the stakeholders. Ideally, the multiple data sources refer to the same time period and to the same farmer population.

4.4 The issue of cost

Directly related to achievements of the IPM Farmer Field School is the issue of cost. Because IPM conducted in smallholder cropping systems in the tropics is highly dependent on local context (or, generally applicable solutions are rare), it often requires that farmers improve their analytical skills and expertise. To help improve farmer expertise, hands-on education is needed for which there appears to be no shortcut alternative (see Case 24). But farmer education is necessarily labor-intensive and therefore can be costly, even though costs may be quickly recovered at the farm-level, through reduced input costs and increased yield. Unfortunately, no rigorous study has been conducted to compare human resource costs of the Farmer Field School and relevant other farmer education and training investments.

Reported costs per Farmer Field School-graduate are highly variable. A main factor contributing to this variation is whether program costs are included in the calculation of the cost per farmer. Clearly, costs will be higher in pilot projects than in programs that have been assimilated and decentralized within existing structures. Field-level costs are influenced by the level of incentives, travel, the involvement of farmers as trainers, and the level of local contributions. There are recent examples of self-funded and partially self-funded Farmer Field School programs from different continents, and this trajectory needs further exploring. The bottom line is that field training can potentially be conducted with limited sponsorship, which poses a challenge to gradually reduce external support in order to increase local program ownership.

Costs have to be rated against a program’s immediate and developmental impact, and how this impact contributes to national or development goals. Ultimately, investment has to be weighed against the cost of ‘doing nothing’. True costs of continued pesticide over-use include the full cost of the externalities associated with pesticide use (impacts on human health, soil, water and biodiversity), plus the costs of disposal of out-of-date and unused pesticide stocks. In this regard, government instruments such as taxation of pesticides could be employed to recover true savings which in turn could be used to support IPM training. To further reduce the fiscal burden of a program on one ministry, the institutional basis of the IPM Farmer Field School could be broadened, for example by recognizing the benefits of the Farmer Field School in the areas of education, environmental protection, and public health.

² In non-IPM farmers, or before FFS training, pesticide expenditure in rice involved approximately 3% of yield value (3.0% in case 5, 2.0% in case 7, and 3.4% in case 11).
³ In addition, several studies recorded a decreased chemical dosage per application and/or a shift towards less toxic chemicals.