Community Forestry Field Manual 1: GUIDELINES FOR PLANNING, MONITORING AND EVALUATING COOKSTOVE PROGRAMMES
prepared by Stephen Joseph, edited and designed by Carla R.S. Koppell
FAO, Rome 1990
Stove programme monitoring is a process which involves collection and analysis of data to ensure that a programme meets the objectives of the policy makers, donors, programme managers, and most importantly, the needs of the users, stove producers and sellers. It is an essential part of stove programme management.
Stove monitoring programmes can help:
Monitoring can begin in any phase of a stove programme. It is optimal if monitoring is initiated at the beginning of the programme and continues until the programme is completed. But, if a programme that is operational does not already monitor, it is always feasible and adviseable to begin monitoring.
Stove programmes can be divided into a number of distinct phases. In each phase different kinds of useful data can be gathered.
Appendix 1 shows how stove programmes develop. Informed decisions can be made using the monitoring information that is gathered.
Evaluation is a process whereby users, producers, project managers, policy makers and funders determine:
Much of the data that are used in evaluations are or should be collected during the monitoring programme. Evaluators can collect further information to verify this information.
Evaluation takes place at key points throughout the life of the programme. It can be at the end of the initial field testing phase, at yearly intervals during the extended field test phase and/or at the request of the donor (e.g. midterm reviews).
Evaluation is carried out using criteria and indicators. Criteria are principles or standards against which an objective or technology is to be judged. For example, users might decide that a stove design must meet the following criteria to be acceptable:
|Fuel Consumption:||30% less than the existing stove|
|Cooking Time:||50% less than the existing stove|
|Level of Emissions:||Eliminate all smoke from the kitchen|
|Aesthetics:||Improve the appearance of the kitchen|
Some of these criteria can be measured directly, e.g. fuel consumption.
If criteria cannot be measured directly then Indicators are used.
For example, to measure the changes in the appearance of the kitchen, users may decide that the following indicators should be used;
To assess whether a stove programme has met its social goals indicators have to be used.
For example, if a programme has the goal of increasing local participation in development programmes, some indicators might be:
|Criteria and indicators need to be established by users, producers, and stove retailers as well as project managers and donors.|
Examine the different criteria and indicators that different groups of participants suggest. It can help in planning, designing, monitoring and evaluating. Appendix 2 provides examples of possible indicators.
Evaluation can be undertaken by programme staff, outside consultants or personnel from the donor agency, and users and producers of the stoves. External evaluators can take a fresh look at the programme, and since they are not closely involved with its success or failure they can be more objective in their assessment. However, external evaluators may misinterpret the information they collect from the programme staff, producers and users, especially if the local staff find the external person threatening. Evaluation is most effective when a team comprised of external agents, programme staff, users and producers work together.
|BACK||TOP||TABLE OF CONTENTS||BOTTOM||NEXT|
Effective management of the stove programme depends on availability of specific data at specific times in the programme's implementation. To ensure the timely flow of information, careful planning and management is necessary.
To plan a monitoring programme there must be:
To properly manage the monitoring and evaluation of a programme, there needs to be:
Stove programmes have been most successful when the users and stove producers have been involved in the stove's selection and/or design, laboratory and field testing. Successful widespread dissemination of the technology is more likely to occur if users, extension workers and producers develop and test extension programmes together. It is important to closely involve entrepreneurs, artisans and small shop owners in the development and testing of marketing strategies when programmes use a commercial approach to achieve large scale dissemination. Effective participation is crucial to good management of a stove programme.
Over the last 5 years a number of programmes have developed methods for participatory management of stove programmes. Their experience has been synthesized to provide the following field tested approaches.
|Modify this programme to suit local needs and conditions. For example, the methods used in urban areas are usually different from those used in rural areas.|
At the beginning of the programme, establish a team including a project manager, a field worker, a stove designer, a respected artisan and a group of people who want to be involved in the stove programme in their area. The local stove committee, LSC, may, for example, be drawn from a local women's, social or religious organization, chosen by the village community to help guide the programme. In urban areas the LSC may include a stove factory owner, a distributor and a retailer as well as representatives from other local groups and users. The LSC's activities can be closely coordinated with the work of other groups.
The LSC carefully defines its role, how it is to operate and when it is to meet. It also decides how it is to communicate with the community. It is extremely important that the programme staff do not dominate. Their role should be to listen.
Because most people's time is extremely scarce and valuable it may be necessary to consider some type of payment for the time that is taken and the work that is done. This may be in the form of cash, food or provision of new stoves at no cost. A cash value assessment of each community participant's work should be used to determine appropriate payment. By assessing work time on an individual basis, the potential for conflict over "appropriate" payment is reduced. Care should be taken that the reimbursement does not cause resentment among other members of the community.
The potential users should be helped to carefully define their needs and develop a set of design criteria with the stove researchers. There are some basic questions that can help the participants determine their needs and objectives (see Appendix 3). In rural areas, villagers should also establish their own set of programme objectives, and the necessary criteria and indicators against which these objectives should be monitored and evaluated. In some urban areas, where there is a strong community development activity, local people should be involved in programme design. Appendix 2 provides information that can help guide the team so that it gathers the information that is necessary for producing a set of criteria and relevant indicators. This information is used at different times throughout the life of a programme. The information needs of different phases are outlined in Appendix 4. Also included in Appendix 2 is a checklist that is an example of an Initial Needs Assessment Survey. It can be adapted for use.
The villagers and urban dwellers should also help designers catalogue the existing stoves, measuring their performance, in both the laboratory and the households.
Members of the local stove committee, designers and extension workers should monitor the acceptability of the early prototype stoves. Regular meetings of these people and project management staff can help assure that necessary design changes are quickly implemented. Make all necessary modifications with the participation of the stove group. The community members, with the assistance of the designers, are often the best people to lead the modification process.
In the extended field testing phase the local extension agent or person in charge of marketing should report to and work closely with the local stove committee. If possible the LSC should have a major say in the appointment of the local extension agent. Experience from other programmes indicates that community meetings need to be arranged by the Committee at certain key times to resolve problems and/or retain the interest and enthusiasm of the rest of the community. It is extremely important that senior management attend these meetings.
The type of participation will probably change once a programme enters the dissemination phase. Programmes are most effective if a national stove programme is established and all institutions collaborate to monitor the national programme. It helps if all the institutions use the same or similar methods to collect and analyse data.
In urban areas, distributor and producer monitoring can be carried out by the programme staff or professional marketing consultants. Local stove groups, if they are interested, can have a major role reporting the effectiveness of the promotion campaign, the quality of stoves being sold in the market and the acceptability of the new stove. The individual consumer, however, should be encouraged to report on the acceptability of the new stove, possibly by filling out a warranty card or answering requests through the media.
In rural areas the form participation will take depends upon the types of stoves being promoted and the dissemination strategy being used. For the dissemination of mud and mud-lined ceramic stoves, monitoring can be carried out by village groups and extension officers or it can be assigned to a particular individual who receives special training. For portable ceramic stoves sold in the local market, monitoring can be done by district industrial development officers or a specially trained artisan.
In very small programmes (where the target is approximately 2000 stoves), the monitoring programme the LSC has outlined can be overseen by the programme manager. This person may have an assistant who is responsible for working with the community members collecting and collating the data from the field. During the initial field test phase the manager may have to spend a lot of time working with users and/or stove builders. Once dissemination of the stove commences the manager can spend less time with users and more time overseeing monitoring. The manager schedules regular meetings with the extension workers, users and producers and design team to analyse data and develop solutions to problems that have arisen. Problems and successes should always be discussed in LSC meetings. The LSC should work together to make decisions and choose solutions. The manager must insure that consultants who are sent to evaluate the programme meet and work with the LSC.
For larger rural based programmes a monitoring and evaluation unit may need to be established. The manager of this unit often reports directly to the programme manager and the LSC. The M & E manager:
The M & E unit will probably need to work closely with both the research and development, and the extension departments. The M & E unit will usually have an input into training programmes other departments run. Regular meetings will need to take place to quickly solve stove design and extension strategy problems.
M & E units often have one or two additional staff who should, if possible, have both technical and social science training. They can assist/ supervise the daily activities of other enumerators. Much of the detailed participant observation and measurement can be carried out by these trained social scientist/technicians. They can organize the regular review meetings with, and training of the field staff and the locals. They can assist, collating, sorting and analyzing the data and helping locals draft reports.
It is always possible for more local people to become involved. Different tasks can be assigned to small groups who, with the help of trained staff, can refine and implement the M&E process that has been outlined by the LSC.
In urban based programmes monitoring and evaluation is an integral part of a marketing strategy. Thus the manager of the marketing programme can oversee urban M & E.
STEP 1: Determine the objectives and principal monitoring and evaluation activities for each phase of the programme.
There are five major steps in monitoring and evaluation. The team can work together to develop a set of programme goals for:
Small programmes may not want to do all of these, however, it is strongly advised that b, c and d be undertaken.
STEP 2: Determine the minimum information requirements for each phase.
Appendices 3 and 4 provides information on minimum information requirements.
STEP 3: Determine the method of collection, storage and processing for each phase to determine the minimum information required.
Section 5 provides information on personnel needs and training for data collection. See Section 3 for detailed information on data collection techniques.
STEP 4: Determine what activities must be undertaken and when they should be undertaken.
Undertaking activities in the following order can help insure the effectiveness of the monitoring:
Graphics often clarify responsibilities, sequence, and timing of activities and relationships. Charts 3.1, 3.2 and 3.2 illustrate how different types of tables and charts can be useful planning tools.
|1.Initial staff training||---||--|
|2. Stove installation||-||---|
|3. Start monitoring||---||---||---||---||---||---||---||---||---|
|present & report results||---|
|4. Design stove alterations||---||--|
|5. Distribute redesign||---|
Fig. 3.1 A detailed bar chart for planning time allocation
|Main components of m&e work programme||Jan||Feb||Mar||Apr||May||Jun||Jul||Aug||Sep||Oct||Nov||Dec|
|5.Redesign & distrib.||-----||-----|
Fig 3.2 A simple bar chart for defining tasks
|Activity||Time needed||Human resources||Notes|
|Training||1.5 months||All: producers, users, promoters, designers, etc.||Phased: 2 weeks each group|
|Installation||1 month||Installers, Designers||Designers and ISC manage and oversee|
|Monitoring||ongoing from installation||Enumerators|
|Evaluation||3 months||Designers, Staff||Begin after collection of monitoring data.|
|Redesign||1.5 months||ISC, Designers||Integrate information from the m&e.|
Fig 3.3 Time and resource planning chart
A bar chart is the easiest method to display this information. Some activities must be carried out before other activities. Bar charts can be detailed to map out specific time allocation (Figure 3.1). They can also be very simple to show tasks that need to take place (Figure 3.2). A time planning chart (Figure 3.3)shows use of staff resources as it relates to the project's M & E scheduling.
STEP 5: Determine the amount of money, equipment and the personnel required to collect, process, analyse and report the information in each phase.
Once all the activities are specified, determine who is to carry them out, what equipment will be needed and the costs that are involved. Ensure that those people who are assigned particular monitoring tasks do not also have other tasks assigned for the same period. Figure 3.1 illustrates how this information can be laid out. The minimum resources requirements are:
STEP 6: Determine if the available resources are sufficient to collect and analyse the information.
Having calculated the number of people, days and the money required to carry out the monitoring and evaluation programme, determine if sufficient funds are available or could be made available. If it is apparent that the monitoring programme is too ambitious the type and quantity of information will have to be reduced or cheaper methods for collecting the same type and amount of information will have to be developed.
STEP 7: Develop an organisational structure that involves users and producers.
The importance of communication cannot be overemphasized. It can be helpful to hold review meetings with the headquarters and field staffs at least once every three months. The head of the M & E unit and the programme manager can meet at least once a week and make regular field visits together (if possible once every two months). Where possible the two should also attend review meetings with donors, civil engineers and/or politicians. If a consultant is permanently attached to the M & E unit, it can help if he/she regularly meets with the manager. Community groups, stove designers and other personnel involved in monitoring, can also meet.
In larger stove programmes report writing is a very useful form of communication both for internal use and for informing donors and the wider public. It is recommended that reports for internal use be short (under five pages). They should state the objectives of the particular monitoring exercise, the work undertaken, the results, and the short and long term action that is required. Guidelines to follow when writing a report are:
|1)KEEP IT SHORT||by emphasising key points and using tables to summarize information.|
|2)KEEP IT CLEAR||by planning the report carefully, avoiding technical jargon, using short sentences and illustrating points graphically.|
|3)EDIT CAREFULLY||Leave a day or two and let someone else look at the report and comment before doing a final edit.|
|4)SUBMIT ON TIME||This not only assures there is quick action to resolve problems, it also builds confidence in the M & E unit.|
At the end of each programme phase a longer report can be issued for general circulation. The following could be included:
|BACK||TOP||TABLE OF CONTENTS||BOTTOM||NEXT|
This monitoring system is based on the experiences of programmes in both Asia and Africa and can be used in small and large programmes. It involves data collection by local people working with stove programme personnel, and immediate data analysis of the data using a question and answer format. There are other approaches to data analysis that can be used with this monitoring and evaluation system. They are described later in Section 3. The participatory approach has certain advantages:
It also has some potential disadvantages:
Some programmes may wish to use a combination of the described participatory and more conventional techniques. Throughout the text relevant complementary Appendices are also listed.
|Adapt these techniques to meet the programme's needs. The following sections guide development of a programme-specific participatory monitoring system.|
At the beginning of the programme, work with members of the community, local artisans and entrepreneurs, extension workers, government officials, and the local stove or development groups to gain a profile of :
For a general list of the types of baseline information that will be needed for a complete evaluation in the later Initial Field Test Phase see Appendix 4.
Information can be gathered and verified if the stove committee develops questions which can elicit important information.
Have a member of the group that is familiar with the monitoring material serve as facilitator. The facilitator can ask a series of questions and then help the community members or the local stove group find answers. One person, who has already reviewed the information collected from the monitoring, acts as facilitator ( such as the head of the M & E unit from the stove programme). For example, the stove group finishes a survey on the community needs and is examining the data. The facilitator asks a series of question such as:
"How will the size and layout of the kitchens affect the design of the stove"
He or she then asks for possible answers from the group. These might be:
"I have noted that 8 out of 10 people that I visited, have very small kitchens in which the stove is placed in the corner of one room. Often the stove is against an internal wall so that heat can be provided to the rest of the house. I feel that the new stove should not be bigger than the existing open fire and that, if a chimney is to be used, it should be taken through some of the other rooms."
The facilitator can ask for other monitors' comments on kitchen size and its effect on stove design. The group builds up a picture and draws some conclusions on the shape of the stove. For example:
"We have now determined that approximately 80% of the people in our area will want a stove that has dimensions that are no more than 40 cm wide and deep. The other group of people, with bigger families and bigger kitchens want a much bigger stove."
After drawing this conclusion further analysis of the data can take place until a full set of design criteria are drawn up. This can then be passed on to the stove designers.
The following give an indication of the range of questions that could be used and their possible answers. There are other methods that can be used for participatory data collection and analysis. These are described in Section 4.
Kitchens vary from country to country and within countries.
They can be a part of the main house or a separate building. In very hot climates, preparation and cooking of food may be done outside during part of the year. Within the house the kitchen may be situated on the ground floor, the middle floor or higher floors.
Some kitchens are small, have very low ceilings and little ventilation. Other kitchens are very large and are well ventilated. Roofs are sometimes made from thatch and sometimes from tile or iron.
Some kitchens are very clean and have special storage places. Kitchens are organized in many different ways. In some, the stove is placed in the middle so that people can eat and socialize around the fire. Some people have special places for worship in the kitchen and some must put their stove in these sacred places.
The following are examples of questions related to the kitchen. Try to predict the answers; it can help design the stove and the strategy for field testing. Sample answers from the Division of Community Forestry Programme in Nepal will be given to guide the reader (Joseph 1982).
How does the position of the kitchen affect the choice and the method of installation for the stove?
How does the size of the kitchen affect the choice and installation of the stove?
How does the degree of ventilation and the level of lighting affect the method of installation that should be used for the stove?
How does the condition of the kitchen affect the choice and use of a stove?
How can the type of roof on the house affect the choice and installation of a new stove?
In most countries a range of stoves/fireplaces are used to cook food, prepare beverages and sometimes process animal feed. Households may have more than one stove. Different fuels may be used and different cooking operations may be performed on different stoves. Some stoves are fixed, others are portable. Some stoves must carry large round bottom pots that are vigorously stirred and others are used with light weight small flat bottomed aluminium pots.
Most stoves are very cheap to build or purchase. In urban areas they are usually made from metal and in rural areas from mud, ceramic or stone. Most ceramic and metal stoves are made by artisans, most mud and stone stoves are made by the users. In some societies stoves can only be made on special days and in others communities they must be blessed by the local priest. Some women prefer to squat when cooking, others prefer to stand. Some women are very careful tending the fire, others pay little attention to it (especially if they have demanding children with them).
Most simple stoves (e.g. the 3 stone fire) can provide both light and heat, use a range of fuels and accommodate a range of pot sizes. They are long lasting, require very little maintenance and if operated carefully have a relatively high efficiency. However, it is often difficult to burn wet wood and residues, protect the fire from wind (which considerably reduces the efficiency), prevent small children from burning themselves and prevent large volumes of smoke from being emitted.
What are the implications for the design and choice of different new stoves, given the range of stoves used by different ethnic groups?
How will diversity in the types of stoves (and the level of their social and cultural significance) effect the introduction of improved stoves?
In most societies a range of pots are used to cook food. Pots are made from clay, cast iron or aluminium and have either round or square bottoms with either flat or curved walls. Most families will have a range of different size pots and often these will be used for cooking different meals.
What are the implications of different pot types and sizes for the design/choice of the stove?
How will different pot types and sizes affect installation of mud and ceramic-and-mud stoves?
In most rural areas people use a range of biomass fuels. The type of fuel that is used depends on many factors including, price, people's income and occupation, availability of different fuels at different times of the year, and location of different types of fuel.Some people store wood fuel, others collect it every week; some people purchase their fuel either in bulk or in very small quantities.
Some species of wood are easy to burn while others are difficult. Some burn with a long flame, others with a short flame (especially charcoal). Some people chop wood into small sizes, other people use very large pieces.
How will different types of fuel affect the stove's design?
How does the type of fuel and the purpose of the stove affect marketing/extension strategy ?
The amount, type and number of times food is cooked each day varies between families, villages and regions in any one country. In some urban areas people find it cheaper to eat in restaurants than to pay for food and fuel. If people travel long distances to work, they will often cook food the night before, heat it up for breakfast, and take the leftovers for lunch. During harvest time cooking is often done once a day.
In some places rice or other grains are staples, in others root crops such as potatoes; in some places no meat is eaten, in others it is a major part of the diet. Some food is cooked quickly, others take many hours of slow boiling. Some people want two main dishes to be prepared at the same time, others will cook dishes sequentially. Some people use their stove for the preparation of food and drink for sale.
How will the range of meals that are cooked in any one household affect the design of the stove?
Similar profiles and questions can be developed for manufacturers, distributors and retailers. Other socioeconomic data can also be examined and the relevant questions formulated and answered. Once these questions have been answered the stoves' design is likely to meet people's needs and fulfill the criteria that have been established. The programme can now field test the stoves.
The project personnel and the local stove or community group will now have to decide those who will test and evaluate the first series of stoves. Involve a representative sample of households from all the different groups in the area. Place the stoves in the households and if possible, visit within one week of installation and then again within 3 months. Members of the local stove group (if available) should accompany either the local people employed to do the extension work and/or the outside project staff.
In-depth discussion should take place with the stove users during the visits. These discussions should be recorded in full, even if they seem unrelated to stove usage. Designers will also need to work with local people to collect information on the performance of the stove. Local people can increase project staff and stove committee understanding of the social organisation and interaction of the community. Hence, they can provide insights into the results of the data analysis. For example, a woman stopped using the stove, but when asked why, would not give an answer. However, she did insist that the stove was acceptable. During discussion she had mentioned that her stepmother had just come to live in the house. One of the local people involved in the monitoring felt that the stepmother was probably the reason why the woman had stopped using the stove. Later the social scientist asked if the arrival of this person had influenced her decision to use the stove. She said yes and explained that she felt too embarassed to answer the question in front of other local people. After collecting data from all the houses, the answers to the questions should be reviewed and analysed by the stove designers, programme managers, local extension workers and as many members of the community as possible. Section 3 provides several methods for group data analysis.
The following set of questions, answers and actions are given as a guide to develop an interview checklist and then to analyse the results. It will be necessary for each programme to make modifications and additions.
Why are people not using the stove ?
|Possible answers||Possible actions|
|it cooks slowly||
|it uses too much fuel||
|it produces too much smoke and is difficult to ignite||
|not enough light and heat||
|cracking or breaking of walls||
|is uncomfortable to use||
|built in wrong position, or time of year, death has occurred, or women/men were not consulted||
It should be noted that some of these actions may conflict. For example to increase the speed of cooking you may find that the fuel consumption is increased. The users need to work with the stove committee and set an order of priorities.
What do the users best like about the stove ? What are their indicators for success ?
What changes to the users' kitchen management practices have been noted as a result of the new stove's introduction ?
Women have made many different changes to their kitchen management practices. Some households have built a new, more spacious kitchen as a direct result of the stove's introduction. Others have built more storage and preparation space around the new stove. Some have started boiling more water for drinking and washing (as they have more time and fuel). In Guatemala, women have been able to cook their beans in the retained heat of the large mud stove overnight, instead of early in the morning. In some African countries women have been able to spend more time caring for their children while the food is cooking (the new stove requiring much less attention than the open fire).
The improvements to the kitchen that have been made can be publicized. Incorporate the methods women have used to improve the kitchen or cooking techniques in the training programmes. Organisations that are involved in women's development and housing improvement programmes can be informed of these changes; they may introduce them into their programmes.
What improvements do people want?
The answers and actions will basically be the same as those presented in Question 1, however additional possibilities could be:
|Possible answers||Possible actions|
|reduce outer wall temperature||
Once an acceptable stove design has been developed the programme can develop either an extension strategy for stoves that are to be distributed through a government or non-government agency, or a marketing strategy for programmes that are to use small industry and business to produce, distribute and sell the stoves.
The technique for collecting information in this phase will be different for stove programmes based in rural areas that use an extension strategy, and those in urban areas that use a commercialisation strategy. For rural programmes, monitoring staff, stove producers, extension workers, stove installers and members of each village stove committee will have to agree on a series of questions that need to be asked and then, if possible, participate actively in the monitoring. Analysis of the results can take place in each village using the techniques described in STEP 2 and/or later in this Section.
In urban programmes much of the monitoring can often be undertaken by the people involved in marketing (although some programmes have a separate monitoring unit). Monitoring forms and questionnaires can be developed in collaboration with the stove committee. It is unlikely that most of the members will have the resources to participate in the actual collection of data. The committee should help the personnel involved in monitoring select the sample, develop strategies and techniques for information collection and analysis and formulate a plan of action to improve implementation of the programme and the design of the stove.
In the dissemination phase other questions regarding reasons for adoption/non-adoption need to be answered. For a list of information projects could gather during the extended field test phase see Appendix 4.
Why are people not adopting the stove? What can be done to increase adoption?
In urban areas:
|Possible answers||Possible actions|
|have not heard of it||
|can't find it||
|unconvinced of time/fuel savings||
|don't like look||
|uncertain of lifetime||
|can't accommodate all pots or can't cook all types of food||
|husband not convinced of benefits||
In rural areas:
In rural programmes, especially those using a commercial strategy, the same answers will be heard from users and the types of action can be taken. However, in programmes that use an extension strategy, there can be other reasons why the stove is not adopted; these are specific to both the type of stove and the extension strategy being used. For example:
|Possible answers||Possible actions|
|extension agent has not visited||
|extension agent has not properly considered users' needs||
|extension agent is seen by users as poor on follow-up||
|requires new kitchen before will adopt stove||
|a death has occurred in the family||
|waiting for a subsidized stove||
Why are people adopting the stove?
Possible answers and actions:
There are many reasons why people adopt a stove. Most frequently it is because they remove smoke, reduce cooking and/or cleaning time, and fuel consumption (which results in a reduction in the time spent collecting fuel or in the amount of money spent each month). Stoves may also be adopted because they improve health, reduce injury to children, increase status or self-worth (a woman may exclaim that she feels really good after buying a new stove), because it is part of a social group's development programme or because of peer pressure.
Managers must fully understand the different reasons why different groups of people are adopting the stoves and reorient their promotion programme to increase the rate of adoption. It is extremely important to try to predict the increase in the adoption rate for a given increase in promotion since the necessary resources/personnel must be available to meet the demand as it increases.
Is the stove programme meeting its operational objectives ? If not, what additional resources are needed?
Possible answers and actions:
In any programme there will be several operational objectives that may be social, technical, economic and/or institutional.
Many of these objectives are related to targets which could include:
Most of the reasons why a programme reaches or does not reach its targets have been given in the preceding sections. For example, consider Objective 1 above). The reasons why the programme may have failed to reach its production and distribution targets and the actions that are needed to remedy this could be:
|Possible answers||Possible actions|
|insufficient incentive for producers / retailers / distributors||
|poor production management||
However, targets may also not be reached because of inaccurate project appraisal and poor planning. If unrealistic targets and objectives have been set, use these as a lesson when designing the dissemination phase of the programme.
Is the new stove having an impact ? Which impacts are beneficial; which are detrimental?
Criteria and indicators for assessing the stove programme's impact need to be developed by local monitoring groups along with other producers and distributors. Once criteria have been established, the answers can usually be ascertained from the data that were required to answer the other questions. For example if the group wants to find answers to the question "What is the impact on producers?"
They can find the necessary information from Questions 3 and 1. The answers from this impact evaluation can assist development of the strategy for the next dissemination.
At the end of the extended field phase an evaluation is usually carried out either by the project itself or by the donor. Since the programme has been monitored by the users, producers and programme staff, an internal evaluation could involve the following (this technique has been used by an NGO in India):
Discussion (managed similarily to intermediate stage evaluation discussions) can answer these questions:
The community and donor may bring in an outside consultant to help in the evaluation. If so, it is strongly recommended that this person be versed in participatory techniques. The outside consultant needs to first become familiar with the project and meet some of the local people that were involved in its implementation. The consultant can review all the previous reports and write a summary. This would then be presented at the evaluation meeting for comment and criticism. The consultant can finalize the statement and it can be presented either individually or as a part of the final report.
If the project is expanded, a monitoring system will have to be established. If the dissemination is to be conducted by other small organisations then the monitoring system that was developed in the previous phase can be used. The coordinating organisation can train the other organisations in the techniques that have now been fully developed and tested. Training can be adapted to suit the needs of different organisations. However, it is recommended that a participatory approach be maintained.
The information needed to monitor a dissemination programme is the same as for the expanded field test phase, however the priorities placed on collection of different types of information will change. Thus in the dissemination phase the programme management will be more concerned with collecting information on:
Thus the programme will collect more information on categories a to l in b) of Appendix 4.
In larger urban and very large rural programmes that use a commercial strategy it may be extremely difficult to use a participatory approach in all areas. Field experience in both Africa and Asia suggest that information on sales, production quality, output, and the number of new outlets being established are more effectively collected and analysed by a full time project person. There appears to be little need to set up a number of stove committees. However, experience in Sri Lanka suggests that regular meetings between producers, distributors and stove staff should be organized. During these meetings the monitoring data can be reviewed, and further data can be collected during discussion. At the end of the meeting a set of proposed actions for improving the programme can be drafted by all the participants. In both the urban and rural settings monitoring can gain information which answers specific questions:
One method that has been successfully used in Africa to interpret the information collected by members of the stove group is role playing. There are a number of different techniques that have been developed. The following is an example of how a role playing session could be run.
The same role play can be refined and presented as a play to villagers to help enlist support for the stove group's work.
Verbal interaction method
Members of the stove group are provided with a cassette player (or use their own) and a chairperson is chosen to help direct the discussion. All participants start their recorders and the chairperson asks individuals to report their findings and suggest methods of improving the stove and/or the extension programme. No conclusions are drawn during the first meeting. The group chooses to meet within a week and agrees that each individual will listen to the tape a number of times before the next meeting. At the next meeting each member is requested to give their interpretation of the data that has been presented by the others. The chairperson then tries to draw conclusions from the ensuing conversation and the group produces some type of graphic output.
The results of monitoring can also be analysed numerically and graphically. In the case of stove testing it is essential that the average of the tests and the standard deviation be calculated for both the new and the old stoves. A special test, known as the student t-test can then be applied. This test will help determine if the difference in performance between the 2 stoves is statistically significant. Details of this test can be obtained from a statistics textbook. The student t-test can be done on a calculator containing an appropriate programme or on a computer. If calculator processing is not used, information from the sample surveys must first be tabulated, and the frequency of a particular observation calculated by hand. For example, a breakdown of the types of repairs that have been done on new stoves could be presented as it is in Table 3.1.
|Type of repair||# of stoves||Percent (%)|
|Repair not required||53||29|
3.1 Number of stoves repaired
More detailed tables can also be constructed. For example, in the Nepal Programme the managers wanted to know, if during the initial field test phase, there was a relationship between acceptance of the stove (measured by how often the stove was used) and the village where the stove had been introduced. Table 3.2 on the next page was prepared and a special type of statistical analysis, a Chi Square analysis, was performed (the interested reader is referred to a standard text on statistics for further information).
|Village||Exclusive||All meals||Most Meals||Infrequent||Not Used||Total|
|For each row, percents are on top and counts are below.
DF = 8, Chi Square X2= 73.52
3.2 Level of acceptance for three villages
The Chi Square analysis indicates that the stove had much greater acceptance in Chapagaon. Previous monitoring had revealed that the main difference between this village and the other two was that in this village fuelwood was scarce. The programme thus concluded that there was a much greater chance of stoves being accepted in villages where fuel was scarce.
The clearest way to present data is graphic. The histogram on the next page (Figure 3.3) clearly shows the number of stoves being used (as compared with the number introduced) in 3 different socioeconomic groups. It can be seen that the middle class people have higher rates for both stove adoption and use than the lower and upper class people.
3.3 Stove usage versus socioeconomic group
3.4 is more complex. It summarizes the major complaints that different groups of people have about the stove. The lower status groups, where time is very scarce, find that the stove's major fault is that it cooks too slowly. Alternatively, high status group are more worried about the stove's appearance.
3.4 Graph of user complaints for different socioeconomic groups
Figure 3.5 indicates that the programme is not meeting its production targets. This graph can be easily understood by both community members and programme staff and can be prominently displayed.
3.5 A plot comparing monthly stove construction targets with actual rates of construction
Figure 3.6 is a Pie chart that illustrates the charcoal consumption patterns of people who have and who have not purchased the new stove. The data indicates that most adopters are people who use charcoal as their main source of fuel. Non-adopters use primarily other sources of fuel such as gas, kerosene and wood.
3.6 Charcoal consumption by new stove adopters and non-adopters
|BACK||TOP||TABLE OF CONTENTS||BOTTOM||NEXT|
Four main methods of data collection have been used by local people, programme personnel involved in monitoring, researchers and external evaluators:
These methods are complementary and can be used to answer the questions that were posed in previous chapters. As has been demonstrated in the previous sections different combinations of techniques are used during different phases of programme implementation. The type of information that is required will affect the choice of method. Each requires pretesting by experienced people before the technique is used on a large scale. Pretesting involves testing of the measurement techniques, creation of observation sheets, check lists, interview lists or questionnaires to determine if:
In this section the techniques and requirements for each method are outlined, giving specific examples. The techniques that are most common and/or most appropriate for use in each programme phase are also identified. The difficulties associated with the implementation of each approach are highlighted.
Local participants, designers and field workers must have accurate information on fuel consumption, quality of production, installation, stove deterioration over time, amount of ash build up on the stove, levels of pollution, and innovations that users have made. Measurements can be taken during the survey or as a separate exercise, but it needs to be remembered that direct measurements are intrusive and in some cases can be either offensive or induce fear, especially if people are occasionally harassed by other groups or government officials. It is important that people receive detailed explanations of the study's purpose and the method of the observation. Taking direct measurements is time consuming and enumerators require considerable training.
Care must be taken when measuring fuel consumption in households, if data are to be both representative and accurate. The sample should consist of both stove users and non-users. Recent experiences in Kenya and Sri Lanka indicate that the following procedure can be used to get reasonably accurate results (for more extensive discussion on woodfuel measurements see Woodfuel Surveys, FAO 1983):
There are two requirements when undertaking measurements. The instruments should be accurate and reliable, and the sheets to record the values must be clearly designed and contain sufficient information to assess variations in the results. Careful instructions must accompany the sheets and a second person should check the measurements in the field. Standard sheets and procedures have already been devised for measuring fuel consumption (Appendix 9). When measuring levels of pollution the procedure developed in Nepal and that of the East West Center (1983) can be used. When measuring the stoves, create a sheet that has both a dimensioned sketch of the stove and spaces to provide the desired dimensional information. The picture can be used to illustrate the exact position of cracks, the position and quantity of soot, and the places where the dimensions or installation are faulty.
When assessing quality of production, it may be necessary to take a random sample of stoves and measure their dimensions. These can then be compared to the project standard. Deviations from correct measurements can then be calculated. Producers should be visited regularly and changes in quality can be assessed over a period of time. Measure changes in the dimensions of the stove in a representative sample at regular intervals for a period of at least one year. During these visits observe changes in kitchen layout, technical changes made to stove design, recent maintenance or repair. Field trip notes should always be filed and continually processed.
It is important to develop special books or registers that allow easy data entry and retrieval. The number of stoves produced, distributed and installed, the number of training sessions or community meetings, the number of new producers and distributors and project expenditure should all be recorded in registers. This is the operational data that are vital for determining if targets are being met. These books should be collected at regular intervals so that the data can be collated in a central registry.
Participant observation is a technique used by researchers. A researcher lives with either one or a small number of communities. Over a period of weeks or months the researchers try to integrate into the community so that they can get unbiased detailed information through observation, structured, and unstructured interviews. They usually collect and cut wood and cook, gaining a greater understanding of the issues that face the stove user. The information they obtain can be on local values, politics, social structure, factors affecting decision making, attitudes to development issues, status of women, time allocation, cooking and fuel gathering practices. At the beginning of research, develop a checklist of the data that are needed. This information is collected over a period of time that the programme managers must specify. The longer the period, the more in-depth and wide-ranging the collected information can be.
The researcher sets up a filing system for recording information from key informants, and details of other events. Key informants are people who have a particular knowledge about a subject or who are opinion leaders in their community. The researcher, serving as facilitator, may also get groups together to discuss certain issues related to use and adoption of the stove. The research may be more or less time-consuming, but it can provide essential background and back-up information. It allows in-depth analysis of a particular event or result, but does not allow generalisation. It may help managers determine why a particular district has a low or high rate of adoption, why the community is or is not actively involved in the programme, or why fuel savings is lower than expected. It can also provide detailed information on the programme's impact. The sex or age of the researcher can bias the data (e.g. a young man may be considered too young, a woman may not be allowed to attend council meetings. Men may not be able to talk with women or to actually use a stove).
Non-participant observation is a relatively non-intrusive method of data collection that can be done without the active participation of the people being observed. Examples of non-participatory observation include:
Observation can yield an instant picture of what is actually happening. Additionally, the chance that the subjects' are changing their behavior patterns due to the presence of the researcher diminishes. The information is however, susceptible to observer bias and the amount of data that is collected can be small compared with the resources required. Non-participant observation is an extremely good method to use to validate and complement information gathered from surveys.
Case studies use participant observation to do an in-depth study of a few households. Local adults or school children and/or researchers live with a few families over an extended period of time and obtain detailed information on patterns of fuel consumption, changes in time allocation and patterns of economic activity. Local or external researchers can gain access to a number of households that are representative either of different socioeconomic groups, or of different cooking patterns. They work with the women over a period of at least one month, helping carry wood or cook food. During this time they measure consumption, time allocation, observing cooking patterns and decision making within the household. Additionally, they have detailed conversations with the women of the households. From these case studies a preliminary hypothesis is developed that can be tested in the larger surveys. The tabulated results from the case study can help determine if there is the possibility of major error in the survey.
This method suffers the same drawbacks as participant observation. The sample may not be representative and a considerable time period is needed to collect and analyse the information. However, the researcher will be able to provide the programme manager with real insights into why certain patterns of usage and adoption are being observed in a particular area.
These are the most popular means of obtaining information in stove programmes. Surveys are used to collect information on rates of adoption, usage and impact. A survey involves interviewers asking representative individuals and groups of people (a sample) a series of questions in a particular area.
A sample is a representative subset of a group of things, people or districts known as the population. A population can be divided into units, either people, households or villages. The list of sampling units is called the sampling frame. In the early stages of a stove programme all the households can be monitored. As the programme introduces more than 200 stoves and expands into new areas it is not possible to do detailed surveys of every households. That is when sampling becomes necessary.
|If you have not received training in sampling techniques seek help.|
Choosing a sample is not an easy task. When cooking and fuel use patterns, income levels, religious affiliation etc. are very diverse, a careful selection process must be devised. In this case the input of a trained social scientist or a statistician is required. However, for those who have already received training in sampling, a number of practical examples follow. For this method to work, a distribution register must be maintained.
|EXAMPLE 1||The sampling method used by the Department of Community Forestry in Nepal.|
|Step 1:||Construct a complete list of all households who have received improved stoves, making separate lists for each year of stove distribution. Number sequentially each household on the list beginning with 1.|
|Step 2:||For each year, divide the total number of households by the total number to be surveyed, to obtain the Sampling Interval (SI), e.g. 693 households divided by 150 sample size equals 4.62 SI. The decimal points should be retained to two or three places.|
|Step 3:||Select a number (R) between 1 and the sampling interval (SI) from a random number table. The household whose serial number corresponds to this random number is the first household in the sample, e.g. R = 2.|
|Step 4:||Successively add the sampling interval to the first random number and round off the results until the end of the list is reached. Each of these rounded off numbers corresponds to the serial number of the household selected, e.g. R = 2, SI = 4.62.|
EXAMPLE 2 Sampling in an Urban Environment
There are two types of questions that can be asked:
open ended e.g. Does the stove do its job or not ?
and closed e.g. Do you use your stove for cooking:
In the latter case people are asked to choose from a small number of alternatives, thus providing fairly precise information that is easy to analyse. In the former, people are asked for an opinion and the possible answers are very diverse. The answers from open ended questions can be used to develop closed ended questions. They also provide much more insight into the differences in opinion between households. Usually both types of questions are included in a survey since they are complementary. Closed questions are easy to ask and process, but they limit the type of information that is obtained. Open questions give useful information, but in a form that may not be easy to process. Questions should follow in a logical order to maximize the amount of information that is elicited. They should fit the local perceptions and use the people's understanding of the way things function.
It is important not to ask direct questions. Ask indirect questions that will elicit the desired information. For example, instead of asking 'does your stove save fuel?' ask
The quantity and quality of the data will depend on the effectiveness of enumerator training, motivation, and the rapport that enumerators can establish with the people being interviewed.
There is a tendency to ask too many questions. Much of the data is never analysed. Experience suggests the following procedure:
Each method that has been mentioned gathers specific types of data. However, the most appropriate data collection techniques can vary based on the stage of the programme and the types of information that are needed.
In Sri Lanka, field extension workers have generated baseline data by recording users' observations, discussions with key informants and group interviews. They have also examined locally available documents using the following procedure: Before visiting a district :
After arriving in the district :
Four main data collection techniques can be used by community groups, extension workers and designers.
Log book system:
At the beginning of the programme, local organisations can establish a very simple log book system. This records who has received the stove and when, how many people participated in its construction or installation, how much was paid by the recipient (or food provided to workers), how much time was used for construction. Appendix 5 contains one example of a log book design.
In some programmes a volunteer working with the local stove committee or a full-time stove worker, makes regular visits to the households. During these visits the worker conducts an informal interview and observes the household. These can be recorded in the log book. Depending on the skill of the interviewer, considerable information can be obtained by talking generally about needs. This can be recorded at the bottom of the log book page.
Minute record system:
A local organisation can also establish a simple system to record stove related activities. This can consist of records of minutes of meetings. An example of a simple minutes book is given in Appendix 5.
A diary of local events can be kept. For the two villages in a project funded by the Ford Foundation in North India, the project leader kept a detailed diary of people's sense of changes in fuel consumption, cooking time and kitchen appearance in the two villages where the field programme was initiated. She initially conducted boiling water tests by recording time water took to boil and the amount of water boiled by identical quantities of wood in traditional and new chulas. In 14 households (seven with old and seven with new stoves) the weight of wood used over a six-week period was recorded by literate children under the project leader's close supervision. Tests were assessed to determine if the new stoves were superior to the existing ones.
In Nepal, RECAST informally monitored stove acceptance in one village by frequently visiting households to discuss any problems. The stove programme manager kept a diary of events. Approximately 15 households completed water boiling tests. At the same time, baseline data was collected by questionnaire from households both with and without the new stove. Follow-up water boiling tests were done after stoves had been in use for six months.
Controlled testing method:
In programmes with considerable resources the following data collection method can be used.
Approximately 50 households are chosen at random to participate in the initial field test phase. A known weight of fuel is given to each household. Consumption of fuel is measured one to two weeks before introduction of the stove and one to two weeks after. Socioeconomic data on each family is collected during this testing. Details of the recommended procedure are provided in Appendix 8. In some cases it has been difficult to follow this procedure due to cultural values. Instead of measuring the fuel, it is possible to question the amount of money that is spent on fuel, frequent cooking tests can also be done. Again, tests are carried out six months after the stoves are installed.
As well as testing, the stove team can also monitor response to the stove. Either a log book or a diary can be used.
Surveys are commonly used for data collection in the expanded field test phase of field programmes.
In Sri Lanka, a questionnaire was administered to 200 of the 500 households that had a new stove. Members of the stove unit, who were chosen from surrounding villages (not trained enumerators) administered the survey. Initially, the questionnaire contained 30 separate observations /questions. Careful pretesting indicated that only 13 observations /questions were needed to obtain reliable information on patterns of usage. The questionnaire was very simple, yet collected vital information. Much of the information was obtained from observation and most of the questions were closed ended. This questionnaire is reproduced in Appendix 6.
Checks were built into the questionnaire. For example, if the interview was carried out after breakfast, the new stove was cold, the traditional stove had not been removed, and the user answered that the new stove was used for all meals, the enumerator tended to discount the reply. Data on the quality of stove construction was collected. This provided a check on users' perceptions of savings since a poorly built new stove was often less fuel efficient than the pre-existing stove. In some households, people would indicate reasons for satisfaction or dissatisfaction with the stove. This was noted at the end of the questionnaire under the 'comments' heading.
Data storage and analysis of information from this short questionnaire was easy. However, while it did provide an accurate picture of design problems and rates of usage, it did not provide in-depth information on why certain patterns of usage were being observed in certain areas.
If possible, this type of survey can be followed by in-depth interviews. This involves choosing four to eight households that represent both a range of usage patterns and socioeconomic groups. A trained social scientist can spend a few days with each household building up a detailed picture of cooking practices, time allocation, decision making practices, patterns of fuel use, etc.
Some programmes, e.g. Nepal Community Forestry Development Programme, have more resources available and thus the surveys are much more complex than those carried out by the smaller programmes. An example of a detailed survey is presented in Appendix 7. Questionnaires are developed either by the Monitoring and Evaluation Unit or, in some programmes, by the manager. They are first pretested in 20 - 90 houses to determine which questions are redundant, ambiguous or poorly worded.
The Nepal questionnaire tried to determine the amount of time a stove was being used, how much fuel it consumed, and why a stove might not be used. Since detailed baseline data on particular households were collected at the beginning of the field programme, usage patterns could be related to such variables as household size, stove type, ethnic group, traditional stove type, stove condition, frequency of maintenance, accuracy of installation, pot size fit, perceived fuel savings, cooking time, convenience, promotion and extension programme effectiveness and socioeconomic status.
Checks similar to those in the Sri Lanka questionnaire were built in. In addition, a question on 'willingness to purchase a replacement' was included to help determine the acceptability of the stove. The questionnaire was designed so that the data could be entered into the project's microcomputer quickly. It was administered by trained and experienced enumerators.
Experiences in Kenya and Indonesia have shown that it is not very cost-effective to try to use one questionnaire to gather both baseline information and information on patterns of usage/ acceptability. Such questionnaires are long, and can be difficult to interpret. They may be poorly administered and the answers may be suspect. Because the questionnaires are long, few are administered if funds and time are limited; hence, conclusions may be unreliable. In most cases, the time taken to analyse the data has been so long that it has been of little use to project management.
In the expansion phase, four programme elements can be monitored: the quality of extension in programmes that introduce stoves that must be installed by stove builders; stove production; stove sales in the market place (such stoves can be portable or may need to be installed by the household. The fourth exercise determines why people are or are not adopting a stove and the impact of the programme.
Appendix 5 a copy of an Improved Stove Monitoring Form that was developed for the Division of Community Forestry Programme in Nepal. It was used to monitor the quality of stove installations. The stove consists of three ceramic parts held together with mud. The stove is placed by a local person and the district promoter (stoves extension officer) comes to visit the house after it has been installed. The data that are collected provide information on the effectiveness of the extension programme (e.g. has the extension officer visited the house on time and do people know how to clean the stove). A check list can help the extension worker. During the first visit the promoter shows how to use and maintain the stove. The promoter then returns three months later to determine if there are any problems, either with the stove itself or with the users' methods of operation. These sheets are regularly reviewed by the project engineer, manager and/or monitoring and evaluation units of the programme.
Information from this monitroing form can help determine the effectiveness of the extension effort. Programme staff should sometimes review the information that is collected with the extension workers, both to check the information and to increase extension workers' incentive to take time and properly fill in the forms.
Production unit monitoring
Various methods can be used to monitor the output from the production unit. In Nepal, a detailed time and motion study of the production process was carried out for two production units. The study was done for all units at the beginning of production and three and six months later. A simple book was created to record the following:
|Date||# made||# stove workers||Kgs clay||# fired||# rejects||# sold|
Stoves sales monitoring
A wall chart can be used to record stove sales. Information can be collected in a book and immediately transferred to the chart which has axes plotting the number of stoves sold at different locations each month, or the price of stoves quoted by different vendors each month. Experience has shown that prices fluctuate with supply and demand as well as location. Alternatively, cards can be kept on each sale. Information on sales, prices, and reactions to stoves from different consumers can be recorded on these cards.
Patterns of adoption and impact
To monitor patterns of adoption and impact on households, collect data over at least two years. Select at least two representative villages (or clusters in an urban setting) in each area where the stove programme is to be implemented. Undertake a detailed socioeconomic survey that examines health, fuel consumption and cooking patterns. Stoves can be introduced in one village and not in the other so that detailed measurements, observations and interviews can be carried-out over the two year period to determine the changes resulting from adoption of the new stove.
In particular the programme can try to determine:
It is very difficult to specify a type of questionnaire that can be used, large scale surveys are not often done. A monitoring exercise that one programme used illustrates how a questionnaire could be structured and designed. This is presented in Appendix 8.
A great deal of data will be produced through a comprehensive monitoring programme. Some of this data can be used immediately, other sections can be stored for use at a later date. Often, data processing and storage is rather haphazard. This results in both loss of data and difficulties in analysis. A range of manual and microcomputer storage techniques are available to programmes at relatively low cost.
This is a technique popular with anthropologists. Data is normally segmented, coded and stored in a master file. The master file can be divided into villages and subdivided into households. Key information can be placed on the card:
|Name:||Type of stove:|
|# of stoves:||Date installed:|
|# of children:||Dates of extension visits:|
|Ethnic group:||Workers who visited:|
|Indicators of socio-economic status:||Date when stove stopped being used:|
On the back of the card a series of reference numbers that refer to other information about this family can be listed.
|Perceived costs and benefits (11):|
|Fuel use data(15):|
|Patterns of usage(14):|
Data on these subfiles will often be coded to facilitate analysis. For example, consider the card 14 below:
|14. Usage patterns||Village #:|
|House #:||Usage pattern:|
|a=all of the time....e= not at all|
Some data is aggregated before being coded. For example, socioeconomic status may consist of a combination of indicators such as housing, land holdings, occupation. A family might have a high status if it had more than five acres of land or a person working in the government. Medium status might include a family who owned a house with three rooms and land holdings of three to five acres.
Some programmes have used a large ledger. All of the data on each family can be coded and then placed in a ledger, as in the example below:
|Date||Name||House #||# People||Usage rate||Fuel savings||Smoke levels|
If the resources are available, data can be stored in relatively inexpensive microcomputers. A special statistical or data management programme can be purchased. This programme allows files to be established for households in each village or district. The data (or field) for each household are held in its record. The programme allows you to collect data on a specific topic (e.g. acceptance) from each record. It also permits searching for a particular record(s) that contains needed data.
Two types of microcomputers can be used. One uses MS-DOS as the operating system (e.g. the IBM PC), the other an Apple Macintosh. The Macintosh is a more expensive computer but is easier to use and can produce much higher quality reports than the MS-DOS machine. If the programme personnel have not used Microcomputers for statistical analysis it is strongly recommended that they either pay for a professional to set up the system and train the staff, or hire an experienced programmer.
|BACK||TOP||TABLE OF CONTENTS||BOTTOM||NEXT|
If there is to be a general acceptance and knowledge of the monitoring, senior staff working in the M & E unit, or designers and programme managers, should receive training.
The training course can examine:
User and producer training is essential if monitoring is to be effective. Training materials could discuss:
Enumerators can be involved in the initial survey phase and monitoring in the expansion phase. Their course can examine:
Field extension officers would probably be responsible for monitoring:
The type of training course that is developed will depend on the responsibility of the officers. For officers who are involved in small programmes training can teach:
The Division of Community Forestry Programme in Nepal has developed a particularily effective training course for field workers that includes a monitoring component (See Appendix 10). In this programme, district stove officers are responsible for production, distribution, and installation of the stoves, and for liaising with the community leaders and other organisations. They are also responsible for monitoring the performance of the potters and installers, and for collecting data for the monitoring and evaluation unit, which is situated in the head office. Before they commence the job they receive training in stove design, testing, production and installation, and instruction on methods of data collection and analysis. The head of the M & E unit and the consultant attached to the unit provide the specific inputs on collection of baseline data and monitoring. The topics include:
Enumerators and designers should receive training at the beginning of the survey phase. The course can focus on how to collect baseline data and choose sites for initial trials. It can be conducted by consultants from national research groups or, in the case of national programmes, by experts in stove programme assessment. Before the initial field trials programme managers may train community groups or individuals to observe and keep simple records of user reactions to stoves.
Many programmes have trained extension officers to collect more detailed information as the field trials proceed. The data record users' reactions, stove performance, production quality and quantity, and installation. Some programmes have also provided further training for senior programme staff and members of the M & E unit, on survey techniques, participant observation and measurement techniques for the expansion phase. During the expansion phase extension and community development workers have received follow-up training, either each year, or before a major data collection exercise has been implemented.
Training is carried out both "on the job" informally and in classes or formal training sessions.
Formal training is preferred when more than 2 or 3 people require instruction. Whether the training is formal or informal, the instructor should prepare for the sessions.
Training involves three processes: preparation, instruction and checking. Preparation involves focussing on desired results, planning logical learning stages and deciding the key concepts that need to be conveyed. In introducing each stage the instructor outlines the objectives and the key points. Each key point is accompanied by a practical demonstration or a question. Trainees complete each practical exercise and the instructor corrects errors. Trainees are encouraged to help each other and role play is used to highlight the most important aspects of different monitoring procedures.
Field work is an essential part of the training programme. In the initial survey phase, and during senior staff training, it is important that participants design a questionnaire, pretest, and evaluate it under the supervision of a trained social scientist.
For monitoring to be effectively done by all members of the programme, participants need to receive basic instruction in:
During implementation of the programme, various groups of people could receive specific training related to individual monitoring activities. If resources are available, all staff should attend refresher courses.
Effective training emphasizes "learning by doing". Field exercises are an essential component of the training.
|BACK||TOP||TABLE OF CONTENTS||BOTTOM||NEXT|