CHAPTER FOUR

Assessing and Monitoring Targeting
Effectiveness

The success of a targeted programme depends on detailed planning, efficient management and continuous monitoring and evaluation, with the results feeding back into improved planning and implementation of the programme.

Understanding targeting effectiveness

Assessing and monitoring are essentially learning processes. In this case, they can help planners, programme staff and communities to understand better the strength and weakness of an adopted targeting scheme, and to assess the need for changes. Poor targeting may result in worse programme outcomes than no targeting at all. So, improvement in targeting effectiveness should be a constant concern of the staff of targeted food and nutrition programmes. Targeting effectiveness relates directly to the programme's outcomes and impact and the degree to which it achieves programme objectives and goals. The assessment of targeting effectiveness should provide clear guidance on what complementary actions must be implemented to improve or change the targeting scheme. Monitoring is essentially the continuous assessment of the targeting effectiveness, as determinants of this effectiveness may change over time. In addition, depending on programme exit rules, programme participants will change over time, while eligibility criteria may also change, for example, in response to changes in programme resources. The need to monitor the targeting effectiveness of a programme is clearly demonstrated by the case of the national school feeding programme in Chile, where it led to changes in eligibility criteria and targeting methods (see the Chile case study in the Annex).

Assessment of targeting effectiveness is most important when the programme is administratively targeted, or when administrative targeting is part of a multistage targeting scheme. In self-targeting schemes, it is possible to assess effectiveness by sampling participation and target group status among those who have access to the benefits. However, it is much more difficult to assess how many members of the target group, or what proportion of the target group, do not have access to the programme benefits.

How is the effectiveness of a targeting scheme assessed? First of all, it is important to define what is meant by targeting effectiveness. Essentially, targeting effectiveness is a measure of how well a particular targeting scheme includes all members of a specified target group, and excludes members of the non-target group from participating, in a targeted programme. Key parameters in this assessment are inclusion and exclusion errors; and undercoverage and leakage rates. Each will be discussed in turn (see also the Box on Targeting terms in Chapter 1).

For example, a particular programme aims to reduce food insecurity in a specific population and targets food-insecure households in that population. A simple table can be constructed (see below), dividing the population into food-insecure and food-secure households and participating and non-participating households. A perfect targeting scheme results in all food-insecure households participating in the programme, and all food-secure households being excluded. For example, if there are 600 households classified as food-insecure, and 400 as food-secure, under a perfect targeting scheme A = 600, and D = 400. Food-insecure households that do not participate are referred to as exclusion errors (B), and food-secure households that do participate are referred to as inclusion errors (D). The most ineffective targeting scheme results in B = 600 and C = 400, that is none of the food-insecure households participate and all the households that do participate are food-secure. Normally, targeting schemes are not perfect, and there will be some degree of inclusion and exclusion errors. The smaller these are, the more effective the targeting scheme is.

The following are other related parameters of targeting effectiveness:

• Undercoverage rates refer to the proportion of the food-insecure households that are excluded from participation, i.e. B/A + B.
• Leakage rates refer to the proportion of participating households that are food-secure, i.e. C/A + C.
• In a perfect targeting scheme, the undercoverage and leakage rates are zero (because the inclusion and exclusion errors are zero).
• In the most ineffective scheme they are equal to one.

Another related concept of targeting effectiveness that is often applied in assessments is the proportion of the target population that participates (A/A + B) minus the proportion of participating households that do not belong to the target group (C/A + C). In a perfect targeting scheme the result will be 1, while with targeting that is 100 percent wrong it will be -1. A result of zero means that there is a complete absence of targeting and households are admitted randomly to programme participation. Thus, the objective is to bring the result as close to 1 as possible. If the result of this calculation does not lie somewhere between 0 and 1, explicit targeting is not worthwhile.

Correcting targeting errors

When assessing and monitoring targeting effectiveness, it is important to understand what factors may produce targeting errors, so that corrective measures can be taken by programme planners and staff.

Leakage rates may be high because eligibility and/or exit rules are poorly defined and/or incorrectly applied, or not enforced by programme staff. If programme eligibility is established on the basis of a means test, candidate participants may purposely underreport their assets in order to qualify for participation; and programme participants who, according to the rules, should exit from the programme may do the same.

Corrective measures that lower leakage rates include:

• ensuring that eligibility and exit rules are clearly defined, and are known, understood and agreed to by the whole population and by the programme staff;
• supervising the programme staff's application of the rules, and raising their awareness and motivation to apply the rules correctly;
• verifying and monitoring the eligibility status of participants through home visits or other means, when feasible.

Undercoverage rates may be high for a number of reasons. Target households may be unaware or poorly informed about their eligibility status, and thus many will not be reached by the programme. The programme design may not have anticipated adequately the constraints to participation faced by target households, such as lack of time and transportation. Target households assess the programme benefits to them in relation to their participation costs (including time costs), and those that assess the net benefits to be small or that place little value on them may elect not to participate. If programme eligibility is established by a means test involving documentation, prospective participants may not be able to produce the documentation required or, for other reasons, they may elect not to submit such documentation or information. Furthermore, if programme participation involves any kind of social stigma, target households may elect not to participate.

Lowering high undercoverage rates normally also requires one or more corrective actions, depending on what has caused the high rate. Such measures include:

• providing more effective outreach to target group members to inform them about their eligibility and the programme benefits, possibly involving the whole community;
• reassessing the programme design to see what changes should be introduced in order to address participants' constraints more adequately and to raise the value to the participants of the programme benefits, applying participatory assessment methods;
• working closely with candidate participants to establish their eligibility by way of a means test, assuring complete confidentiality of the information provided;
• working closely with community leaders and the community as a whole, to make them members of the stakeholder group and to remove social stigmas associated with participation in the programme.

When both leakage and undercoverage rates are high, programme staff may have to prioritize the implementation of corrective actions. If the major concern is with the efficiency of programme resource allocations, or if the programme is faced with budget cuts, the priority is likely to be to lower the leakage rate and improve programme efficiency (cost-effectiveness). If the main concern is improving the programme's impact on the target population and increasing its social benefits, corrective measures to lower the undercoverage rates should be the highest priority.

Improving targeting cost-effectiveness

The assessment and monitoring of targeting effectiveness is a critical component in measuring the programme's impact and cost-effectiveness. Low leakage and undercoverage rates mean that the programme's social benefits to a given population group are maximized, subject to the programme budget. Participation levels can vary among participating households or individuals, and this variation will also influence the programme's overall impact. Eligible members of the target group may participate with varying degrees of intensity in the programme, or participate in only a subset of the programme's components. To assess targeting effectiveness, some minimum level of participation, as well as a minimum intensity of participation, should be defined, perhaps by applying indicators such as number of feeding days per beneficiary per month. This may also be useful in the evaluation of the programme's impact.

Providing a breakdown of participation by age, sex or other criteria is also useful in evaluating the programme's impact. Well-targeted programmes may offer services to fairly diverse groups. In many food security and nutrition programmes, benefits may be targeted to pregnant and lactating women and to children under five years of age. While aggregate coverage rates may be fairly high, some particularly vulnerable sub-groups, such as children under two years of age, may be under-represented among actual participants. This suggests that outreach to increase the participation of particular sub-groups would significantly strengthen the programme's long-term impact.

The urban poor often lack the physical and economic access to good quality, safe and nutritionally adequate food supplies.