Previous Page Table of Contents Next Page


Appendix A

Examples of uses of the data


Example 1. National food insecurity mapping in Mozambique, 1995 -96

NAME OF SURVEY: Mozambique Inquérito Nacional aos Agregados Familiares Sobre As Condições de Vida (MIAF).

DATA COLLECTION AGENCY: Instituto Nacional de Estatistica.

METHOD OF COLLECTION OF FOOD EXPENDITURE DATA: Households were visited three times over a seven-day period. On each visit, the households were asked what food was acquired that day as well as the preceding two days (on the second and third visits). Households reported both quantity and total expenditures for each of 200 food items.

METHOD OF ANALYSIS: The household energy availability for each food item was calculated from the reported quantities and translated into daily values given the total number of days the household was expected to consume the item. The daily value was then divided by the number of adult equivalents in the household to give the daily energy availability per adult equivalent unit. The WHO (1985) energy intake requirement for a male aged 18 -30 engaged in moderate activity (3 000 kcal) was used as the cutoff for a food secure household. Food security mapping was undertaken by geographic location as well as by sociodemographic group. The number of households was 6 463.

MAIN FINDINGS: Food insecurity is a major problem in Mozambique. Sixty-four percent of the population, over 10 million people, lives in households that are food insecure (Table A1). Surprisingly, the prevalence of food insecurity is higher in urban than rural areas (67 percent vs. 63 percent). There are fairly strong differences in the prevalence of food insecurity across the main regions of the country. Food insecurity is the least severe in the rural north, at 48 percent. It is the most severe in the southern region, at 75 percent. Even though the prevalence of food insecurity is highest in the urban areas, the largest numbers of food insecure people live in the rural areas. Seventy-eight percent of all food insecure households (about 7.8 million people) are found in rural areas. Forty percent of all food insecure households are located in the central region, more than twice the percent in the north or the south. Table A2 shows that in Mozambique, female -headed households are no more vulnerable to food insecurity than male - headed households. Households containing migrants of the recent war and households with young children (roughly 65 percent of all households) are more likely to be food insecure than the rest of the population. Elderly people living in households with no other members of working age are much less likely to be food insecure than the rest of the population in both rural and urban areas.

Sources: Datt et al. (2000); Ministry of Planning and Finance, Government of Mozambique (1998).

TABLE A1. FOOD INSECURITY IN MOZAMBIQUE, 1996 - 97


Percentage of food insecure households

Average energy availability per adult equivalent unit per day

Locational distribution of food insecurity (%)

Overall

64

2 761


Urban

67

2 645

22

Rural

63

2 792

78



Urban


Maputo

67

2 663

8

Large citiesa

71

2 477

6

Others

65

2 751

8



Rural


North

48

3 305

20

Central

69

2 601

40

South

75

2 389

19

a Beira, Nampula, Matola.
Source: MPAR (1998).


TABLE A2. FOOD INSECURITY AMONG SOCIODEMOGRAPHIC GROUPS IN MOZAMBIQUE, 1996 -97


Percentage of food insecure households

Percentage of population

Rural

Urban

Rural

Urban

Sex, head of household

Male

63

68

83

81

Female

63

64

17

19

War migrants

Yes

73

71

7

3

No

62

67



Children (<5 years) in the household

Yes

65

69

62

67

No

60

64



Elderly (with no other members of working age)

Elderly

35

30

1

1

Others

63

67



Source: MPAR (1998).

Example 2. Food quality in Bangladesh, 1991/92 - 1995/96

NAME OF SURVEY: Bangladesh Household Expenditure Surveys 1991/92 and 1995/96.

DATA COLLECTION AGENCY: Bangladesh Bureau of Statistics, Statistics Division, Ministry of Planning, Government of the People's Republic of Bangladesh.

METHOD OF COLLECTION OF FOOD EXPENDITURE DATA: Face-to-face interviews were conducted by a diary-keeper/enumerator who recorded daily food acquisition in quantities for seven days in a row. The number of food items was over 50.

METHOD OF ANALYSIS: Quantities available of food in each of 12 major food groups were converted into energy and protein values, summed, and divided by the number of usual household members. These values were then averaged over the sample households in each survey to arrive at an estimate of per capita energy and protein intakes for each of the food items in the two periods (1991/92 and 1996/96). Averages were also calculated separately for urban and rural areas. The number of households surveyed was 7 420 in 1995/96 and approximately 1 200 in 1991/92.

MAIN FINDINGS: The overall quality of the diet in Bangladesh is very poor. The large majority (78 percent) of the energy consumed comes from cereals, of which rice makes up nearly three -quarters (Table A3). Cereals, considered to be a low-quality protein, also contribute the most to protein intake (Table A4). Consumption of protein and micronutrient-rich animal products is very low but is much higher in urban than rural areas. In terms of changes over time, while energy consumption has remained roughly constant, protein consumption has declined slightly, signalling a decline in diet quality in the country. The decline can be linked to an almost 40 percent reduction in the energy derived from fish consumption.

Source: Bangladesh Bureau of Statistics (1998).

TABLE A3. DAILY PER CAPITA ENERGY AVAILABILITY (kcal) BY FOOD ITEM, BANGLADESH 1991-92 AND 1995-96

Food item groups

1995-96

1991-92

National

Rural

Urban

National

Rural

Urban

Total

2 254.0

2 263.1

2 208.1

2 265.6

2 266.8

2 258.1

Cereals

1 758.8

1 804.5

1 528.2

1 810.1

1 834.9

1 650.2

Rice

1 607.8

1 658.6

1 351.5

1 659.9

1 690.9

1 460.7

Wheat

115.6

111.2

137.7

126.3

120.5

163.8

Others

35.5

34.8

38.9

23.9

23.5

25.7

Potato

44.0

41.5

56.4

39.4

37.4

52.6

Vegetables

72.5

74.4

63.0

61.3

60.6

66.1

Leafy vegetables

17.4

17.3

18.0

10.6

10.5

11.4

Others

55.1

57.1

45.0

50.7

50.1

54.7

Pulses

47.7

44.0

66.3

62.2

60.2

75.4

Masoor

20.8

16.0

45.4

27.3

23.4

52.2

Kheshari

15.8

17.5

7.0

26.1

28.5

11.0

Others

11.1

10.5

14.0

8.8

8.3

12.2

Milk/milk products

37.5

34.7

51.1

19.8

18.4

29.3

Edible oils

88.1

75.3

152.9

91.7

82.7

149.7

Mustard

51.0

55.7

27.5

46.3

49.0

28.9

Soybean

33.7

16.3

121.5

42.4

30.7

117.6

Others

3.4

3.3

3.9

3.0

3.0

3.2

Meat, poultry and eggs

19.7

16.0

38.9

18.4

16.9

27.8

Mutton

1.5

1.3

2.5

1.4

1.3

2.1

Beef

7.5

5.6

17.1

6.1

5.2

11.5

Chicken/duck

4.3

3.5

8.2

2.5

2.3

3.8

Eggs

5.6

4.7

10.3

8.4

8.1

10.3

Others

0.8

0.9

0.8

0.0

0.0

0.1

Fish

63.6

59.7

83.4

45.8

42.7

65.2

Condiments and spices

38.0

37.0

43.2

48.4

47.4

55.0

Onion

5.8

5.0

10.1

6.0

5.6

8.6

Chillies

10.5

10.7

9.6

9.2

9.4

8.0

Others

21.7

21.3

23.5

33.2

32.4

38.4

Fruits

18.9

16.9

28.9

14.0

13.4

18.0

Sugar/gur

36.8

36.2

40.0

35.5

34.3

43.4

Sugar

15.1

11.7

32.3

15.3

12.7

32.2

Gur

21.7

24.5

7.7

20.2

21.6

11.2

Miscellaneousa

28.4

22.9

55.8

19.0

17.9

25.4

a Miscellaneous includes tea, soft drinks, bread, biscuits, betel nut, betel leaf etc.
Source: Bangladesh Bureau of Statistics (1998).


TABLE A4. DAILY PER CAPITA PROTEIN AVAILABILITY BY FOOD ITEM, BANGLADESH 1991-92 AND 1995-96

Food item groups

National

Rural

Urban

1995-96

1991-92

1995-96

1991-92

1995-96

1991-92

Total

66.01

62.72

65.38

62.29

69.19

65.49

Cereals

38.63

39.78

39.53

40.23

34.08

36.90

Rice

33.87

34.97

34.94

35.62

28.47

30.77

Wheat

3.88

4.23

3.72

4.03

4.61

5.48

Others

0.88

0.58

0.86

0.97

0.99

0.65

Potato

1.48

1.33

1.40

1.26

1.90

1.77

Vegetables

4.49

3.28

4.49

3.18

4.46

3.88

Leafy vegetables

1.85

1.02

1.82

0.99

2.02

1.20

Others

2.64

2.26

2.67

2.19

2.44

2.68

Pulses

3.56

4.74

3.31

4.61

4.84

5.56

Masoor

1.52

2.00

1.17

1.71

3.32

3.82

Kheshari

1.29

2.13

1.43

2.33

0.57

0.90

Others

0.75

0.61

0.71

0.57

0.95

0.78

Meat, poultry and eggs

3.08

2.51

2.45

2.27

6.26

4.10

Mutton

0.18

0.17

0.16

0.16

0.31

0.26

Beef

1.49

1.20

1.11

1.03

3.40

2.27

Chicken/duck

0.85

0.50

0.70

0.46

1.64

0.77

Eggs

0.42

0.64

0.35

0.62

0.79

0.79

Others

0.14

0.00

0.13

0.00

0.20

0.01

Fish

9.15

6.78

8.85

6.52

10.68

8.49

Condiments and spices

1.23

1.61

1.21

1.59

1.36

1.71

Onion

0.14

0.14

0.12

0.14

0.24

0.21

Chillies

0.45

0.60

0.46

0.61

0.40

0.52

Others

0.65

0.87

0.63

0.84

0.71

0.98

Fruits

0.65

0.43

0.57

0.40

1.04

0.63

Sugar/gur

0.01

0.01

0.01

0.01

0.01

0.01

Sugar

0.00

0.00

0.00

0.00

0.01

0.00

Gur

0.01

0.01

0.01

0.01

0.00

0.05

Milk/milk products

2.08

0.84

1.90

0.77

3.00

1.30

Edible oils

1.13

1.03

1.24

1.09

0.61

0.64

Mustard

1.13

1.03

1.24

1.09

0.61

0.64

Soybean


0.00


0.00

0.00

0.00

Miscellaneousa

0.52

0.38

0.43

0.36

0.96

0.50

a Miscellaneous includes tea, soft drinks, bread, biscuits, betel nut, betel leaf, etc.
Source: Bangladesh Bureau of Statistics (1998).

Example 3. The impact of a national social programme on household food availability and diet diversity in Mexico, 1998-99

BACKGROUND: The Mexican government's Education, Health and Nutrition Program, Programa de Educación, Salud y Alimentación (PROGRESA) provides monetary assistance, nutritional supplements, educational grants and a basic health package to its beneficiaries for a minimum of three years. The programme started in 1997 and by the end of 1999 covered approximately 2.6 million families. In an effort to evaluate the impact of the programme on the well -being of the beneficiaries, the Mexican government in collaboration with the International Food Policy Research Institute launched a series of national surveys in which data on a number of measures, including food expenditures, were collected.

NAME OF SURVEY: Encuesta de Evaluación de los Hogares collected under the auspices of an evaluation of PROGRESA.

DATA COLLECTION AGENCY: Instituto National de Salud Publica.

METHOD OF COLLECTION OF FOOD EXPENDITURE DATA: Households were interviewed to determine how much food was available in the last seven days with reference to 35 different foods.

METHOD OF ANALYSIS: Household energy availability per day was calculated from the food quantity data using energy conversion factors and divided by the number of household members to arrive at daily per capita household energy availability. Dietary diversity was calculated as the total number of individual food items consumed. Descriptive analysis and multivariate regression were employed to determine whether project beneficiaries had higher per capita household energy availability and dietary diversity than non-project beneficiaries, controlling for socio-economic characteristics.

MAIN FINDINGS: The multivariate analysis showed that by November 1999, households receiving PROGRESA benefits were able to acquire 7.1 percent more energy than comparable households that did not receive benefits. The descriptive results show that, additionally, PROGRESA beneficiary households were more likely to consume a wide variety of fruits, vegetables and meat products, as evidenced by higher dietary diversity scores. The multivariate results confirm that the greatest source of the program's impact on energy availability came from increased availability of energy from vegetable and animal products. Thus, PROGRESA enabled people not only to eat more but also to obtain a higher quality diet.

Source: Hoddinott, Skoufias and Washburn (2000).

Appendix B

Methods of estimating global and regional food energy deficiency prevalences when data are not available for all countries

Let countries be indexed by i =1, ..., N. The countries for which household survey data exist are denoted C1, ...Cm. Those for which household survey data do not exist are denoted Cm+1, ..., CN.

Method 1. Extrapolations based on cross-country regression

This method draws on the extrapolation technique employed to estimate national poverty rates by the World Bank's Global Poverty Monitoring Facility. The estimation relies on individual country-specific characteristics to estimate energy deficiency rates for countries where survey data do not exist.

The extrapolations are undertaken using cross-country ordinary least-squares regressions, where the regressors are variables describing various economic, social, political and demographic factors related to food insecurity. Candidates for these factors may be:

arable and irrigated land areas;
foreign exchange earnings;
per capita dietary energy supplies;
per capita national incomes;
income distribution;
life expectancy;
mortality rates;
school enrolment rates;
labour force participation rates;
urbanization rates;
conflict prevalences;
degree of democracy;
infrastructure index;
population size;
age distribution of the population.

Where the goal is to estimate a developing world prevalence, the region of location would also be included as a regressor.

The following is the estimating equation for the extrapolations:

The Xk are regressors, the bk are parameters to be estimated, and u is a stochastic error term. The logit transformation is used to restrict the predicted values to lie within the theoretical bounds (0, 1) and to assure that the error term is unbounded (Ravallion, Datt and van de Walle, 1991).

Method 2. Extrapolations based on country-specific non-parametric density functions and cross-country regression

This method relies on the estimation of empirical household energy availability probability density functions, fi(x), for the countries for which survey data exist to predict density functions for the remaining N-(m+1) countries. The method is non-parametric. Rather than requiring an a priori specification of a particular function form, the non-parametric approach is fully flexible. It allows the data to select the most appropriate representation of the distribution (DiNardo and Tobias, 2001; Goodwin and Ker, 1998). The estimation takes place in two stages.

In the first stage, the populations of countries C1 through Cm are divided into D groups, indexed d=1, ..., D, based on equally sized energy availability intervals. Following, gdi, the proportion of the countries' populations falling into each group is estimated from the survey data. D cross-country regressions are then undertaken (corresponding to the D energy availability groups) in which the dependent variable is gdi, and the predicting variables used in Method 1 are the candidate regressors. The resulting estimating equations are then used to predict gdi, I=m+1, ..., N.

The set of D estimating equations is as follows:

In the second stage, a non-parametric density estimation technique is employed to trace out fi(x) for the N-(m+1) countries for which survey data do not exist, using their predicted gdi. The specific technique employed is the kernal method of smoothing to build continuous density functions. Here, each country's observations are surrounded by a symmetric weighting function, K, satisfying the condition[19]

Let xd represent the mean energy availability level of interval d. The kernal estimator of the probability density function of x is given by

,

where h is a bandwidth parameter that controls the amount of smoothing. The proportion of each countries' population acquiring less than z kcal can then be estimated numerically.

Appendix C

Energy requirements by age and sex for light and moderate activity levels

TABLE C1. ENERGY REQUIREMENTS (KCAL) BY AGE AND SEX FOR LIGHT AND MODERATE ACTIVITY LEVELS

Age/sex group

Requirement for light activity (kcal)

Requirement for moderate activity (kcal)

Children 0 5 years

0 1

820

820

1 2

1 150

1 150

2 3

1 350

1 350

3 5

1 550

1 550

Males 5 and over

5 7

1 850

1 850

7 10

2 100

2 100

10 12

2 200

2 200

12 14

2 400

2 400

14 16

2 650

2 650

16 18

2 850

2 850

18 30

2 600

3 000

30 60

2 500

2 900

60+

2 100

2 450

Females 5 and over

5 7

1 750

1 750

7 10

1 800

1 800

10 12

1 950

1 950

12 14

2 100

2 100

14 16

2 150

2 150

16 18

2 150

2 150

18 30

2 000

2 100

30 60

2 050

2 150

60+

1 850

1 950

Source: WHO (1985).

Discussion opener - household expenditure survey methods

Antonia Trichopoulou
University of Athens
Athens, Greece

The paper by Smith in this series is an in- depth review on the possibility of using data collected through household expenditure surveys to assess food insecurity in the developing world. The author points out the several problems encountered in the use of these surveys. Nevertheless, additional information on how to tackle these limitations may contribute to the value of the paper.

Indeed, the household expenditure surveys represent a valuable source of information on food availability at the household level. The paper discusses several methodological issues that limit full exploitation of this method. It would be wise therefore, before implementing the proposed approach on a broad scale, to undertake feasibility studies using raw national data. In this process, it might be realized that there are several pitfalls in the data - sets and various unforeseen constraints in the data analysis, and that additional or alternative approaches may be required. Cross-country comparability is an even more complex issue. In the European region, there have been studies and initiatives to exploit the use of the food and socio -economic data of the household expenditure surveys for food monitoring purposes at the intra -and international level. Future analysts could benefit from this accumulated experience.

The paper by Smith is adequately focused on how the data could be used, but a proposal for robust methodology to assure data validity and comparability seems to be missing. The following obser vations illustrate this point.

The author's comment on the necessity of guidelines and standard protocols for future data collection is well taken. Nevertheless, there is a need for some basic principles to be considered before the currently available data can be used. An example of this can be seen in table 2 of the Smith paper where the recall period varies between one day (Kenya) and one month (South Africa).

The author refers to household expenditure survey data as a tool for evaluating and comparing dietary diversity. However, the coding system used to evaluate the variety of diets is of crucial importance. Thus, it should be noted that, once more according to table 2 in the Smith paper, the number of reported food items ranges from 31 (South Africa and Zambia) to 405 (Malawi).

In some instances, household expenditure surveys collect data only on the food expenses of the household. The author needs to explore other available and reliable data sources for converting food expenses to food quantities.

The author frequently mentions calorie insufficiency. More explicit information is needed on the availability and comparability of the food composition data that would allow such estimations.

It is also mentioned that food availability could predict food intake, given several assumptions. The number of required assumptions may be reduced, however, if solid approaches towards making these predictions are taken into consideration. These approaches, available in the literature, are not highlighted in the text.

It is clear that household expenditure surveys in the developing world have other priorities and that deriving estimates of individual availability from household data are a secondary consideration. Nevertheless, it will add to the completeness of the paper if a section is included that refers to statistical modelling, also available in the literature, for individualizing household expenditure data.

Any approach that provides data on food insecurity is beneficial in formulating a nutrition policy for the developing world. The approach described by Smith is valuable, but short-and long-run planning should be considered for its implementation. The extensive experience of Europe in the use of household expenditure survey data could contribute to this approach.

Discussion opener - household expenditure survey methods

Sergio H. Lence
Iowa State University
Ames, IA, USA

The value and usefulness of any particular method of obtaining data pertaining to some measurable variable are ultimately determined by the method's reliability and by how conceptually close the measured variable is to the theoretical construct of interest. That is, to be of much use in analysing a particular issue, a method should provide data that are both reliable and valid with respect to the problem under investigation.[20] For this reason, the following discussion will centre on the reliability and validity of nutrition data obtained from food expenditure surveys.

As noted by Smith, there is a notorious scarcity of datasets that can be used to shed some light on the issue of reliability and validity. Fortunately, I have had access to two such sets, which allowed me to perform some exploratory analysis on the matter. The datasets were constructed from surveys conducted by the International Food Policy Research Institute in Kenya and the Philippines, and are essentially the same sets as those used by Smith to prepare table 1 and figure 2 of her report.[21] Confirming Smith's findings, the summary data reported in Table 1 reveal that for both countries, the mean and median of the calorie availability figures calculated from the food expenditure surveys are very similar to the mean and median of the calorie intake figures from the food recall surveys. Based on such evidence alone, one may be led to conclude that the food availability and the food intake data are consistent with each other, and that there are no grounds to suspect the reliability or the validity of either method.

Note, however, that the above is a very weak test of reliability and consistency. Indeed, reliability and validity require not only that the population's food availability mean (or median) be close to the population's food intake mean (or median), but also that the respective figures for each household have similar magnitudes. Therefore, a more powerful test of reliability and validity involves comparing the values of food availability and food intake for each individual household. This is done by means of Figures 1 and 2, which depict per capita food availability against per capita food intake for each household in the Kenya and Philippines surveys, respectively.[22] In both graphs, the medians for food intakes and food availability are displayed as the thick horizontal and vertical lines, respectively. Note that if the intake and availability measures were the same for each household, all observations would fall on the diagonal 45 degree line. In contrast, Figures 1 and 2 reveal that quite often, there are substantial differences between a household's food intake measure and the corresponding food availability measure. Not surprisingly, the correlation coefficients associated with Figures 1 and 2 are only 36.9 percent and 41.6 percent.

TABLE 1. MEAN AND MEDIAN FOOD AVAILABILITY AND FOOD INTAKE IN KENYA AND THE PHILIPPINES


Logarithm of daily household per capita calorie intakea

Logarithm of daily household per capita calorie availability

Kenya

Mean

7.80

7.75


Median

7.83

7.87

Philippines

Mean

7.38

7.39


Median

7.40

7.41

a For Kenya, values are expressed on a per adult equivalent basis rather than on a per capita basis.

FIGURE 1. COMPARISON OF HOUSEHOLD CALORIE INTAKE AND HOUSEHOLD CALORIE AVAILABILITY MEASURES FOR THE KENYA SURVEY*

FIGURE 2. COMPARISON OF HOUSEHOLD CALORIE INTAKE AND HOUSEHOLD CALORIE AVAILABILITY MEASURES FOR THE PHILIPPINES

In summary, Figures 1 and 2 raise severe doubts about the reliability or validity of the intake measure or availability measure, or both. Unfortunately, the data available do not allow us to attribute the observed discrepancies between measures to low reliability or to low validity, so the remainder of the discussion will refer to them indistinctly as reliability/validity problems.

The evidence provided by Figures 1 and 2 stands in stark contrast to Smith's conclusion that "owing to the fact that the basic data are collected from households themselves, ..., estimates of food insecurity [from expenditure surveys] are likely to be reasonably reliable." As such, it seems important to analyse why data collected from households themselves might not be as reliable or valid as it might be presumed. In this regard, Bertrand and Mullainathan (2001) show that survey responses often exhibit various types of validity/ reliability problems, two of them being "cognitive" and "social desirability" problems.

"Cognitive" problems, according to Bertrand and Mullainathan (2001), refer to situations where cognitive factors affect responses. This may be due to the structure of the survey. For example, the order of the questions and their wording is known to affect the answers. The scales used to elicit responses for quantitative variables have also been shown to impact survey results. In addition, the mental effort required to respond has proven to have an effect on answers (Bertrand and Mullainathan, 2001). Other cognitive problems are those arising from a lack of knowledge and a low ability to process information. As an example, when asked about their household's food expenditures in a survey held in Pakistan, husbands provided much lower values than wives. Also, far more than 50 percent of respondents typically report being above -average drivers (Shefrin, 2000, p. 41).

"Social desirability" problems arise when respondents attempt to avoid looking bad. The following examples illustrate this kind of behaviour: immediately after an election, about 25 percent of non-voters usually report having voted; self-reported racial prejudice decreases when the interviewer is a black person; and people often provide answers about fictitious matters (Bertrand and Mullainathan, 2001). Another example comes from the US National Health and Social Life Survey where men report having had about 75 percent more partners over the most recent five years of their lives than women do (Lewontin, 2000, p. 262). More specific to food surveys is the finding that in the Brazilian Escudo Nacional da Despesa Familiar (ENDEF) Survey, "... among the poorest of households, there is evidence of a decline in intakes after a day or two; the survey staff attributed this to an attempt by the respondents to impress the enumerators (or downplay their own poverty)." (Strauss and Thomas, 1998, p. 794).

A third type of reliability/validity problems is associated with the possibility of respondents' strategic behaviour. If respondents perceive that their responses will be used for policy actions that may have a direct impact on them, they will have strong incentives to answer questions strategically in order to maximize their potential gains or minimize their potential losses.

A fourth type of reliability/validity problem is due to respondents' lies for unknown reasons. Research on lies and deception has found that telling lies is a standard feature of people's everyday interactions (Feldman, Forrest, and Happ, 2002; Kashy and DePaulo, 1996), that people do not consider lying as a serious issue, and that they do not worry about being caught (Kashy and DePaulo, 1996). Further, it has been found that lies are more likely when the interactions are less intimate (DePaulo and Kashy, 1998; DePaulo et al., 1996). A nice example of the potential reliability/validity problems induced by lies in surveys is a study by Lilienfeld and Graham (1958) in which a group of 192 men were first asked whether they had been circumcised or not, and then were examined to determine whether they were indeed circumcised or not. A massive 34.4 percent of the men were found to have lied about their circumcision status, as 19 out of the 56 (33.9 percent) men who reported to have been circumcised were found not to be circumcised, and 47 out of the 136 (34.6 percent) men who stated that they had not been circumcised were in fact circumcised.

A fifth set of potential explanations for reliability/validity problems stems from the execution and administration of surveys. First, mistakes in recording, documenting and processing data all lead to reliability/validity problems. Second, surveyors may behave strategically. For example, because of the time and effort involved in interviewing each household, surveyors have strong incentives to shirk by fabricating questionnaire responses. This problem can be greatly reduced by setting an adequate monitoring structure for the survey. However, anecdotal evidence suggests that fabricated responses are not uncommon in practice.

In addition to the just-described reliability/ validity problems associated with surveys in general, there are well-known reliability/validity problems specific to food expenditure surveys. Among others, these include (1) the use of food composition tables to convert food into energy, (2) the omission of some food stock and/or food flow items in the administered questionnaires, and (3) the use of price indices to convert food expenditures into food quantities.

To summarize, the existing literature provides ample evidence about potential reasons as to why food survey data may be far less reliable /valid than expected a priori on the grounds that data are collected from households themselves. That is, Figures 1 and 2 should not be too surprising when interpreted under such evidence. Importantly, a particularly negative implication of the findings of the literature on reliability/validity problems of survey responses is that such problems seem very difficult - if not impossible - to overcome.

My conclusions are threefold. First, given the reliability/validity problems discussed earlier, Smith's conclusions regarding the strengths of food availability data from expenditure surveys should be strongly qualified. Second, the reliability/validity problems found in the food availability data should be disclosed to the potential users of such data. Third, the severity of the reliability/validity problem implied by Figures 1 and 2 suggests that much more effort should be spent in studying the method's shortcomings, as opposed to continuing to conduct surveys as usual under the naïve presumption that data are reliable or valid simply because respondents are asked to tell the truth.

References

Bertrand, M. & Mullainathan, S. 2001. Do people mean what they say? Implications for subjective survey data. Am. Econ. Rev. Pap. Proc., 91: 67 -72.

DePaulo, B.M. & Kashy, D.A. 1998. Everyday lies in close and casual relationships. J. Pers. Soc. Psychol., 74: 63 -79.

DePaulo, B.M., Kashy, D.A., Kirkendol, S.E., Wyer, M.M. & Epstein, J.A. 1996. Lying in everyday life. J. Pers. Soc. Psychol., 70: 979 -995.

Feldman, R.S., Forrest, J.A. & Happ, B.R. 2002. Self-presentation and verbal deception: Do self-presenters lie more? Basic Appl. Soc. Psychol., 24: 163 -170.

Kashy, D.A. & DePaulo, B.M. 1996. Who lies? J. Pers. Soc. Psychol., 70: 1037 -1051.

Lewontin, R. 2000. It ain't necessarily so -the dream of the human genome and other illusions. London, Granta Books.

Lilienfeld, A.M. & Graham, S. 1958. Validity of determining circumcision status by questionnaire as related to epidemiological studies of cancer of the cervix. J. Natl. Cancer Inst., 21: 713 -720.

Shefrin, H. 2000. Beyond greed and fear: Understanding behavioral finance and the psychology of investing. Boston, MA, Harvard University School Press.

Strauss, J. & Thomas, D. 1998. Health, nutrition, and economic development. J. Econ. Lit., 36: 766 -817.

Trochim, W. 2002. Research methods knowledge base (available at http://trochim.human.cornell.edu).

Discussion group report - household expenditure survey methods

Josef Schmidhuber
FAO
Rome, Italy

The main areas of the discussion were:

The comparability of survey information across countries.

Differences between food accessibility and food intake. The discussion of this problem was taken far beyond the traditional focus on differences owing to periodicity problems, food consumption by guests, workers and pets, and losses in terms of household waste.

The need for a better understanding of what the notion of food vulnerability really entails, how important it is for food insecurity across different countries and how the probability of facing a future food shortage can be related to an average (mean) food availability level.

Ms Trichopoulou opened the discussion with a report about her own experience in analysing household surveys. One of the outstanding problems in the work with household expenditure/budget surveys is the lack of comparability of results across countries. She stressed that even in advanced European economies, the comparability of the results from these surveys is limited, and more should be done to make the results of household surveys comparable. Such efforts could include:

common guidelines for data collection, compilation and interpretation;

common guidelines for error detection and remedies to solve inherent data problems; and

most importantly, a common coding system for food composition tables so that food preparations have the same meaning across different countries.

A first step towards solving these problems would be a study to identify the main problems associated with the collection, compilation and interpretation of household surveys. Focus should be placed on the comparability of food composition tables.

Mr Lence focused on the "predictive validity" (or more precisely "convergent validity") of household expenditure surveys in assessing food intake levels. The predictive validity captures the extent to which the results from household expenditure surveys (food availability) are good predictors for the results from actual food intake information. This is an important question because predictive validity was identified as one of the main advantages of household expenditure surveys.

The discussant sought to clarify the question on the basis of two country cases (Kenya and the Philippines) for which data on both food intake from 24 -hour recall and food expenditure surveys were available. A close inspection of the data gave rise to a number of questions.

What are the results of comparing food intake information to food expenditure data?

The means of the two surveys, i.e. the energy intake from the 24 -hour recall and food expenditure from the seven-day food acquisition surveys, are very close to each other.

Also, the marginal distributions of the two "experiments" are very similar in both countries and over the four repetitions of the two experiments.

However, when juxtaposing the information from the two experiments for the individual observations, i.e. the reported food intake and the reported food acquisition at the level of an individual person, the correlation was rather weak. The correlation coefficient was 0.35; a simple regression of food intake on food acquisition (for individual observations) would have rendered an R2 of merely 0.12.

What could be the reason for the low correlation and thus the possibility that one may overestimate the predictive validity of household expenditure surveys for actual food intake?

The discussant suggested that there is a need to have a closer look at problems that plague survey information in general, not only household expenditure or food intake surveys. Quoting a study by Bertrand and Mullainathan, he identified cognitive problems and social desirability problems as the two main areas to look at. Cognitive problems include those that arise, inter alia, out of the structure of the survey. The order of questions, the wording, the scales and the mental effort required to answer questions can have an impact on the results. Social desirability problems occur when, for instance, respondents do not want to look bad in front of interviewers. Over and above social desirability and cognitive problems, strategic behaviour or simply not telling the truth can affect the validity of survey responses.

What are the main conclusions from this exercise?

The main suggestions forwarded by the discussant were that the predictive validity of the household surveys for measuring food security can easily be overstated. In fact, the data available (although scarce) suggest that household expenditure surveys may be plagued with non-negligible validity or reliability problems in assessing food security. This calls for added efforts to improve the validity and reliability of household surveys in assessing food security.

The free discussion focused on two main areas:

Can, and if so, how can the possible lack of reliability in household expenditure surveys affect the estimates of undernourishment and food insecurity?

How can we better define and assess food vulnerability?

The starting hypothesis for evaluating the first question was that we observe a surprisingly low correlation between two measurement methods for the same theoretical construct of food availability. If this lack of correlation is due to unreliability of the information observed, i.e. that most of the observed variability is due to a high amount of random noise, there is a risk that the measured level of undernourishment overestimates its true level. The reason is that at the cutoff point, we evaluate a distribution that has a larger variance, resulting in a higher percentage of hungry.

In response to the second question, it was generally felt that the concept of food vulnerability requires a more precise definition. While the share of food expenditure in total expenditure may be a good starting-point for assessing vulnerability, it is not sufficient within a given economic environment, and the same food expenditure share would not necessarily represent the same level of vulnerability across different economic environments. There was a consensus that other factors need to be taken into account. These include:

Seasonality of food availability, notably differences between rainy season and dry season.

A clear definition of the reference period and the time horizon. Neither the measurement period nor the survey period of household surveys may be appropriate to assess vulnerability.

Vulnerability should also include the availability of (food) credits, e.g. an account at a food shop may help bridge periods of high prices and insufficient access to food.

Price variability for basic foodstuffs is in itself an important factor to consider in deriving the probability at which average expenditures may become insufficient to purchase enough food. Price spikes and their probability could be of particular importance in this regard.

There is also the need to distinguish truly random factors (e.g. civil strife or an abrupt change in the social security system) from non-random developments (seasonality). The main difference lies in the predictability of such events. As the effects of seasonality are predictable, precautionary measures can be put in place to mitigate possible future problems.


[19] The weighting function K (t )is normally a symmetric probability density function (Goodwin and Ker, 1998)
[20] "Convergent validity "is the degree to which the data on a measure are similar to data on other measures that it should be similar to (Trochim, 2002). A measure is reliable if it yields the same result over repeated samples, assuming that the underlying object of measurement remains constant (Trochim, 2002).
[21] Data availability prevented me from analysing the dataset from Bangladesh.
[22] To reduce “noise”, Figures 1 and 2 depict household medians over the available rounds for Kenya and over four rounds for the Philippines.

Previous Page Top of Page Next Page