M&E considerations at design stage

The design of an M&E system should begin at the same time as overall project preparation. As a general rule, the M&E system should be designed in close partnership with all relevant stakeholders as it contributes to ensuring that the project/plan objectives and targets, and how they will be measured are well understood and shared. This understanding can then potentially facilitate the establishment of new institutions to take on the M&E role. Adequate resources need to be allocated for implementation of M&E.

A successful M&E system must allocate the following:

• sufficient budget (for information management, participatory monitoring activities, field visits, surveys, etc.);
• sufficient time (for a start-up phase that is long enough to establish the M&E system, conduct  a baseline survey, train staff and partners, include primary stakeholders in M&E, monitor and reflect);
• sufficient capacity and expertise (to support M&E development, skilled and well-trained people required for good quality data collection and analysis) for M&E. If appropriate, external expertise in design for a baseline study and an impact evaluation should be engaged;
• sufficient flexibility in project design enabling the M&E system to influence the project strategy during implementation.

M&E considerations at implementation stage
Good practice of M&E during implementation requires that result indicators and target values have been well-defined and agreed upon in the result framework (see RBM). It is essential to establish a clear distinction at project design stage between outputs, outcomes and other higher level development objectives. This will ensure that selected indicators are appropriate to their respective level along the results chain and also help determine institutional responsibilities and timelines for M&E.

For each selected indicator, M&E tools (means of verification) have to be defined. Examples are semi-structured interviews; focus group discussions; surveys and questionnaires; regular workshops and roundtables with stakeholders; field monitoring visits; testimonials; and scorecards. Frequency and responsibilities for applying the tools, for analysing relevant information and for reviewing this information must be specified in an M&E plan.

Figure 1: Comparison of monitoring, review and evaluation

Baseline study and related impact evaluation

Conducting a baseline is a necessary reference point to assess change. A baseline study defines the benchmarks against which project progress is to be measured. Depending upon the results indicators of the project, the baseline study may record food security characteristics, incomes and basic household assets and services, environmental parameters and/or productivity and profitability levels. Subsequent mid-term, end-of-project and impact assessments measure key  outcomes and impacts with respect to changes from the baseline (see Impact Evaluation).

M&E for learning and performance improvement

M&E systems will only add value to project implementation through interpretation and analysis, by drawing on information from other sources and adapting it for use by project decision makers and a range of key partners. Knowledge generated by the M&E efforts should never stop at basic capturing of information or relying exclusively on quantitative indicators, but also to address the “why” questions. Here the importance of more qualitative and participatory approaches become particularly important, to analyze relationship between project activities and results.  Evaluation therefore serves the purpose to establish attribution and causality, and forms a basis for accountability and learning by staff, management and clients. The challenge early on during project implementation is to design effective learning systems that can underpin management behavior towards results, and come up with strategies to optimize impact.

In this context, learning is defined as formulating responses to identified constraints and implementing them in real time. Useful at project/plan level are knowledge-sharing and learning instruments which can pick up information and analysis from the M&E systems as studies,  such as: i) summarized studies and publications on lessons learned; ii) case studies documenting successes and failures; iii) publicity material including newsletters, radio and television programmes; iv) formation of national and regional learning networks; v) periodic meetings and workshops to share knowledge and lessons learned; vi) research-extension liaison or feedback meetings; vii) national and regional study tours; viii) preparation and distribution of technical literature on improved practices; and ix) routine supervision missions, mid-term reviews or evaluations and project completion (end-of-project) reports (see Implementation support).

Reporting and data collection in a management information system

Project interventions in dispersed and often remote locations, diverse implementation units, and collaboration with multiple partners and the complex M&E requirements pose important challenges to project managers and other stakeholders for tracking events on the ground. However, without information on outreach achieved and outputs or services delivered, it would be difficult to decide on appropriate responses, or make needed adjustments and refinements to plans, implementation approaches and strategies. To maintain good oversight and manage information on indicators and systems used for data collection, recording and reporting, projects therefore turn to management information systems (MIS). Whereas all projects use Excel spreadsheets as MIS, once a project reaches a certain size it becomes more convenient to use a web-based solution, with a more complex database system. The MIS and the M&E system complement each other, as the MIS supports: 

• day-to-day business processes;
• transaction processing;
• management control;
• budget planning;
• tracking progress against expected targets and thereby comparing performance between different regions where the project is implemented;
• systematic data capture, storage and retrieval;
• flagging implementation issues and alerts;
• communication among stakeholders; and
• accountability to project management and stakeholders with regards to reliable project data.  

While the MIS helps to identify output indicators and basic outcome indicators, the M&E system draws from MIS data to assess progress in achievement of results and impact and to support learning and adaptation. Additional inputs for the M&E system to produce the desired information are surveys and other interactions with stakeholders on a periodic or ad hoc basis, such as thematic studies, baseline and impact surveys and stakeholder workshops. These often need to use basic and reliable MIS data to develop survey frameworks, and to assess basic assumptions on scope of outreach etc. 

Internal and external project responsibilities and capacities required for M&E

As various actors are involved in the implementation of the M&E system, each has specific roles and responsibilities namely: programme implementers and technical officers generate progress reports on the components they manage; the planning unit sets targets for data and information requirements with input from technical specialists and allocates financial resources for the independent or integrated M&E unit; the M&E unit collects the information from implementers and beneficiaries and submits consolidated project reports to main stakeholders (in close coordination with the planning unit); project stakeholders analyse progress made in order to identify issues and opportunities and revise the implementation strategy accordingly.

For the system to work, planning and M&E units must analyse and report in close coordination on results, as well as constraints and bottlenecks, and draw and implement lessons for continuous improvement. Project-internal M&E specialists have to be equipped to lead work related to progress/implementation monitoring. Often the M&E unit takes on the role of guiding detailed project planning and target setting without the clear mandate or responsibility to do so, as planning units are not equipped to do so.

External specialist expertise may be relevant in areas such as:

• design, development, operation and maintenance of project MIS;
• capacity strengthening for M&E in general (statistical capacity, etc.);
• process monitoring;
• results monitoring and sample-based validation of findings of progress/implementation monitoring;
• thematic studies, including economic analysis (see FEA);
• baseline, mid-term and end-of-project impact studies and reports; and
• design and piloting of participatory M&E tools for institutions, and training.

The implementing unit may hire the services of specialized agencies to undertake these activities. However, the final responsibility to maintain oversight and act on the results of such activities as for MIS development lies clearly with project management.

Gathering information at the results level of each investment plan component facilitates oversight of the technical and financial performance of projects, programmes and components [this is operational M&E]. It is particularly important to monitor the financial progress of ongoing investment projects because investment plans are often not fully financed from the start and can therefore serve as a tool to mobilize resources. Regular analysis of budgetary allocations and disbursement is necessary, as well as monitoring and updating the gaps between them and recording existing financial commitments [this is strategic M&E].

Critical reflection on and strategic communication of M&E findings are fundamental aspects of the results-based management approach, especially given that investment plan assessments provide the evidential basis for building consensus and raising awareness between stakeholders. The assessments also generate lessons to inform and influence future sector and country assistance strategy planning and/or revision. 

An example of regular monitoring of a national investment plan can be found in Bangladesh. Since its adoption, the national authorities have produced annual reports covering the entire national policy and investment framework. They are reporting on progress towards national food security and agriculture targets as well as on the financial commitment and disbursement. The relevant material can be found on the Food Planning and Monitoring Unit of the Ministry of Food’s website.

With regards to evaluation, there have been various initiatives to quantify the impact of the Comprehensive Africa Agriculture Development Programme on e.g. agricultural expenditure, agricultural value-added, land and labor productivity, income, and nutrition. This is always done using country-level data (International Food Policy Research Institute discussion paper).