As other participants have pointed out, technology and especially mobile devices open up enormous opportunity for capturing data at a low cost. While on its face, this good news, the ease with which mobile devices can capture numerous indicators and vast quantities of data has often been a detrament to good design in M&E in the ICT4Ag field. Interventions often capture far more data than they can effectively analzye and sometimes confuse capturing data with interpretting it in ways that improve results in the field. In that regard, the previous post on keeing it simple provides some easy to implement techniques to avoid the trap of measuring everything.

Another temptation ICT4Ag projects seem to fall into, as noted in other posts, is the tendency to over rely on output measures, and especially absolute numbers reached, and to then equate large scale and lower cost/interaction with impact. Because technology is a means for achieving low-cost scale, it's not surprising that many ICT4Ag initiatives quickly reach a scale that other Ag4D projects cannot. However, not enough effort has been put into measuring and communicating outcome related indicators or proxies associated with interactions generated from ICT.  

Lastly, ICT4Ag initiatives often jump directly to measuring impact before getting the operations right. While it's important to build in outcome measures from the start, understanding what's working and what's not and where users see the most value, should be a high priority during early stage piloting and scaling.

The previous post which suggests measuring user satisfaction, which can be measured via repeat usage, is a solid approach for capturing the value that users percieve. It is also one that can "fall out of the data" if designed well since most technology products capture each interaction and associate that interaction with a unique user, this is particularly true when users are paying for the service. Disaggregating usage by poverty level, gender and other characteristics, which can be captured via mobile registrations, can help practioners better design and target services. Analyzing usage by product or content area (e.g. cattle diseases vs. chilling hub services) can help programs zero in on where to focus new content and services and when to course correct. These types of insights should serve as the basis for deciding what to scale but unfortunately are often frequently overlooked in the rush to scale.

Similarly, capturing adoption of best practices that have proven ranges for productivity and/or quality increases can be another way to get at outcomes in a short period while awaiting phase II, more costly Randomized Control Trials and other impact assessment approaches.

Finally, in our own projects, the most effective indicators have been those that value chain players already measure. By monitoring changes in key indicators that are important to value chain players and that are measured regularly as part of doing business, for example number of boxes of bananas rejected for ripeness issues each week, we can assess how particular ICT4Ag efforts (for example sending SMS on when to harvest bananas) are affecting results and tie those results back to farmer returns (e.g. a decrease in number of boxes of rejected bananas is tied to an increase in weekly payments to a farmer, both of which are already measured by the commercializer). Linking these types of indicators back to a cost-benefit analysis that captures the cost of introducting technology into business processes can also make the business case for ICT4Ag initiatives by not only measuring impact but also demonstrating value that commercial players will be willing to pay for.