Michael,

You raise a really important question.  Given the proliferation of m-based agricultural MIS in many countries, we should be asking:  What is the impact of these services on people's lives?  (Traders, farmers and consumers?)  ?  And what do we mean by impact?

There are now four rigorous (independent) studies that try to address these very questions.  The studies span the globe, with two from India (Fafchamps and Minten; Mitra, Mookherjee, Torero and Visaria); one from Colombia (Camacho and Conover); and one from Niger (Aker and Fafchamps). Overall, three of the four studies measure the impact of mobile phone (SMS) based market information systems on farmers' well-being (prices received, crops), as compared to farmers who do not have access to the service.

While the programs and results differ across the studies, in general, the bottom line is the following:  1) access to price information via mobile phones reduces price differences across markets; 2) this reduction in prices doesn't necessarily translate into higher (farm-gate) prices for farmers; but 3) in some cases this allows farmers to have fewer losses or face less price variation over the year.  So, in general, these projects make markets work better (which is good for development), but they aren't always leading to improved well-being for farmers in terms of higher incomes (which is okay, unless the primary objective of the project is to increase farmers' incomes).  

Why do the authors see these effects?  The reasons vary across the studies and contexts, but in general, they are due to a variety of potential reasons, depending upon the context -- low take-up of the service; non-competitive markets (so even if farmers know the prices, they don't have much bargaining power); and 3) credit constraints (again, even if farmers know the prices, they are credit constrained and need to sell anyway).

Overall, I am not too surprised by the results of these studies.  We know that mobile phones can really reduce communication costs, which is wonderful in contexts that have limited infrastructure (power, roads and landlines).  And while providing information via mobile phones can overcome an important constraint for many farmers, information is necessary but not sufficient to improve farmers' well-being.  Other things -- like credit, more competitive markets -- are required.

As we move forward, I encourage us all to continue asking whether our m-based MIS projects are having a positive impact upon people's lives as compared with the traditional way of doing things.  If they are, great.  If they aren't, then we need to understand why and improve such projects.  But in order to answer these questions, we need to have a good monitoring -- and evaluation -- system in place, at least in the first few years, to ask (and answer) these hard questions.

I agree with Jim that logframes can be a very useful tool for evaluating the impact of ICT for agriculture programs.  While ICT is innovative, and offers new opportunities for outreach and engagement with program participants, at its heart, it is like a lot of other programs.

But there are some ICT-specific challenges that are necessary to consider while thinking through the program.  These might seem evident to all of you -- if so, please forgive me for repeating them -- but these are lessons we learned with our IMAC program (a mobile phone-based price information program) in Niger:  

1. Sometimes the ICT for ag projects involve developing new partners with the private sector.  How can we bring these partners into our planning process and support?

2. Does the program manager have the necessary technical support within the organization (or outside of it) to ensure that the ICT interventions run smoothly?  

3.  If we observe that farmers are "doing better" after the ICT for agriculture program, does this mean that we should replace the "traditional" way of doing things with ICT?  Or are they complements?  In other words, what would have happened in the absence of the program? (This is the attribution issue).  

We faced many of these challenges in our IMAC program in Niger.  We learned that figuring out these partnerships and ensuring that the necessary technical support is available is something that must be decided early on, during the project planning stage.  More is available here:  http://sites.tufts.edu/projectabc

In my opinion, one of the biggest constraints to learning about ICTs and development is the absence of an "information clearinghouse". There is so much information, in a variety of areas and on a variety of topics, that it can be difficult to find, understand and keep up with what is happening in the ICT4D sector in different fields. ICT-based projects are funded and implemented by NGOs, donors, the private sector and public-private partnerships; they span agriculture, labor, education, health and governance areas; there are qualitative and quantitative evaluations of these projects; and they are disseminated in different formats (journals, newspapers, blogs). This learning is so important, however, so that we can learn from potential "successes", especially in other sectors (m-health, for example); reduce duplication of effors in the same areas; and avoid repeating failures (or learn what those failures are, and improve upon them). mobileactive.org does a nice job of summarizing what is happening in this field -- is there room for another online information clearinghouse that could summarize (or link to information sites) ICT4D impact evaluations of different types? The issue is, of course, sharing negative evaluations -- this is always difficult to do.