What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

Dear Shehzaad,

Good to see all these contributions and I just like to add couple of points from our working experience in e-Agriculture.

I think there is no short cut way to measure impact o ICT4D initiatives. It is always complex to measure the impact, specially the direct impact (in terms of income or productivity) of ICT usage in development.

There are couple of critical factors in this process, one is considering the social-cultural factors including behavioral pattern of the target groups while setting indicators for measurement of impact, and second is consideration of changes in the eco-system of development dynamics i.e. integration of ICT in the value chain.

What do others think?

Thanks,

Shahid 

Dear All,

Thanks Asad for bringing this very interesting issue of ''Universal set of indicators'' for impact assessment.

Surely, there are scopes to set some common indicators which may be benficial for different stakeholders like development agencies as well as private sectors. In some cases, we already experienced this kind of common or universal indicators in many initiatives, specially in MDGs.

In ICT initiatives for Agricultural projects, the very basic indicators mostly includes productivity issue which is little complex to identify and measure the magnitude of contribution. Based on last few years limited global experience of e-Agriculture initiatives, I am sure there are few indicators common in different initiatives which can be reviewed and studied further.

Since integration of ICT in agricultural initiative is a new trend, we have to be more innovative and flexible to come up with a set of indicators, particularly to define some universal indicators. 

Any other comments on this from the SMEs ?

Regards,

Shahid

Shahid suggests that "integration of ICT in agricultural initiatives is a new trend.." I would like to agree and supplement the comment by noting  that integration of ICT in agricultural initiatives goes back to 1956 - tracing to an inititive to integrate Linear Planning into agricultural management methodology. As I commented yesterday several imsightful indicators regarding the rate of ICT adoption, ICT impact on production and productity and innovation adoption in general are detailed in various areas of agricutural production in the free e-book "ICT in Agriculture: Perspectives of Technological Innovation"  http://departments.agri.huji.ac.il/economics/gelb-main.html                  

It is important to note that the book is NOT an historical review. It is geared to present overtime proven insights for ICT4D project decion makers engaged in evaluating their past activities and planning to undertake new projects.

As before comments are solicited and welcome.

Ehud Gelb g[email protected]

Hi. Asad, or someone else, would you please tell us what M4P means? Just to be sure we all have a common understanding of our terminology. Thanks!

I see two different categories to consider when trying to measure "impact" of ICT4D initiatives.

I. Impact of Project

The first deals with the initiatives impact on the community, nation, market, etc. that it's trying to serve. In this way, I think you can begin by measuring ICT4D initiatives in the same way that you measure any development project. The same debates, considerations, and challenges apply. 

Within this impact analysis category, ICT4D initiatives often have an unique advantage. If smartly built, a ICT system will routinely capture a lot of important data that will make an impact analysis faster and cheaper.

II. Impact of technology

Secondly, it's important to consider the role that technology is playing in an initiative. ICT is a means, not an end, so it is crucial we ask ourselves if ICT is the correct tool for the job.  Could the same information, service, resource, etc. be delivered in a less ICTish way? Does the information, service, resource, etc. already exist without ICT?

A big part of this is the cost. Does introducing ICT lower the cost? But that isn't the only consideration. Perhaps ICTs speed up the process or make M&E easier or increase transparency. All of these benefits are tangible and should be measured/estimated before a new tool (in this case ICTs) are fully adopted.

I'm very wary to suggest that all ICT4D projects or even all ICT4Ag projects can embrace a common set of indicators. Simply put, different initiatives have vastly different goals. To throw a blanket set of indicators over a project that connect farmers to markets and another that provides weather alerts risks a poor evaluation of both projects.

Even assuming that all projects are ultimately about improving the quality of life of farmers, good luck trying to demonstrate that your intiative did so.

That said, I do think there is important information that all development projects should publically release. Regardless of what your stated results, they should include a comprehensive look at how much money was spent and how. 

We assess projects for two reasons:

----So that donors and the public can understand better the results of our organizations' efforts.

----Other organizations can learn from our work and adopt similiar approaches if they are found to be sound.

Without financial information, there is no way that another organization can evaluate a projects fully. With finite resources, it is crucuial that we consider not only if a project works but how much it costs to make it work.