What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

Dear All,

It seems that the forum is warming up very well with great contributions from Surabhi, Shahroz, Arafat, Bulbul and John so far. Now building on Jim's suggestion as above, let me pose the following questions to the forum for further thoughts and opinions for this week -

What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

Please let us know what you think is the most effective way and why you think so.

Shahid Akbar
Shahid Akbar Bangladesh Institute of ICT in Development (BIID) Bangladesh


Dear Shehzaad,

Good to see all these contributions and I just like to add couple of points from our working experience in e-Agriculture.

I think there is no short cut way to measure impact o ICT4D initiatives. It is always complex to measure the impact, specially the direct impact (in terms of income or productivity) of ICT usage in development.

There are couple of critical factors in this process, one is considering the social-cultural factors including behavioral pattern of the target groups while setting indicators for measurement of impact, and second is consideration of changes in the eco-system of development dynamics i.e. integration of ICT in the value chain.

What do others think?




Shehzaad Shams
Shehzaad ShamsAmnesty InternationalUnited Kingdom

Dear Shahid,

With reference to Asad's opinion on having universal set of indicators, please share with the forum your thoughts - should we or should we not have a set of indicators which are uniform or treated as globally accepted across any development agency for any ICT4D project?

Or, starting with agriculture only, is it feasible to have a universal set of impact assessment for ICT impact on agriculture?

Thanks, the forum will wait to hear from you.

Shahid Akbar
Shahid Akbar Bangladesh Institute of ICT in Development (BIID) Bangladesh

Dear All,

Thanks Asad for bringing this very interesting issue of ''Universal set of indicators'' for impact assessment.

Surely, there are scopes to set some common indicators which may be benficial for different stakeholders like development agencies as well as private sectors. In some cases, we already experienced this kind of common or universal indicators in many initiatives, specially in MDGs.

In ICT initiatives for Agricultural projects, the very basic indicators mostly includes productivity issue which is little complex to identify and measure the magnitude of contribution. Based on last few years limited global experience of e-Agriculture initiatives, I am sure there are few indicators common in different initiatives which can be reviewed and studied further.

Since integration of ICT in agricultural initiative is a new trend, we have to be more innovative and flexible to come up with a set of indicators, particularly to define some universal indicators. 

Any other comments on this from the SMEs ?





Shahid suggests that "integration of ICT in agricultural initiatives is a new trend.." I would like to agree and supplement the comment by noting  that integration of ICT in agricultural initiatives goes back to 1956 - tracing to an inititive to integrate Linear Planning into agricultural management methodology. As I commented yesterday several imsightful indicators regarding the rate of ICT adoption, ICT impact on production and productity and innovation adoption in general are detailed in various areas of agricutural production in the free e-book "ICT in Agriculture: Perspectives of Technological Innovation"  http://departments.agri.huji.ac.il/economics/gelb-main.html                  

It is important to note that the book is NOT an historical review. It is geared to present overtime proven insights for ICT4D project decion makers engaged in evaluating their past activities and planning to undertake new projects.

As before comments are solicited and welcome.

Ehud Gelb gelb@agri.huji.ac.il

Asad Rahman
Asad RahmanBangladesh
Considering the present complex scenario of development all over the world the role of technology is becoming more and more prominent. Among all types of technologies the access of the mass ICT has been observed quite prominently. Once again information proved to be the key to many turning points of history starting from the revolutions in different countries like Yemen, Tunisia, Egypt, Lybia to Wikileaks.
The impact of ICT in development scenario is increasing exponentially. However the process of capturing impact ICT on development and it’s attribution is still at a very early stage. Considering the ICT4D practitioners and M4P practitioners the universal acceptance and recognition of different tools are needed to be gained. But before that the development of such universally accepted processes are needed, the tools and indicators that can be usable by the development practitioners all over the world for measuring impact.
I hope this forum would lead to that process and I would request participants to share their views regarding a universal set of indicators and tools for measuring ICT impact on development


Michael Riggs
Michael RiggsUN-APCICT/ESCAPRepublic of Korea

Hi. Asad, or someone else, would you please tell us what M4P means? Just to be sure we all have a common understanding of our terminology. Thanks!

Shehzaad Shams
Shehzaad ShamsAmnesty InternationalUnited Kingdom

Dear Asad,

Interesting when you mention about 'universal set of indicators' - do you mean to say that there should be a certain list of indicators to capture ICT impact on development across the board? Something which should be applied across any project, any country and across any context?

I wonder if you are indicating towards any kind of compatibility or uniformity between indicators used by different agencies for tracking the same kind of sought after impacts.

Would appreciate if you could share your thoughts.

Eric Couper
Eric CouperAfrica Soil Information Service (Columbia U. | Earth Institute)Tanzania

I see two different categories to consider when trying to measure "impact" of ICT4D initiatives.

I. Impact of Project

The first deals with the initiatives impact on the community, nation, market, etc. that it's trying to serve. In this way, I think you can begin by measuring ICT4D initiatives in the same way that you measure any development project. The same debates, considerations, and challenges apply. 

Within this impact analysis category, ICT4D initiatives often have an unique advantage. If smartly built, a ICT system will routinely capture a lot of important data that will make an impact analysis faster and cheaper.

II. Impact of technology

Secondly, it's important to consider the role that technology is playing in an initiative. ICT is a means, not an end, so it is crucial we ask ourselves if ICT is the correct tool for the job.  Could the same information, service, resource, etc. be delivered in a less ICTish way? Does the information, service, resource, etc. already exist without ICT?

A big part of this is the cost. Does introducing ICT lower the cost? But that isn't the only consideration. Perhaps ICTs speed up the process or make M&E easier or increase transparency. All of these benefits are tangible and should be measured/estimated before a new tool (in this case ICTs) are fully adopted.

Shehzaad Shams
Shehzaad ShamsAmnesty InternationalUnited Kingdom

Dear Eric,

Great the way you categorise the impact assessment indicators. Do you have any thoughts on these being able to be specified in the field of agriculture only? What I am trying to ask is are indicators for tracking ICT4D impact essentially very different in agriculture or all ICT4D interventions across any sector should have a general common set of indicators? What do you think?

Eric Couper
Eric CouperAfrica Soil Information Service (Columbia U. | Earth Institute)Tanzania

I'm very wary to suggest that all ICT4D projects or even all ICT4Ag projects can embrace a common set of indicators. Simply put, different initiatives have vastly different goals. To throw a blanket set of indicators over a project that connect farmers to markets and another that provides weather alerts risks a poor evaluation of both projects.

Even assuming that all projects are ultimately about improving the quality of life of farmers, good luck trying to demonstrate that your intiative did so.

That said, I do think there is important information that all development projects should publically release. Regardless of what your stated results, they should include a comprehensive look at how much money was spent and how. 

We assess projects for two reasons:

----So that donors and the public can understand better the results of our organizations' efforts.

----Other organizations can learn from our work and adopt similiar approaches if they are found to be sound.

Without financial information, there is no way that another organization can evaluate a projects fully. With finite resources, it is crucuial that we consider not only if a project works but how much it costs to make it work.