Do you explicitly construct logical and linear paths for your programmes?

I guess this is very important issue that is perhaps somehow getting missed in the discussion. I have just come back from an internal discussion where we have tried to map how the changes we have introduced in the ICT market system for the poor, have been internalized by the markets we are intervening in. And what we arrived at is that this process of internalization by the market system is anything but linear. While it is perhaps prudent to have a game plan in mind in seeing how the changes should happen (and that perhaps could be linear), it is far more realistic and practical to see from a logical or result chain perspective. Even then I would argue that there are certain other dimensins, qualitative, which you as implementator need to be aware of, if you are trying to capture your changes proprely.

Overall the point I am trying to make is that by thinking it in a logical path has certain advantages but also be mindful that you need to keep track of many other changes in the market systems, which could be qualitative and which are perhaps very important for you to understand how much effect your interventions are having on the market system itself.

First of all let me confess. I am not an ICT4D professional. Hence my knowledge is somewhat limited. However based on what I have seen and the implementation that I have done for ICT initiatives in agriculture, my assessment is more often than not, under ICT4D, indicators that predominate generally concentrate on ICT indicators like service market adoption, penetration or new services offer, users satisfactions with ICT services etc. While that is all very essential, it perhaps stops at what happens after that; what is the benefit the users experiences after using the service or more explicity how do the lives of the poor change. One could argue that there are far too many attribution challenges in doing such exercise however I see this differently. In our project, we make all the effort to relate how adoption of the ICT services we are promoting are impacting on the poor. I agree that there are difficulties but in the end it is possible to draw the logical chain and measure the impact. In agriculture we have found that availing agriculture information services largely benefit poor farmer in having better crop care leading to lower risk of crop loss through diseases. Doing such measurement at the last mile more importantly also allows us to focus on what we should do best and how we should we design and implement our interventions. 

Hence coming back to your question I don't think it is that much to do with agriculture,although perhaps it does make it easier to measure. It has more to do with who are your target beneficiary and to what level changes do you expect and how do you go about measuring it.

Dear All,

Thanks Shahroz for giving some practical references from Katalyst experience.

Its quite interesting to see diversified 'impact' in ICT based agriculture interventions or initiatives. We do see it from two different perspectives, one is more on ICT component based and another is impact on agriculture. And these two perspectives are equal important for the practitioners, specially for the private sector players. We feel more comfort to see that ICT component (in terms of technology, innovation, service) function properly, and if it works successfully, we can frame it to become more beneficial for the target customers. In private sector we have to consider that there is no other options rather ensure 'benefit of usage' to the customers, so I think it is much complex from the development perspective to define the indicators and measure impact in ICT interventions in agriculture. 

Lets hear more from the practitioners and private sector.

Regards,

Shahid

Just as we have seen a lot of the discussion on the measurement of impact has been referring to Agricultural Extension, I feel that when we are dealing with assessment of impact of technologies (whether agricultural or information & knowledge management technologies) in the agricultural and rural development context, we are privileged to be able to go back to see how agricultural extension has dealt with the issues.

ICTs are tools that enable us to do things in info & knowledge management faster, targeted or broadcast, including fast feedback and interactive, which are now portrayed as ICT-enabled services. Yet, these tools are still enabling us to do many of the things we have been trying to do in Agricultural Extension.

We can still relate to the steps involved:
1. Believing we have understood the information needs of our stakeholders, for them to achieve some socio-economic development,
2. We gather, package and present to the user, the information that we feel is required
3. We expect them to be able to assimilate what has been packaged for them - or we hope that they will assimilate it in the same way we would if we put ourselves in their shoes;
4. We expect the information to lead to a learning moment for the user, which will increase their tacit knowledge to a certain extent
5.We expect that the increased knowledge will lead to an assessment of the situation, a change in attitude, and hopefully a behaviour we thought was desirable in order to achieve the socio-economic improvement

Now, whether we are talking of static web pages or dynamic market information on the web or through sms, we are still in the business of providing information, reacting to feedback we receive to focus the next round of information provision etc. It requires that we deepen our knowledge and understanding of the use of the information we provide until it can lead to impact. This may not be the case as we start off a project or when we develop our log-frame (so we include a process of validation of our understanding in the project activities)

But, in the end, we realise that at each and every step of the process, there is a possibility of the targeted stakeholder not following the path we have traced for them in our conceptual model. This is a risk that agricultural extension has been dealing with throughout its existence, so lets not re-invent the wheel and learn from their wisdom.
(I will get to the point in the next post, it was too long)
 

Although this is difficult to admit at the time one is preparing a project proposal and developing the log frame, we have to point out that with experience in applying ICTs in agric information management, there is an inherent risk that the targeted beneficiary may stray from the path way we have mapped out for them in our intervention. This has to be factored in any such project

However, with formative evaluation incorporated in the process, we may have tools to correct our approach and pathways, and if need be also document how we have identified the bottlenecks and the other factors  (or additional information) that have caused the information not to have the desired effect we hoped, or achieved it to a certain extent only

So, if we had to provide tools for impact assessment, we should take the evaluator for the road trip that the information packages went through, to better appreciate how effective the interventions have been. [It reminds me of the film 'The Adjustment Bureau' where there are multiple pathways towards a desired objective and we are trying to guess the best way to lead an individual towards a desirable outcome]

Therefore, to go back to learning from extension approaches, I still find that the Targeting Oucomes of Programs (http://citnews.unl.edu/TOP/english/index.html ) is still a very valid approach to guide any formulation of evaluation and impact assessment of information management programs. The TOP enables the conceptual framework used to design the information provision approach to be described on one hand and on the other hand it enable each step in the process also to be evaluated for its effectiveness in leading to the desired outcome. It is one way of trying to render a complex system linear by stating that 'this is the pathway I think will lead the group to its destination, but these are the possibilities of losing members of the group from that pathway, along the way'. So with that model in mind, we are implementing a program, but we will also be monitoring what is happening so that we can redraw the pathway ahead (or sidepaths to join the main pathway again) to ensure that we have as many individuals of the group completing the journey

The TOP may be complex, but as we read earlier, some of the tools we have today can also provide us with the monitoring information as we move along, so it may not be as demanding on the staff involved if the automated processes and feedback systems are designed to capture the information as the user proceeds with this knowledge gathering journey, which leads to behaviour change and socio-economic progress. To me, this is one way we can make the processes explicit to the evaluator of impact of the program
Krishan Bheenick