The SLA Forum was evaluated with the use of a questionnaire (Annex 13), sent by e-mail to all participants immediately after the Forum. Of the 71 participants polled, 25 (35.21 percent) responded. The questionnaire included four open-ended questions and 23 scaled questions. The results of compiling the responses to the scaled questions are included in tabular form in Annex 13. The open-ended questions were concerned with the ways in which the Forum benefited the participants, what the participants thought were the strengths and weaknesses of the Forum, whether they wanted a follow up to the Forum and, if so, of what nature.
Fifty-two percent of the respondents felt that the Forum had met their expectations "very well" and 28 percent went further and said, "exceptionally well". In what ways did they benefit? There were basically two kinds of responses to this query. The first set of replies focused on what was learned about SLA, the opportunities and the directions such learning provided for future work, the experience of synergy (of minds), the excitement of the emerging convergence of thoughts and reaping the benefits of networking. The second set of replies referred to the actual processes in the Forum and how they made a difference.
The learning began with a better understanding of SLA, which "opened the mind to a range of perspectives and possible applications of SLA". Forty-eight percent of the respondents felt that the Forum had contributed "very well" to their learning about SLAs, while 28 percent rated their learning as "exceptional". Respondents said that the Forum had given them the opportunity to "reflect on current development approaches and to measure these against SLA", which not only reinforced the confidence they had in their own best practices but showed ways of overcoming weaknesses in existing practices. Such reflection also revealed the implications of mainstreaming SLA in organizations with a wide range of mandates and working cultures.
The impact of the Forum on the participants' views of SLA can best be judged when considered with the level of initial scepticism about the relevance of SLA. Forty-four percent of the respondents came to the Forum with very high levels of scepticism and almost an equal percentage ranked their scepticism "fairly sceptical" to "sceptical" - all in all an extremely doubting group. In the end, 68 percent of the respondents felt that the Forum had had a very positive impact on their view of SLA.
The coming together of people from different agencies and disciplines and the high level of synergy was exciting and satisfying in and of itself, but it also showed, the participants felt, that "a convergence was emerging from the variety of approaches amongst the various agencies", which was very positive. The Forum experience helped to "create contacts, excellent links and a common base of understanding with colleagues in other agencies", paving the way for cooperation in the future.
Some participants said that the most beneficial thing about the Forum was the process itself. Most respondents, hinting perhaps at the paucity of opportunities in most agencies for people of different disciplines to have the time and space to meet and reflect on their work together, felt that the "free and frank expression of opinions", "the intense interaction with like-minded people leading to iterative and reflective thinking", and "the right combination of participation, healthy scepticism, critical thinking, positive energy and a sense of fun" was a reward in itself.
The open-ended question, "what in your opinion were the strengths and weaknesses of the SLA Forum's design, organization and implementation", along with the scaled responses of questions 3.2 to 3.20, generated a lot of interesting views and insights, which would prove useful to organizers of similar events.
Clearly the participants liked the venue of the Forum. Seventy-six percent thought it "exceptional" and 16 percent thought it "very good". The venue was not only beautiful and peaceful but also far removed from the day-to-day realities of most participants, giving them the sense of freedom and space that nurtures discourse. Duration was another matter. When one sets out on a Tuesday afternoon and returns on Saturday evening, you essentially write off the week. Was it too long? Was it too short? Fifty-two percent felt it was just right in length, with almost equal numbers feeling it was either too short or too long. The open-ended question brought out a wide variety of thoughts on the Forum's duration: "too little time for discussing the crosscutting issues"; "I was sceptical about spending two whole days on a case study, however, I was wrong because it was time well spent and it was an excellent way to approach the task"; "too much was stuffed into the four days, especially on mini-case study presentations, which while being informative and entertaining were not really useful"; and "poor time management".
Sixty percent of the respondents rated the quality of participation as "very high", with 20 percent pushing that rating up to "exceptional" and another 20 percent indicating "just good". Some participants saw the diversity of disciplines and levels of experience with SLA as a weakness (because it was difficult to move the sceptics and newcomers forward), whereas others saw it as a strength (because not knowing sometimes empowered them to ask questions and challenge strongly held beliefs that even made "believers" revisit their assumptions). A few respondents felt that having more representatives from agencies with SLA experience among the participants could have strengthened the discussion at the Forum.
Participants felt that the Forum was well designed and implemented, particularly given the large number of participants and the complexity of the ideas under discussion. Several participants complimented the spirit both of the organizers and of the participants in terms of their receptivity and openness. The high degree of energy and motivation of the participants made for rich discussions that spilled over into free time and had participants talking animatedly and socializing often into the wee hours of the morning. The participatory nature of the Forum, where participants could adjust the schedule and content depending on need, provided a flexibility and freedom that added to the spirit and intensity of the discussions and empowered the participants to meet their expectations. However, a few participants complained that the Forum lacked discipline, as in its failure to stick to its original programme, and felt that occasionally it was difficult to decide where participation ended and anarchy took over, but even these participants said that (luckily) in the end everything turned out fine.
People came to the Forum with a wide range of expectations, and some participants felt that the purpose of the Forum could have been sharper and clearer. A few complained that while the discussion was rich and intense, a lot was lost because the thoughts, findings and recommendations were not adequately documented and distributed as the meeting progressed. A few criticized the Forum for spending too much time discussing diagnostics and design (in which quite some experience has been built up in SLA) and too little time dealing with important areas such as implementation and sustainability (in part due to the lack of experience). Many felt that the Policies, Institutions and Processes box in the DFID SL framework remained a "black box" to the end with little illumination from the Forum. There were some who felt that the Forum had focused too much on particular agency frameworks and had not looked enough at historical, cultural and linguistic antecedents outside the anglophone tradition. The responses to questions 3.7 to 3.20 (Annex 13) provide an indication of the respondents' feelings about the materials that were distributed and used at the Forum.
Two contributions from respondents perhaps best describe the main strengths of the Forum's design, strategy and implementation: "the mutually reinforcing sequence of sessions, each one building on the outputs of the earlier sessions with a high degree of flexibility to respond to emerging consensus and participants' needs", and "the immense amount of work that had gone into preparations and the provision of facilitators and resource people, which increased the participants' ability to concentrate on ideas and be creative".
There was a near consensus on the need for follow-up to the Forum. While supporting and emphasizing the need for the individual cooperating agencies to take forward their own agendas as spelled out at the Forum, the respondents felt that inter-agency fora, especially the more informal ones like the IWG-PA, would be the ideal vehicles for moving forward the development and evolution of SLA.
A wide variety of suggestions were made as to the type of follow-up necessary. These include:
The evaluation was extremely positive, while providing enough constructive critical feedback to make the organizers ponder how they could have avoided the odd problem or done things better. As one participant put it, "Even though there were some problems, and there always are, I cannot remember participating in a workshop where preparations and arrangements had been so well thought out". Another said, "Participating in the meeting was a very pleasant activity: the organization, the venue, the people, the general atmosphere, the wine, everything contributed to create just the right environment to work and we did work hard".
