Project “Advancing Evaluation Theory and Practice in South Asia – Building a Community of Experts”
Six Monthly report
October 2008 to March 2008
Submitted to:
International Development Research Centre (IDRC)
208, Jor Bagh, New Delhi
By:
Association for Stimulating Know – How (ASK)
V30 /3, DLF – III, Gurgaon, Haryana
Introduction
ASK is implementing the project “Advancing Evaluation Theory and Practice in South Asia – Building a Community of Experts” for a three year period beginning April 2008. The Project aims to facilitate a systematic and dynamic exchange amongst evaluation experts in the South Asian region expanding the knowledge and experience base of individuals and of the region as a whole and provide opportunities to share these experiences with the International Evaluation community.
The objectives of the project are to:
Ü Create a space and platform to stimulate dynamic exchange and interaction among evaluation experts in South Asia
Ü Increase the knowledge and experience base of individual experts and the organizations they represent
Ü Expose the evaluation experts and practitioners in South Asia to the knowledge and experiences of international evaluation experts, creating new links and expertise
Ü Introduce new evaluation approaches and methodologies in the evaluation field to South Asian evaluators to foster experimentation, innovation and modification of these methods
Ü Foster interaction and collaboration between evaluation experts, practitioners and other development stakeholders (the multi lateral and bilateral agencies and other international, national and regional organizations, policy makers) to promote dialogue on improved quality and use of evaluation
Ü Promote utilization of the improved and systemized knowledge by the evaluators (both experts and new practitioners)
Ü contribute towards improving evaluation quality in South Asia, by promoting increasedrigour in design and implementation of evaluation (and evaluation research) through the project seminars and trainings
Ü Document and publish the experiences and insights of these South Asian evaluation experts, contributing to the field of development evaluation and research through articles, evaluation research pieces, books, and other forms of documentation
Major achievements of the project in relation to the outputs planned for the first year
Note: One of the major event of the project was organized in April 2009 which technically was a part of the second financial year. However in order to provide a more complete picture, major highlights and progress made during and around the second event have also been included in this report.
Formation of steering committee
The steering committee is in place with 6 members. The steering committee had had its first meeting on December 13th, 2008. The two members from outside of ASK and IDRC, namely Ms. Mallika Samaranayeke and Ms. Rashmi Agrawal are actively involved in the activities of the CoE. Ms. Mallika Samaranayeke who is the President of Sri Lanka Evaluation Association (SLEvA) actively cooperated in the organization of the meeting of the CoE in April. The CoE meeting was organized back to back with the SLEvA international conference and professional development Courses. This allowed CoE to be made more visible before the evaluation practitioners from across the South Asia region and beyond.
Dr. Rashmi Agrawal also attended the CoE meeting and represented the CoE at a panel presentation on CoE at the SLEvA conference.
Establish and make functional a “community of experts”
The Community of Evaluation Experts has been formed with 29 members, besides ASK representatives. The country wise breakup is as below:
S. No / Country / Number of members1. / Bangladesh / 7
2. / India / 16
3. / Nepal / 3
4. / Sri Lanka / 3
Total / 29
It is important to share here that the members have decided to keep the name of the community as “community of Evaluators” and not “community of evaluation experts”.
The members have been selected in two rounds based on a selection criteria. The first round ended in October 2008 with 23 members and round two ended in March 2009 with an addition of 6 members.
The members have now had two meetings. The first meeting was held within the period of this report from December 11 to 13, 2008. The December meeting achieved the following:
· Familiarization of the members with each other, with ASK as a facilitating organization and with IDRC as a support organization.
· Agreement on the Goal and purpose of the CoE as well as on the Progress Markers. All of these were developed by the CoE members during the meeting
· Preparation of Action Plan for CoE
· Deciding the norms for CoE functioning
· CoE name, logo and website
The December meeting was also utilized to spark off discussions on aspects and issues important from the point of view of Evaluation practice in the region. The members held discussions on the following aspects:
· Evaluation context in South Asia (and in the countries to which the members belong)
· Making Evaluation participatory and use focused
One of the important follow up actions on which the members worked during the period following the CoE meeting was to initiate writing of papers on the above two topics and also on “Evaluation Ethics and Guidelines”.
These papers were presented by the members in the second meeting in April, 2009.
A copy of the proceedings of the first meeting is attached as annexure 1 with this report.
Area of concern and steps planned
One of the matters of concern at this stage is that not all members are active in the CoE. Two members have not attended any of the two meetings held so far. There are quite a few other members whose contribution during the meeting is not very strong in terms of contribution in discussions during the meeting or in the period in between the meetings through internet and virtual platforms.
It was discussed during the April 2009 meeting in Colombo to develop a performance appraisal system. This will be worked upon in the days to come and proposed to the members.
Another way out is to still improve the quality of membership in terms of know- how and expertise. The Colombo meeting provided useful leads on this front. Good contacts could be established with professionals and organizations active in the field of evaluation. It has been decided that ASK will target specific organizations (31e, J-PAL) and groups (Curriculum Development Group working at South Asia level to develop curriculum for Evaluation courses). ASK will also target countries which do not have representation currently in the CoE. This will include Pakistan and Afghanistan.
Signed agreements with the community of experts
It had been decided during the course of implementation of not signing formal agreements with the members. However members have communicated through emails their consent to terms of reference communicated to them.
Selection and design of virtual platform to supplement community of experts
CoE web portal has been designed and is now active. The website has various features to make it interactive. Besides information on CoE, its objectives and activities, it also provides space for members’ updating their profiles and discussing issues related to evaluation with each other. The website also has a facility for the project staff to provide information to members and non members about new postings on forthcoming Evaluation events of CoE and others and also Evaluation opportunities.
Besides the CoE web portal, the project also has a list serve to reach out to all the members. A very new beginning is a list serve for non members but this one yet to become active.
Areas of concern and challenges
The challenge is to make the community members actively interact through the web portal and the list serve. The project team will work on this aspect in the forthcoming months.
Another challenge is to ensure that the members are active between the meetings. One of the ways to do this will be to set in motion the decisions that have been taken to work on the three tracks, namely the “Writing Track”, “Training Track” and “Practice Track”. It has already been decided to form sub groups for the first two tracks which will work together towards a South Asian Volume on Evaluation and an International Training in March 2010. In response to a mail from the project team, interested members have expressed their willingness to be a part of the sub groups. Based on the credentials, the members will be shortlisted. One of the possible actions may be to invite a meeting of the sub group to allow them to set a course of action to work towards the two items on the agenda.
Positioning ASK as a credible organization
Efforts have been made in this direction during the first year of the project. The members have been first of all reached out to. A visit to ASK of the members was facilitated during the first meeting in December 2008 and a sharing of the work of ASK was done. An effort to enhance the credibility of the organization has also been made by effectively and efficiently conducting the two meetings. The feedback of both the meetings has been very positive from the members.
The next task has been to enhance the credibility in the outside world of evaluation. There have been some efforts in this direction but much more needs to be done. In fact, it was concluded in a review meeting held between the project team and Katherine Hay of IDRC on March 24th that the project team needs to work towards creating much greater visibility of the project, its objective and of the CoE. The SLEvA conference was a good opportunity to work towards this and the following steps were taken:
· Dr. Aqueel Khan, Director of ASK presented a theme paper on the Status of Evaluation in the NGO and Private sector.
· Khilesh, the Project coordinator, led a panel with other members of CoE, sharing about the objectives and activities of CoE. There was immediate response and the curriculum development group in South Asia expressed interest in becoming a part of the CoE. They have been invited already and Project team is awaiting their response.
· Khilesh was invited to attend a meeting with the Afghanistan contingent to discuss the state of evaluation in Afghanistan. The project team will follow up with the Afghanistan participants in the conference, trying to enrol two members.
· Eminent speakers, Dr. Howard White of International Institute for Impact Evaluation (3ie). and Dr. Sanjeev Sridharan had been invited to make presentation to the CoE. Both of them acknowledged the role of CoE. Dr. Howard White has in principle accepted the membership of CoE for 3ie.
· Project team has shared about the CoE with the organizations and Evaluation professionals on informal basis wherever possible.
But much work needs to be done on this and some of the plans are as below:
· Write to international and national organization, informing them about CoE, future plans and inviting them to the SouthAsiaEvalgroup, the email list serve created for non members.
· Write to Evaluation practitioners in South Asia through existing list serves urging them to enrol to soutasiaevalgroup.
· Participate in evaluation discussions, both virtually and physically
· Constantly update the CoE website with new developments
One seminar proceedings published
The events which were held were not strictly seminars, although the second meeting had presentation of papers by the members. The first meeting, besides having sessions for community building, also had sessions to discuss thematic issues of making evaluation participatory and use focused. The proceedings of the first meeting were prepared, finalized and placed on the website.
The report of the second meeting are being prepared. It must be mentioned here that the second meeting has only been held in the second financial year (in the thirteenth month). So in effect, only one meeting was organized instead of the planned three events. This was because the project followed a natural flow of activities in selection of members which took until October and the first meeting was planned in middle of December. The second meeting could have been organized in March, 2009 but was postponed by a month to coincide with the SLEvA conference.
Besides, as part of the future plan of action it has been decided that CoE will work on the “Writing Track”. The main task as part of the “Writing Track” will be to publish a South Asian Volume on Theory and Practice of Evaluation. There will be a sub group constituted to coordinate the functioning of the “Writing Track”. The members have volunteered to be on the Writing sub group and it will be finalized soon. It may be an option to invite the volunteering members for a preparatory meeting so that they may decide the process to steer the process further.
IDRC has provided access to all the members to the entire resource library. This will assist the members in their own knowledge building and also when they write any papers.
Two web newsletters circulated
The preparation and circulation of web newsletters has not been done. However the website has been activated. The highlights of the second meeting in Colombo have already been updated on the website. The newsletter will be prepared now and circulated. The first newsletter should be out by August 2009.