Open Society Institute – Assistance Foundation
A Z E R B A I J A N
belongs to the Open Society Foundation Network established by famous investor and philanthropist George Soros. The main goal of OSI-Azerbaijan is to foster transition of a closed society to a more open one. Since its establishment in the country in 1996, OSI-Azerbaijan has encouraged the development of the third sector by awarding grants and through operational activity to support civic initiatives in education reforms, communication technologies, human rights and rule of law, mass media, public health, gender equality and arts and culture. Responding to new challenges of the country development, OSI-Azerbaijan has recently concentrated on increasing civil society involvement in the democratization process, good governance and transparency of the use of national resources.
Baku
May 2005Introduction
Resource-rich states throughout the world face an unusual paradox. Natural resources, once touted as a blessing for poor countries, have more often contributed to poverty, violent conflict, corruption, and repression. The Open Society Institute sees the transparent use of revenues generated by the sale and transport of natural resources as well as a related social and environmental impact as issues of great importance for development and the promotion of civil society. Open Society Institute seeks to build the capacity of local groups to monitor oil revenues and hopes to ensure that existing and future natural resource revenues be invested and expended for the benefit of the public, such as poverty reduction, education, and public health - through the promotion of transparency, civic involvement, and government accountability.
In Azerbaijan Open Society Institute –Assistance Foundation has launched a program of Transparency of Oil revenues and Public Finance in 2002. Through various mechanisms the program aims to generate and publicize research, information, and advocacy on how revenues are being invested and disbursed and how governments and extraction companies respond to civic demands for accountability.
In April, 2004 Open Society Institute –Assistance Foundation and BP Exploration (Caspian Sea) Limited (BP) signed a Memorandum of Understanding for supporting NGO monitoring and capacity building project. OSI-AF and BP acknowledged that construction of the Baku-Tbilisi-Ceyhan Pipeline (BTC) is one of the most significant economic developments in the Azerbaijan Republic. It was agreed that during construction, various monitoring programs shall assess issues related to the BTC including: social issues in communities near the BTC; land ownership issues; environmental protection; historical preservation; local business content and protection of the rights of workers involved in the construction of the BTC.
Subject of this report is evaluation of the monitoring project from the perspective of OSI-AF acted as a principle donor, facilitator and coordinator of NGO monitoring groups. The evaluation will go through identification of strength and weaknesses of each project phases. It will also provide recommendations for second stage of NGO monitoring in 2005 to respond to identify weak points, to improve the process and to further promote capacity of local NGOs.
Memorandum of Understanding
Key elements of the Memorandum of Understanding signed by Open Society Institute –Assistance Foundation and BP Exploration (Caspian Sea) Limited (BP) include:
Goals and objectives
-Development of experience and proficiency of local NGO’s for conducting effective BTC pipeline monitoring
-Bring successful international standards and practices of monitoring to Azerbaijan
-Improvement of local NGO’s monitoring skills
-Continued improvement to BP activities in Azerbaijan via feedback from NGO monitoring process
Principles of the Project
-Integrated process involving both formal training and “learning by doing”
-Transparency of the process
-Process will reflect international best practice
-Number of monitoring group members not more than – 30
BP duties
-Support and assistance to participating NGO’s in the form of technical and organizational support (health, safety, environmental training, transportation)
-Discuss monitoring issues with working groups and answer relating to it questions
-When an information request is denied, provide a written explanation stating the reasons for the denial
-Provide an information on methods being employed, current construction activities and upcoming work
OSI-AF duties
-Hold a meeting for selection monitoring group members
-Act as the facilitator and coordinator of the NGO monitoring process
-Assist monitoring group members during the monitoring and reporting phases
-Assist the NGO’s to work as teams rather than individually
-Financial assistance to monitoring group activities (trips, salaries, etc. in the frame of projects)
Monitoring group duties
-To work as a team
-To select coordinator and evaluator for a each working group
-To conduct objective and dispassionate monitoring
-Design the work plan together
-Produce high-standard reports
-Assess their own performance so that they learn from the process
Strengths:
-A first for Azerbaijan and the industry
-Mechanism to understand Azerbaijan NGOs and their capacity.
-NGOs have increased awareness
-New practices for OSI to extend to new initiatives
-New practices for NGOs to extend to new initiatives
-Enhanced BP / OSI relationship
Weaknesses:
-MOU deficient in some areas, for example, confidentiality, data security, archiving, use of photography, audio and video recording.
-MOU did not provide for the fact that a greater than anticipated amount of training and mentoring would be needed
PROCESS
NGO selection
OSI-AF initiated the process with announcement broadly circulated in national mass
media. The Open Society Institute-Assistance Foundation invited National NGOs with
relevant experience to participate in monitoring of the BTC oil pipeline. Monitoring by NGOs was set to focus on the following areas:
1.Environment;
2.Social problems;
3.Human rights (particularly, labour and land rights);
4.Conservation of historical monuments;
5.Use of local resources.
Eighty six national non-governmental organizations have applied to OSI-AF to participate in the monitoring process. Since there were many NGOs interested in monitoring the BTC pipeline project, OSI-AF in agreement with BP wanted the process to be as inclusive as possible, ie not limited to only a very few NGOs. To ensure more ownership and capacity building for a larger number of NGOs, OSI-AF introduced greater self-regulation into the selection process by providing the following:
-More NGOs had involvement in the process
-The NGOs had greater decision making powers and accountability
-Capacity building was made available to a greater number of NGOs
-The selection process was transparent and the problem of selection of NGOs by outsiders was removed
-Teamwork between NGOs was promoted and maximised.
Five NGO working groups were created, one for each of the five areas listed above. Each working group consisted of four to seven members each, with one representative from each national NGO that was selected to participate. Thus, twenty-seven individuals were identified as local NGO representatives set forward to undertake a BTC pipeline monitoring. Having identified these individuals, the NGO meetings also elected an Evaluator for each Working group. OSI-AF facilitated meetings for Working Groups, hence members for each Working Group voted for a Coordinator.
Strengths:
-A transparent process led by an organisation with a commitment to civil society and capacity building.
-A participative and democratic process.
Weaknesses:
-No screening or assessment process to eliminate unsuitable participants at the start or during the process
-Not all participants best fitted to a review, monitoring, and audit project
Capacity Building
Many of the NGOs in Azerbaijan have limited required skills and experience in monitoring (methodology, planning, data collection, interview techniques, data analysis, report writing, presentations etc.). Capacity building was therefore essential to ensure that monitoring activities and reporting are efficient and produce meaningful outputs, which are useful to all parties. Thus, a strong capacity building element was identified as a priority for successful development of the project.
OSI-AF with financial support of BP has brought into a process Catholic Relief Services’ experience in implementing the “Independent Monitoring Project for the Chad Cameroon Oil Pipeline Project” conducted from March 2001 to February 2004. The Chad Cameroon project was the first of its kind in Africa, in that it provided a number of mechanisms for community inclusion in the monitoring and evaluation of construction activities by the people who actually lived and worked alongside the pipeline, and whose lives were directly affected by the changes that occurred during construction. Catholic Relief Services used its institutional expertise delivered a four-day monitoring training, for representatives of 27 local NGOs. The trainings covered the following topics:
- General Monitoring and Evaluation Methodology and Techniques
- Data Collection Methodology and Techniques
- Data Analysis and Evaluation (Quantitative and Qualitative)
- Teamwork, Planning, and Proposal Writing
- Project Management
- Project Evaluation and Assessment
- Public Relations Methodologies
One of the most important elements that CRS brought to the capacity building process was the inclusion of Mr. Oliver Mokom, Chad Cameroon Pipeline Monitoring Project Manager. Local NGO representatives were provided practical overview of the process and developed a better sense of processes scale.
A need for close mentoring of the workgroup activities and constant training and capacity building exercises were outstandingly fulfilled by contracting Dr. Clive I. Morgan of Ridgeway Environmental Management (based in United Kingdom), this was also possible with BP funding. Dr. Morgan was with workgroups all throughout the monitoring process. As the project progressed new areas of need for capacity building were revealed, such as monitoring techniques, report writing expertise, communication skills etc. A sound capacity building element provided by Dr. Morgan has been one of the key factors for successful development of the project in 2004 and it consisted of the following:
- An independent perspective on BTC monitoring and audit practice across the pipeline project
- Mentoring and technical support relevant to monitoring and auditing at the home base of individual project groups in the field
- Support during NGO monitoring report writing,
- Interact with NGO’s through email, to prepare feedback report on NGO’s performance and individual’s performance,
- Contribute to the development of a communication strategy for the output of the project
Thus, local NGO involved in the process have gained considerable monitoring capacity as a result of the project in 2004. Success in capacity building element of the project has lead to the emergence of an advance group of civil society representatives. Newly acquired skills of these individuals shall be further refined and strengthened for the ultimate goal of establishing a pool of local experts that could effectively serve as mentors and trainers for local civil society development in future.
Strengths:
A significant training component
Weaknesses:
Not all essential training needs addressed
Proposal Development
Each Working Group produced a proposal regarding their theme. Proposals were developed within certain boundaries related to monitoring (number of trips to construction site, budget, etc) agreed between OSIAF and BP/BTC and provided to NGOs after the formation of the Working Groups. All proposals included the following items:
-Roles and responsibilities of each member
-Geographical coverage of the monitoring
-Description of the methodology of conducting the monitoring
-Schedule of site visit/meetings/events with BP/BTC
-Budget
OSI-AF received all the proposals and reviewed them, to ensure that they are within the financial and practical parameters. After assessing the proposals from each Working Group, and making necessary suggestions for amendments, OSI-AF awarded grants to the organization of a lead member/Coordinator in each Working Group.
Strengths:
-NGOs learned a lot through having to correct and improve their proposals
-Group effort team-work was involved
Weaknesses:
-Variability of group size (smaller groups worked best)
-Project proposal review process was not rigorous enough
-Several projects with too wide a scope
-Too many experts on the fringes of projects
-Project objectives often poorly matched to methodology and action plans
-Most projects required an audit element but no audit training given.
Development and Implementation of Action Plans
Each Workgroup reviewed with Dr. Morgan (Mentor) their proposals to ensure that objectives were consistent with methodologies and action plans set out in the proposals. This analysis revealed some inconsistencies in the original proposals.
Workgroups refined their proposals in the light of this analysis to make them more coherent. At the end of this phase of the process workgroups with help of the Mentor have achieved the following:
-Better definition of the objectives
-Better definition of scope of work to be done
-More realistic approach to field visits
-More detailed action plans
-More realistic view of work to be done within the timeframes of the project
-Clear definition of roles and responsibilities for Working group members
-Identified robust and reproducible methodologies, that were statistically valid
Strengths:
-Use of the Project’s Mentor
-Opportunity to refine the action plans before actual commencement of the project
-Iterative approach aided process learning by Working group members
Weaknesses:
-This phase of the process should have been done prior to proposals submission at the early stages of the proposals development
-Some Working groups were forced to cut back the proposed scope of work to make the it more achievable
-Setback in project implementation by two weeks
Monitoring Realization
Monitoring activities before report writing were scheduled to take place between October 1st and December 10th, 2004. In reality, some groups found it necessary to continue filed work into mid-December and other groups extended data collation and analysis into early February 2005.
Monitoring process involved following components:
-Document request
-Document review and analysis
-BP/BTC arranged site visits to CCIC a Spie Petrofac construction contractors
-Working group arranged site visits to the communities along the pipeline
-Site visits involved making observations, conducting interviews, carrying out surveys, and making video, audio and photo records
-Data collation and analysis completed by Working groups coordinators
-Accumulated data on paper and electronic format was consolidated and prepared for archive storage at OSI-AF
Strengths:
-Opportunity of acquiring more information about BP/BTC/Contractor business procedures and processes
-Significant field based component to all projects
-Opportunity to observe actual construction facilities
-Sufficient support from BP/BTC for site visits
-Independent work by Working groups in the 75% of communities along the BTC pipeline
-Enhanced ability of NGOs to conduct monitoring of complex and controversial projects
Weaknesses:
-Key documentation took longer to source than promised
-Some Working groups requested large number of documents but failed to adequately review them
-Lack preparation for visits to construction sites for some of the Working groups
-Presence of pre-conceived ideas about BTC related operations
-Lack of proper environment necessary for objective monitoring during some site visits
-Limited number of visits to BTC sites
-Limited opportunity for individual or small group visits to BTC sites
-Poor coordination of site visit plans between different Working groups
-Lack of mentoring during site visits
-Lack of sufficient level of monitoring for some of the Working group members
-Significant number of case when issues and allegations were accepted without thorough investigations
-Failure to meet original project deadlines
Reports
Original process timetable reserved a month for completion of report writing phase of the project, including reports review by OSI-AF. This was to be followed by translations of the reports into English and BP/BTC internal process review of the reports. As stated in the MOU, BP/BTC was to be allowed twenty days for internal review of the reports. By agreement with OSI-AF this was extended to thirty days. This revised timetable assumed public disclosure of the reports in the first week of February 2005. In reality, it proved impossible to meet this revised timetable. Some groups were inefficient to pull information together from different elements of their project. One group had too much data collected in non-systematic way, and another group had serious divisions internally about final content and format of the report, which lead to delays.
Strengths:
-Current process and program have generated five reports of a reasonable quality
-The report writing phase clearly demonstrated a need for soundly written proposals, detailed action plans, correct choice of methodology, well coordinated Working groups, in order to ensure that sound results are generated and reported on to sensible timetables
-Tremendously useful learning experience for NGOs, which will have enhanced their capacity on behalf of the civil society
Weaknesses:
-Underestimation of time required for report writing phase of the project
-Overestimation of potential of some working group members to produce a reasonably good copy of a report in agreed time frames
-Lack of fully dedicated OSI-AF resource to drive the reporting process to the original timetable
-Mechanism of internal review within OSI-AF was not sufficient for the scale and the diversity of the project
-Difficulty to achieve consensus within some Working groups in order to produce a balanced and objective report
Outcomes
As of today, May 24th, 2005, all five reports have been submitted, reviewed and responded by BP/BTC. As the NGO Monitoring Programme drawed to an “Outcome” phase, aims were:
-BP/BTC provides a respectful, factual responses to the issues raised by those commenting on BP/BTC activities,
-Those NGO report findings, suggestions and lessons that suggest ways in which BP/BTC performance can be improved are identified. As a result BP committed to exploring means of operationalizing aspects of these key findings and recommendations,