AIF Self-Assessment Exercise

AIFOM: Annex 1

University Grants Commission of Bangladesh

Higher Education Quality Enhancement Project (HEQEP)

Academic Innovation Fund (AIF)

Concept Note on Self-Assessment

I. Background

Responding to massive increase in the numbers of secondary education graduates during the last decades and consequently, numbers of students eligible to enter university and aspiring for upward social mobility, a phenomenal growth of public and private universities has occurred. This resulted in a trend of enhanced allocation in state funding for higher education anda growing concern of government and public for the results of investment and accountability. The rapid move towards a globalized, intenselycompetitive knowledge based economy increasingly demands an effective education system capable of churning out high quality human capital who would be able to compete successfully in the global knowledge economy and labor markets. Under this scenario a genuine concern forthe improvement, evaluation and quality assurance of higher education sector has become imperative.

In practical terms this has produced an increased interest and use ofself-regulatory procedures for academic programs and institutional improvement. In order to respond to external evaluation and regulatory requirements such as accreditation and strategic planningfor overall management improvement and accountability of the higher education institutions the self-assessment (SA) exercise was initiated. The self-assessment procedure involve internal determination of quality by self-evaluation, external accreditation for the public certification of quality and information to stakeholders and users, and the planning and organization of change for academic improvement. A simplified graphical description is given in Diagram 1 below.

Self-assessment in an organizational setting refers to a comprehensive, systematic and regular review of an education institution’s activities and results. The SA process allows the organisation to discern clearly its strengths and areas in which improvements are required and culminates in planned development actions which are then monitored for progress and achievement.

It is well known that self-assessment in universities is a good start for academic involvement and participationin institutional strategic analysis (in a bottom-up approach) that may lead to increased academic quality and better planning. The creation of this institutional capacity is normally also a necessary condition for an effective response to external regulation requirements (such as accreditation) and, when well established, to start institutional strategic planning (in a top-down approach). A simplified process description is given in Table 1.

Self-assessment is about academia and its participation in analyzing their role and effectiveness in producing good teaching and student learning, relevant and productive research that can support the generation of advanced human capital and produce results that can generate new knowledge and help national innovation, competitiveness and development. For this purpose, it requires motivation and incentives.

The Academic Innovation Fund(AIF) will consider sub-project proposals for conducting a Self-Assessment exercise as a sub-project activity at the Department or or/and academic program levelsi.e., at the undergraduate, graduate/master’s and Ph Dprograms in universities.

At the end of the self-assessment exercise, the Department or/and academic programs and the university will be in possession of an updated strategic analysis, a set of suggested improvement measures and actions and a preliminary improvement plan, that will be ready forinitiating actions for change or presentation in the next round of the AIF or to UGC for funding.

II. Self-Assessment Operative Framework

It is convenient for SA implementation to establish from the beginning the basic “rules of the game”:

1. Process purposes. Motivation and primary objectives will be key elements that will guide a SA process. So, they should be communicated clearly to the different academic stakeholders (academics, students, officials, non-academics).

2. Voluntary participation. SA should be a voluntary process to be decided at the bottom of the institution’s academic programs or Departmental levels. Enforcing this process can only produce negative reaction and resistanceand limit motivation and open-mindedness. However, motivation and induction activities to academic sectors that are felt to be interested in improving their overall status can be very helpful. The main task is to give participating actors an appropriate sense of process “property”. It is known that well developed academic units and programs have a better disposition to self-analysis than others that cannot show such conditions.

3. Experimental or pilot character. SA processes have certain cultural connotations both at the country’s higher education and at university levels. So, it is not wise tototal transfer of external SA designs, guidelines and experiences for undertaking the exercise.It is better to find references and to make adaptations and, above all, to learn from own experience. It is well known that learning occurs essentially with action. Therefore, in time, SA experience will build up and with the necessary adaptations it will reach academic quality improvements. On the other hand, an experimental character will give SA processes gradual strength which is important when no previous experience is at hand.

4. External validation. The integration of an SA process to an external validation or auditing contributes to strengthen its validity. Therefore it is important from the beginning to involverenowned external experts and peers and foreign academics and researchers that will add new perspectives to the exercise.

5. Effects or consequences. When implementing SA, it is important to ensure some changes in the field of institutional and academic management. Some can be achieved during and after the process without new resources; others will require additional ones, such as the AIF or resources from the government. The important thing is to realize that the results of an SA exercise provides an updated diagnostic that would helpin acquiring additional resources and improvements.

6. Cycles. SA has, in principle, to be integrated to the institutional “culture” and therefore to be operated in cycles. The cycle should consider the self-assessment phase (self-study and external validation) and the design and implementation of an improvement plan (that can be internal or respond to external regulation, as accreditation).

7. Training/Facilitation. As there won’t be sufficient experience to address and start SA processes, training for university SA facilitators will be organized. Foreign consultants might be considered in the initial phase of SA awareness, motivation, process design and facilitation, and implementation.

III. Suggested Self-Assessment Process Schedule

The following self-assessment process schedule is suggested for proposal design and preparation and (if selected and allocated) for sub-project implementation:

Preparation Phase

A1. Implementation of an Institutional Self-Assessment Unit

Each university that receives allocations for program and/or departmental self-assessment exercise, should organize a small institutional SA Unit (SAU) that can manage ongoing SA processes. This unit will be trained by the consultant or visiting expert for SA process design, facilitation and overall management. It should be headed by an experienced academic staff member.

A2. Appointment of the Self-Assessment Committee

Each university should organize its self-assessment processes according to its administrative culture and practices. However, it is advisable that a small SA Committee is designated by the participating actors to be responsible for the process implementation; a highly motivated staff member with experience in evaluation and/or planning matters should chair it. Committee members should be in general well accepted by all participating actors and, academic members in particular, have outstanding qualifications and ascendance within their academic community.

A3. Training and preparation of the SA Committee

All SA Committee members should receive technical information on self-assessment and university’s quality improvement processes, in particular, on the institutional SA process that they will have to implement and manage. All this information will prepare them for the different process steps that will have to be organized and developed.

A4. Resources necessary for SA implementation

Human resources. Definition of time allocation for academic and non-academic staff participating in the SA Committee
Support staff. Secretarial, data retrieval and processing, surveyors
Material resources. Meeting rooms, computers, e-mail, photocopying, communications, office materials

A5. Preparation of the SA process

Limits of the self-assessment
Objectives
Areas of intervention
Quality criteria and performance indicators
Selection of the population and/or sample
Preparation of information gathering instruments
Steps and chronogram
Expected difficulties and consequences; remedial actions

A6. Selection and designation of external peer visitors

Potential external peer visitors should be prospected and contacted at the beginning of the SA process. Accidental “last minute” peers should be avoided.

A7. Information and motivation phase

Meetings with all relevant stakeholders that relate to the program and/or departmental self-assessment process. The SA plan should be explained clearly and completely and suggestions for plan improvements be considered. Meetings should be planned for university officials, academics, undergraduate and post-graduate students, non academic staff and workers (if necessary).
The SA plan should be communicated through appropriate written and/or electronic means and a specific Internet web page be designed and provided.

B. Implementation Phase

B1. Information gathering

Work intensive step. Crucial in SA, but requires a lot of work and time.

B2. External peer visit planning

As external visitors are normally very busy, agreement on their participation in the SA external validation visit has to be planned and agreed upon well in advance. The visit should consider at least three full days of site work.

B3. Organization and tabulation of the information

In this phase, the SA Committee should analyze and supervise the organization and tabulation of the information that has been gathered by different means and media and that is considered relevant and useful for an integrated SA analysis. To this purpose, clear and precise criteria have to be established for the:

Tabulation of surveys
Organization of quantitative data
Results from structured discussions
Selection and prioritization of all collected documental material.

B4. Analysis of results

The SA Committee should perform the first analysis of the results, considering the mission and objectives of the program, department, center or institute and of the SA exercise. Usually, a SWOT (Strength, Weaknesses, Opportunities and Threats) analysis is recommended
Validation, analysis of results and exploration of recommendations are sought from the unit in process
Validation, analysis of results and exploration of recommendations are sought from the participating academic community.

B5. Preparation of the Internal SA Report

The SA Committee should be responsible for the preparation of the Internal SA Report. It’s typical contents are:

Presentation and introduction. What are the objectives of this SA process?
The self-assessment process. What is to be evaluated?
Description of the program and/or academic unit in an evaluation context. Who will do the SA? In what period of time?
Program strategic (SWOT) analysis. How will it be done? What instruments will be used?
Evaluation findings. Are we sincere while communicating our findings?
Orientations for change management. What is our commitment to change?
Annexes. Are all findings (qualitative and quantitative) well supported by data?

B6. Presentation of the Internal SA Report to the university authorities

When finished, the Internal SA Report should be presented to the program or unit academic officials and then to the university officials for their information and sensibilisation and eventual decision making.

B7. Socialization of SA results

The SA Report should be shared and discussed with the university and academic community; this will avoid mistakes and deviations and motivate commitments for change management.

B8. Delivery of the Internal SA Report to the external peers

The Internal SA Report should be sent to the institutional SA Unit and external peers at least one month before their site visit. Ample time should be given for reading and analysis.

External Validation Phase

C1. Planning of the external peer site visit

Preparation of the agenda and site visit by the institutional SA Unit and SA Committee
Discussion and mutual agreement of the agenda with the external peers
Successful implementation of the agenda through good planning, time management and motivation and information of all university actors that are supposed to participate in the visit in meetings or interviews.

C2. External peer visit

The length of the visit should be agreed with the external peers; however, a minimum of three working days is recommended. A leader should be designated. Appropriate terms of reference should be written for their work and included in their contract. A typical visit can include the following (indicative) program:
Mission coordination meeting before the start of the visit
Day 1:
Meeting with the Vice Chancellor, Treasurer and Registrar
Meeting with the Deans, Department Chairperson/SA program Director
Meeting with the Institutional SA Unit and program SA Committee
Meeting with students
Working lunch. Meet other participants
Meeting with professors and/or researchers
Visit to program facilities, library, laboratories, study rooms, administrative offices, cafeteria
Day 2:
Mission coordination
Meeting with professors from the general education cycle
Reading SA documents
Working lunch. Meet other participants
Meeting with program graduates
Meeting with external stakeholders
Day 3:
Mission coordination. Preparation of preliminary oral report
Final meeting with theDepartment Chairman, Dean, Faculty staff and professors and/or researchers. External peer mission delivers a brief oral report with the preliminary findings of the SA process
Final meeting with the Vice Chancellor, Treasurer and Registrar
Lunch with program academics
The institutional SA Unit and SA Committee should ensure compliance of the agenda and avoid improvisations and the inclusion of activities that have not been approved or programmed
The SA Committee should have organized all available information, survey materials and statistics that may be requested by the external peers during their visit
Suggested entrance and exit meetings should include the President of the University and collaborators
A final exit meeting should be programmed with all the SA participants; the leader of the external peers should make an oral report with the preliminary findings of the visit and of the Internal SA Report.

C3. External SA Validation Report

Generally, one month is given to external peer visitors to prepare and deliver the External Validation Report

D. Final Self-Assessment Report

The SA Committee should be in charge of preparing the Final SA Report. This report should integrate observations and recommendations included in the External Validation Report with the findings of the Internal SA Report.

E. Orientations for Change Management

Even when the self-assessment exercise ends with the Final SA Report, it is highly recommended that,starting from the reported results, some orientations for academic improvement, goals, conditions and commitments are also included. This text can be considered as a Preliminary Improvement Plan.

Diagram 1

Table 1

STEPS IN AN AUTO-REGULATION CYCLE

(Can be applied to programs, units and Institutions)

Self-Assessment Exercise

Beginning of cycle (t=0)
Preparatory design and activities
Initial motivation visits and process guidelines
Process development. It is participative! Interaction with stakeholders. User meetings, SWOT analysis, focus groups, surveys, and interviews.
Intermediate progress seminars. Internalization
Preparation of “Internal Self-Assessment Report”
External validation. Peer visits. “External Validation Report”
Climax. Program or Unit Diagnostic. “Final SA Report”
Final workshop. Internalization. Approval
End of self-assessment exercise (t=1 academic year).

Consequences and Quality Adjustments. Change Management

Prioritization of problems and issues. Decision making
Preparation of a “Program Improvement Action Plan” that is coherent with the self-assessment results, diagnostic and Final Report
Availability of resources. AIF and other sources (t=1 year)
Implementation of Action Plan. Quality adjustments. Change management
M&E. Accountability
End of quality adjustments (t=2-3 years)
End of cycle (t=4-5 years). New auto-regulation process starts. Virtuous improvement cycles.

SWOT = “Strength, Weaknesses, Opportunities and Threats”

AIFOM: Concept Note on Self-Assessment