Revised:
24 August 2017 / Reg. no. V 2016/378
Published: medarbetarportalen.gu.se/styrdokument/
Guidance for external review of study courses and programmes at first and second cycle
This guide offers support for faculties and departments using external review of the study courses and programmes which is one of the processes that is described in the university Policy for Quality Assurance and Continuous Quality Improvement of Education. For increased clarity, this document also includes excerpts from the policy. These are underlined and set out compulsory parts of the reviews. Several faculties have chosen to develop a faculty-oriented guidance.
Review object
In line with the goals stated in national statutes, local objectives and Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG, see pages 11 to 16), the faculty boards are to initiate review of all study courses and programmes over a six-year period. The faculty boards can select other national and international contexts (accreditation, “field networks” or similar) for external assessment. Faculty-initiated external reviewof study courses and programmes is not necessary for those study courses and programmes that are being evaluated by the Swedish Higher Education Authority (UKÄ) in the period in question. For the period 2017-2022, education is excluded from self-initiated review regarding teacher training, nursing education, specialist nurse education and medical education.
After consultation with the responsible department, each faculty board decides which courses, programmes, degrees and/or main fields of study are to be included in each single evaluation[1].
Student participation
Students are to be offered the possibility to take part in the planning and monitoring of review. If possible, those students whose study courses and programmes are being reviewed shall be given the opportunity to take part.See also section 21 under the heading "Input documents".
As regards student participation in the external panel, see below.
Timetable
Recruitement of the panel may be time-consuming and should begin well in advance of the actual review. Already during the recruitment process, a preliminary timetable may be communicated with the potential panel, such as suggested meeting days and other deadlines such as deadline for preliminary panel statement. The department´s work with collecting and analysingthe input documents, can be done in parallel with the recruitment process.
When the date of the site visit is decided the relevant staff and students should be notified.
The input documents should be communicated to the panel at least one month before the panel´s preparatory meeting. The time between the preparatory meeting and site visit can usually be relatively short. After the site visit the panel should have allocatedtime for writing and reading draft versions, so that the statement can be completed as soon as possible after the site visit.
An example of an overall timetable is given below.
Preparations
Establishment of preliminary timetable for current reviews
Recruitment of panel of assessors including chair
Collecting and analysing of input documents
Month 0
Panel established
Input documents established and sent to panel
Month 1
Panel of assessors holds a preparatory meeting
Month 2
Site visit carried out
Month 3-4
Preliminary panel statement sent out by panel
Comments on the preliminary statement sent to panel
The panel finalises its statement and the faculty publishes it
Month 5
Feedback meeting held
Month 5-7
Faculty’s action plan finalised and published.
Faculty-external panel of assessors
The overall task of the panel is to examine the academic and pedagogic quality of the provided education and its relevance to students and society. The peer review format and the panels´ recommendations will contribute to the long-term quality improvement and quality assurance of the University’s study courses and programmes. The various members of the panel contribute with their different perspectives to the work.
Each review is to be carried out by scientific/artistic and pedagogic experts and at least one student representative. The scientific/artistic experts are to be in the majority and at least two of them are to come from higher education institutions other than the University of Gothenburg. The panel is to be established by each faculty board after consultation with the department responsible for the study courses and programmes in question. Representatives of other relevant fields of expertise may also be included in the panel.[2]
The scientific/artistic experts may be complemented by an expert from another faculty at the University of Gothenburg. Such an expert can contribute overall knowledge of the University and inspire innovative thinking through good examples from field-unrelated educational activities at the University. University-internal participation in the panel would also stimulate and contribute to a general exchange of findings within the University.
A chair should be from amongst the scientific/artistic experts.There may be advantages with the faculty already during the recruitment of the panel submitting a proposal of whom to be the panel chair.
Within the framework of these principles, the composition of the panel is open. It could, for example, comprise the following members:
- Two scientific/artistic and pedagogicalexperts from another institution
- ) One scientific/artistic and pedagogical expert from another faculty at the University of Gothenburg.
- One or two students (from another department at the University of Gothenburg, and/or from another institution, with experience of studies in similar courses and programmes.
- A “working-life representative” or a representative with other relevant competence.
Equal gender distribution in the group should be taken into account.
Input documents
Producing input documents is an important part of the continuous quality improvement of study courses and programmes. What discussions are stimulated by the content of the input? What possible areas for improvement emerge that can be immediately discussed further (without awaiting the external assessment)?
The purpose of “Comments and reading instructions” (section 20) is to facilitate for the panel to form an opinion and to get a good overview of the input documents. For example, the reading instruction can comment on what the panel need to draw attention to in the input documents.
The students' comments (section 21) are a form of student influence and student participation that makes it easier for the panel to review the education also from a student perspective. Such comments may provide support to the panel regarding issues that should be addressed during the site visit.
What is to constitute the input for the panel is, after consultation with the department and the panel, to be decided by each faculty board. The panel should have the opportunity to express any particular wishes as regards input. The input is to set out how, within study courses and programmes, quality is assured and developed in student learning and the results of education.
Examples of possible input documents are:
STEERING DOCUMENTS, MINUTES, REPORTS, ETC.
- Programme and course syllabuses (course literature lists included therein).
- Examination assessment criteria.
- For educational activities relevant steering documents, procedures and memos.
- Documents describing continuous monitoring and development activities.
- Action and operational plans and any documents monitoring these.
- Minutes/memos from programme councils, working-life reference groups, etc.
EDUCATION-RELATED DOCUMENTS
- Course guide/equivalent.
- Selections from timetables and documentation describing teaching forms and teacher staffing.
- Random selection of examination papers and, if possible, student accomplishments with assessments.
- Random selection of independent project (with assessments).
EVALUATION AND MONITORING RESULTS (MEASURES INCLUDED THEREIN)
- Course and programme reports, compilations of course evaluations, etc.
- Compilations of other questionnaires (student, studies while on placements, etc.).
- Plans of measures drawn up in response to questionnaire results.
- Other evaluation results (e.g. UKÄ 2011 to 2014 – self-evaluation reports and UKÄ´s assessment and any planned measures alike).
MISCELLANEOUS
- Teacher list with form of employment, academic qualifications, pedagogical training, scope of teaching and research activity (see e.g. UKÄ´s teacher list template for the 2011 – 2014 evaluation).
- Relevant statistical input.
- Links to relevant websites (both for information and for assessing the quality of the information).
- The views of working-life representatives and alumni on the provided education.
- Documents/results concerning the inventory of the quality of study courses and programmes 2016/17
- Comments and reading instructions to the input documents
- Students’ comments on the input documents and/or the programme.
Review criteria
The panel is to review whether the provided education meets the following criteria[3]:
-Achieved study results matching intended learning outcomes and the qualitative targets of the Higher Education Ordinance.
-Teaching being focused on student acquisition of knowledge/skills.
-The content and form of teaching resting on good scientific and/or artistic bases and proven experience.
-Teachers having up-to-date and adequate competence as regards their subjects, higher education pedagogics and subject didactics, and that said teacher being in proportion to the scope and content of study courses and programmes.
-Study courses and programmes being relevant to the needs of the students/doctoral students and society.
-Students/doctoral students having influence in planning, implementing and monitoring study courses and programmes.
-There being a study and learning environment that is accessible and purpose-oriented for all students/doctoral students.
-There being continuous monitoring and development of study courses and programmes.
The panel should also report any perceived weaknesses that can not be clearly referred to any of the criteria. The faculty/department may also communicate with the panel if any of the criteria, parts of them or other criteria and aspects are of particular interest.
Preparatory meeting
During the preparatory meeting (physical or virtual) – which should be conducted when the panel has read the input documents -the faculty/department can provide the panel with background information and answer any questions. The panel should also during the meeting plan the site visit;what groups the panel wants to meet and what questions should be asked to supplement the input documents.
A purpose with the preparatory meeting may be that the panel submits a proposal to the faculty/department on the schedule for the site visit and any wishes regarding supplementary documents as well as preparing preliminary questions for the various sessions of the site visit.[4]
The panel´s interviews/site visit
Once the panel has analysed the input, it is to carry out physical or virtual interviews with course/programme representatives and students.Representatives of collaborating parties outside the University may also be included as interviewees. Before the site visit comes to an end, a verbal summary of preliminary observations can advantageously be communicated to faculty and department managements.
Interviews are to be planned in accordance with the wishes of the panel.
Below is one example of a possible programme for a physical site visit.
Day 1
13.00-13.45The panel carries out an internal meeting
13.45-14.45Students
15.00-16.00Teacher representatives
16.15-17.15Department’s/programme’s managementand for example study guidance
The panel allocates time during the evening to summarise impressions
Dag 2
8.30-9.15External stakeholders and/or alumni
9.20-10.00Any further groups
10.00-10.30Panel preparesfeedback to faculty management and department management/equivalent
10.30-11.00Feedback to faculty management and department management/equivalent
11.00-15.00Panel workswith review statement
Statement of the panel
A simple template for the review statement is available on the web (Swedish only).
In a statement, the panel is to summarise what needs to be developed for the course/programme to satisfy the above-mentioned criteria. The panel can also draw attention to good points in the provided education.With the aim of promoting future development initiatives, the panel can also advantageously put forward, other higher education institutions/study courses and programmes for benchmarking purposes.
The faculty/department is tentatively given the opportunity to comment on a draft version of the statement and submit any clarification before it is determined.
Feedback meeting with the panel
Once the panel statement has been published and disseminated, the panel (or parts of it) may hold a physical or virtual feedback meeting with faculty, department and student representatives. The aim is to give a possibility to ask questions and discuss the contents of the panel statement.
Measures
It is suggested that the department draws up a plan of measures in consultation with the faculty board. The latter ratifies said plan.Each faculty board is responsible for measures being implemented on the basis of the reviews. This includes, where relevant, phasing out courses and programmes.
Publication, monitoring and development
The panel statement and planned measures are to be published and brought together ona purpose-specific website hosted by the University of Gothenburg. The faculties/departments can also benefit from developing their own web pages for this purpose. The faculties follow up on the implementation of the action plans according to faculty-adapted procedures.
In operations dialogues with the faculties, the vice-chancellor is to annually monitor the external reviews. Simultaneously, there is to be monitoring of measures that have been implemented following previous evaluations. Prior to the dialogues, the faculties are to give a summary report of reviews that have been carried out and ongoing development activities.
1 (7)
[1]See Fakulteternaspreliminärasexårsplaner (in Swedish only).
[2] Examples of other relevant fields of expertise include: interdisciplinarity; pedagogic development; continuous quality improvement; studies while on placements; internationalisation; sustainable development; broader recruitment; gender; and, equality.
[3]The criteria were drawn up by SUHF (The Association of Swedish Higher Education) in 2014 for this type of purpose, see page 5, SUHF's version (Swedish only).
[4]Uppsala University has developed a battery of questions that relates to the corresponding criteria that may provide inspiration (in Swedish only)