A Report for the Higher EducationAcademy External Examiner Project

Authors: Colleen Connor & Dr Bill Davies

CONTENT

1. Introduction

2. Background Information

3. Methodology

4. Analysis & Discussion

5. Conclusion

6. References

Appendices

(i)Letters to University Registries

(ii)Covering letters to External Examiners

(iii)Questionnaire

  1. Introduction

1.1The purpose of the following report is to provide a description of a study of external examiners working in non-academic settings (i.e. not employed in a higher education institution) and their experience of induction & support for the external examiner role.

1.2The study was conducted following a workshop led by Professor Howard Colley at the University of Wales Institute, Cardiff (UWIC) in February 2005. One issue arising at this workshop concerned the appointment of external examiners from outside the higher education context and whether they had any specific needs in terms of induction.

1.3The study thus sought to establish the number and nature of external examiners drawn from non higher education contexts within a small group of Welsh universities and to investigate their experience of induction.

1.4The following section explains the rationale for undertaking the study together with a short review of background literature that informed the investigation. The methodology of the study is then outlined (Section 3), followed by an analysis and discussion of the results (Section 4). Finally some conclusions are drawn together with suggestions for further investigation (Section 5).

2. Rationale and Background

2.1 The role of external examiners has been extensively articulated and discussed (Ashworth and Harvey, 1994; Ashworth, 2004 and Jackson 2004) and embodied in a code of practice (QAA, 2004).

2.2 While there is some variation in the tenor of this literature and some different interpretations of the role (Smith & Oliver, 2004) there seems to be broad agreement that the fundamental purpose of an external examiner rests on three functions, namely:

  • to act as an arbiter of standards and to confirm that the awards made to students are comparable in standards with awards in higher education generally.
  • to monitor, assist and advise on the enhancement of the quality of programmes;
  • to provide advice particularly on matters of student assessment, learning outcomes, content, and the inter-relationship of these elements

2.3 Reflecting on the fundamental contribution external examiners make to the quality assurance of courses a number of writers have argued the case for a coherent approach to the development of external examining as a professional progression within Higher Education. Cowan (2004) suggests that

‘The enhancement of professional development for external examining should be integrated with thestrategic development of infrastructures to support the wider professional development of HE teachers’ p.2-3

2.4 Similarly Jackson (2004) suggests that external examining be seen as one of a raft of peer review, consultancy and evaluation activities that constitute advanced professional practice in the field of teaching and learning.

2.5 The notion of locating external examining expertise within an academic or higher education context is echoed / replicated by analysis of the skills and attributes required to be effective as external examiners. Jackson (2004) reviews the discussions and researches of the Higher Education Academy external examiner working group and suggests that the observations and judgements that external examiners make, to be effective, have to be informed by contextual knowledge of being a higher education teacher for example

“to understand the technical cultural and procedural language of the discipline, the institution and the national frameworks for describing and evaluating standards” p2

2.6 In a later paper (working paper 7) Jackson reiterates the expressed view from within the sector that academic experience and understanding are central to the examiner role.

“locating external examining within the career profile of a teacher/ scholar is an important step in understanding the nature of the experiential learning and more formal professional development that underlies external examining” p2

2.7 It can be seen therefore that a consistent theme is the central relevance and inescapable importance of a facility and familiarity with the standards and processes that operate with academic institutions and that these attributes can only be gained through lengthy immersion in such institutions.

2.8 Whilst this is intuitively and empirically valid, the rise of diverse vocationally oriented programmes and the continued involvement of professional bodies in approving qualifying programmes have meant that institutions have to assure congruence of programmes with the practitioner and marketplace context and needs. This may become particularly important if modes of delivery become even more workplace located and oriented.

2.9 Ashworth (2004) offers a range of added value functions offered by external examiners who are practitioners rather than academics. These include the provision of a focused business perspective to vocational academic areas; an input ofcurrent technical and professional knowledge into the scrutiny of academic endeavor; a view on appropriate transferable skills and graduateness which complements the academic view; and the furnishing of an additional dimension to the programme enhancement. These inputs and perspectives are likely to be more highly emphasized where professional accreditation is a programme outcome.

2.10 Ashworth indicates that the professional bodies covering the built environment sector tend to have a high level of interest in the contribution made by external examiners to their approved programmes for example stipulating that at least one examiner per programme should be a member of the relevant professional body or requesting a combination of academic and practitioner examiners. Other professional disciplines such as social work and speech therapy have similar requirements and expectations

2.11 The QAA Code of Practice section on External Examining (August, 2004) also notes the need for those with practice experience to take the external examiner role.

The increasing diversity of programmes and modes of study within higher education suggests that institutions will, where appropriate, want to appoint external examiners with professional or practice-based expertise, in addition to those with more typical academic backgrounds.

2.12 The literature repeatedly emphases the need for academic pedigree and depth of experience of external examiners to enable them to discharge their responsibilities of maintaining standards across institutions and ensure quality within institutional frames of working. Therefore it is not surprising that there are significant concerns about the ability of examiners located in non academic contexts to fully fulfill the examiner role.

2.13 Smith and Oliver (2004) from an investigation of external examiners perceptions of the role indicate the centrality of the academic grounding required by citing one respondent

“all external examiners are primarily engaged in teaching. Neither managers nor quality officers are suitable” p4

2.14 It was however acknowledged by respondents in the survey that the situation may be slightly different for those working in a vocational discipline, where elements such as cutting edge knowledge of professional practice, and access to a network of external contacts and understanding of employment opportunities are of critical importance. Even so, there was a clear view that having experience of higher education was essential and that ‘non academic external examiners would operate in clearly circumscribed circumstances.’

2.15 Jackson’s (2004) investigation into the institutional perspective on supporting external examiners in general found a similar view to be held among academic staff. The workshop discussions with the SE England group raised questions about the issue of non academic externals, noting that the role of the external examiner requires a lot of knowledge about assessment, generic, subject specific and institutional. None of these areas are ones where a practitioner from a non academic context can be assumed to have the requisite expertise

2.16 It is not just academics and the institutions that recognize potential problems in the use of non academic examiners. Ashworth (2004) presents the difficulties perceived by some practitioners in fulfilling an external examiner role. In addition to those relating to assessment and understanding institutional systems, criteria and procedures for marking and grading practitioners feel thatthe limited experience they may have across institutions makes comparison of standards difficult. They also profess to have problems in resisting stereotyping of academics as lacking real world experience; they have difficulties in understanding the pressures felt by academics within their own community; they may perceive academics as too lenient with failing students , or alternatively be unable to accept the reluctance of academics to use the full range of marks or grades. Finally there may be incongruence between to concepts of graduate competence and professional competence that are not easy to reconcile.

2.17 So, a number of forces can be identified; the imperative to maintain the effective monitoring and development of quality and standards; the need to embed external examining within a career structure that ensures individual professional development based on experience within the academic context; and the increasing need to draw upon the services of external examiners from non academic backgrounds to permit continued validation and appropriate alignment of vocational courses with employment needs and the various professional worlds of students emerging from vocational courses.

2.18 It is evident through a number of consultations and discussions that have taken place and been widely reported (Jackson, 2004; Colley, 2005) that while the establishment of support and induction systems for all examiners is under way there is still considerable room for the development of an agreed and consistent framework. This is not surprising given the diversity of needs, contexts and environments in which external examiners are required to operate.

2.19 But if the institutional responsibility highlighted by Cooke (2003) and neatly paraphrased by Jackson in LTSN working paper 2 that…

“HEI’s… have the responsibility to prepare their external examiners for their work in their institution. External examiners have to master the peculiarities of the institutional environment in which they are examining in order to fulfill their role”

is to be accepted and operationalised there is one specific area demanding of attention.

‘An emergent issue is how such frameworks might also support non-academic external examiners drawn from business, industry and the professional world.’

Cowan (2004, p.3)

2.20 The support, integration into the institutional context and efficient utilization of examiners from non academic backgrounds remains an area for investigation and development. Given the potential for dissonance and the greater potential for an effective contribution to the development of programmes aligned with academic and professional demands it is clear that a greater understanding of the training and support needs of non academic examiners must be achieved.

3. Methodology

3.1 A multi- stage research design was followed consisting of an exploratory stage to establish viable sample parameters and a questionnaire survey stage to obtain preliminary responses from the population of relevant examiners. The intention was to complete a follow up stage comprising telephone interviews with a selection of survey respondents to further explore issues raised in the survey. This final stage has not yet been completed due to external factors that could not be foreseen.

3.2Stage – (i) Identification of population and sample parameters.

In order to obtain a good spread whilst also maintaining manageability it was planned to draw the sample of external examiners from Welsh universities that attended the HE Academy network meeting in UWIC on 03/02/05. These were: North East Wales Institute; CardiffUniversity; AberystwythUniversity; SwanseaUniversity; University of Wales, Newport and University of Wales Institute, Cardiff

3.3 The advantage of this were seen to be

  • Both pre and post 92 universities are represented
  • A wide range of disciplines are covered
  • The participants at the network meeting would provide a contact point and they identified the issue as important.
  • It represent a discrete geographic area
  • It would raise the profile of the HE Academy in Wales

3.4 It was difficult to predict the number of ‘non-academic’ external examiners appointed to these universities. A snapshot of UWIC external examiners showed a total of 23 from non-academic settings. This was made up as follows:

School of Applied Sciences-4

School of Art & Design-2

BusinessSchool-1

School of Education-3

School of Health & Social Sciences-4

School of Lifelong Learning-1

School of Product & Engineering Design1

School of Sport1

School of Hospitality, Tourism & Leisure Management - 3

3.5Stage (ii) Institutional survey

This suggested that the numbers that could be drawn from other institutions would provide a sustainable and realistic sample for worthwhile conclusions to be drawn. However it was felt important to actually ascertain the number of externals from non-academic settings and the programme areas they serve as appointed by the different universities in order to be confident in the appropriateness of the sample spread to be used and to permit a representative sampling grid.

3.6 It was also necessary to work closely with the Registries of the collaborating institutions as the release of examiner data was problematic under the application of the Data Protection Act

3.7 The six potential collaborating institutions were contacted and asked to return a brief proforma containing information on current examiners. This included an indication of those “not working in a higher education context” or working in “non academic settings”.

3.8 The following table provides the distribution of “Academic” and “Non Academic” external examiners received from Registries of participating universities. (NB responses are provided according to whether the university is pre 92 or post 92 in order to retain anonymity)

Table 1 Distribution of “Academic” and “Non Academic” external examiners

Externals from academic settings / Externals from non academic settings
Undergrad / Post grad / Undergrad / Post grad
Pre 92 (A) / 107 / 127 / 5 / 13
Pre 92 (B) / 60 / 41 / 13 / 4
Pre 92 (C) / 135 / 85 / 5 / 3
Post 92 (D) / 90 / 62 / 1 / 2
Post 92 (E) / 86 / 18 / 17 / 2

Post 92

/ No response to request
TOTAL / 278 / 333 / 41 / 24

3.9 Thus responses were received from two Post 92 universities and three Pre 92 universities. The total number of external examiners from non academic settings in the three Pre 92 universities numbered 43 (23 serving undergraduate courses and 20 serving postgraduate courses). The total number of external examiners from non academic settings in the two Post 92 universities numbered 21 (18 serving undergraduate courses and 4 serving postgraduate courses).

3.10 Development and pilot of questionnaire

A questionnaire was devised based on the key issues identified in the literature (Jackson 2005, Colley 2005) covering the themes of:

Occupational and academic background;

Examining experience;

Current examining role;

Experience of induction and support;

Perceived needs in relation to induction and support.

Respondents were also asked to indicate their willingness to participate in a follow up telephone interview.

3.11 Administration of survey

All institutions agreed to distribute questionnaires and covering letters on behalf of the researchers. Questionnaires were distributed in January 2006 and a follow up prompt was sent out via collaborative institutions in March 2006.

Table 2

Distribution of, & responses to, questionnaires to “Non Academic” external examiners

Externals from non academic settings
No. sent / Responses
Pre 92 (A) / 18 / 7
Pre 92 (B) / 17 / 5
Pre 92 (C) / 8 / 3
Post 92 (D) / 3 / 2
Post 92 (E) / 19 / 13
TOTAL / 65 / 30

3.12 An initial response rate of 46% was achieved. The follow up prompt failed to elicit any further responses. (The reason for this is unknown however it can be surmised that industrial action and its effects on university registries may have had an impact). One return questionnaire was not valid and thus a total of 29 questionnaires were analysed (i.e. valid responses as a % of total = 45%).

4. Analysis and Discussion

4.1 Number of external examiners not working in an academic environment

The following proportion of external examiners from non-academic settings was revealed in the information received from Registries regarding the number of external examiners serving their university.

Table 3

On undergraduate programmes

University / Total Number of external examiners / % of external examiners from non-academic settings
Pre 92 (A) / 112 / 4.5
Pre 92 (B) / 73 / 17.8
Pre 92 (C) / 140 / 3.6
Post 92 (D) / 91 / 1.1
Post 92 (E) / 103 / 16.5
TOTAL / 319 / 12.8

Table 4

On post-graduate programmes

University / Total Number of external examiners / % of external examiners from non-academic settings
Pre 92 (A) / 140 / 9.3
Pre 92 (B) / 45 / 8.9
Pre 92 (C) / 88 / 3.4
Post 92 (D) / 63 / 1.6
Post 92 (E) / 20 / 10
TOTAL / 356 / 6.5

4.1.1 Whilst external examiners working in non-academic settings appear to be in the minority -whether the programme is undergraduate or post-graduate- there were nearly twice as many working on undergraduate programmes in terms of total numbers in the study. However this pattern is not reflected in the distribution between the programmes of individual universities. Rather there is considerable variation between individual universities even in this small sample. Furthermore the study did not show any distinction between pre and post 92 universities in their appointment of external examiners from non-academic settings. Although, as stated, the study confirms that this is a minority group within the external examiner population a total of 65 external examiners from non-academic settings in a sample of only 5 universities is not inconsequential and supports the necessity to recognize the needs of this group.

4.1.2 It is commonly accepted that external examiners from the professions and industry are most likely to be sought by professional/vocational programmes. The University Registries in the study initially provided a breakdown of the School/Faculty external examiners were attached to. The distribution of non academic external examiners between different disciplinary areas according to this breakdown is shown in Table 5.

Table 5 Disciplinary areas

University / School/Faculty
Pre 92 (A) / 3 Architecture
3 City Planning
2 History &
Archaeology
4 Media & Cultural Studies
4 Medicine & Dental
1 Nursing
1 Pharmacy
Pre 92 (B) / 10 Arts
7 Social Sciences
Pre 92 (C) / 1 Human Sciences
5 Health Sciences
2 Engineering
Post 92 (D) / 2 Health & Soc Science
1 Business
Post 92 (E) / 4 Applied Science
2 Art & Design
3 Education
7 Health & Social Sciences
1 Lifelong Learning
2 Sport & Leisure

4.1.3 It is difficult to draw any firm conclusions from this information owing to the different categorization of Schools/Faculties across institutions. Also one Registrar highlighted the difficulty of identifying external examiners “not working in a higher education academic context” particularly in the fields of medicine and other health-related disciplines where “the lines between higher education and the professions are often intertwined”.