CHIEF EXECUTIVE
PROFESSOR SIR STEVE SMITH AcSS
Northcote House
The Queen’s Drive
Exeter
UK EX4 4QJ
Telephone +44 (0)1392 263000
Fax +44 (0)1392 263008
Web www.exeter.ac.uk
SS/RB
30 August 2012
Dear Andrew
Professor Small: External Examiner’s Report 2011-2012
College of Humanities: English
BA English
BA English with Film Studies
BA Film Studies
BA English with Study in North America
BA English and French
BA English and German
BA English and Italian
BA English and Spanish
BA Classical Studies and English Outside Students
Good Practice:
First, my thanks to The Chair of Examiners and to all the other members of department and the administrators involved in the examining process this year. The Chair of Examiners work in support of the external examiners was a model of efficiency and clarity, and I’m grateful for all the quick answers to a new-comer’s questions. I’m also grateful for the kind hospitality from the department during our stay. Like all the externals I have been very favourably impressed by the high standard of work produced by second and third year students of English, and by the almost complete absence of really poor work. You are clearly recruiting good students, but you are also turning them quickly into adept and (at the top end) stylish critics of literature. The various programmes to assist with writing skills (including the Literary Fund Fellows) are evidently playing an important part, but the essential element is the quality and specificity of the feedback they are receiving on their written work from all your tutors. Module design was without exception intellectually attractive and properly demanding, and much thought had clearly been given to the various ways in which students could be helped to progress during the course of a term. The graded essay proposal, being used as a ‘middle element’ in some of the modules seemed to me to be working well as a means of requiring students to articulate early on the scope, the structure, and the perceived originality of the 3000 word essays to come. With an eye on my particular area of responsibility, Victorian literature, I was very impressed by the currency of the work seen with debates at the fore of Victorian Studies right now, and by the richness of the supporting research tools made available to students through reading lists and websites. The department is clearly making good use of the university’s special collections, and producing work that is not only conceptually exciting but that involves, at its best, genuinely independent research.
Recommendations:
Essential: Areas of concern which, in your opinion, place academic quality and/or standards at risk and require an immediate response from the Associate Dean for Education.
The procedures set out by the department and division are appropriate. One case arose this year in which they were not correctly followed, and corrective 'remoderation' had to be employed. I have made some comments, below, on the general points emerging from that case. I have also commented, below, on the need for close attention to the spread of marks on the larger modules, where, in general, greater generosity might be employed to discriminate achievement at the top end, and, in one case, greater toughness was needed to distinguish weak performance at the other end.
This comment arises out of the handling of one exceptionally difficult case (dealt with in a separate report for the department) in which the initial standard of marking and the criteria employed in giving written feedback were problematic and clearly departed from departmental approved practice. No department can entirely protect itself against cases where the judgment of a tutor suffers for reasons not fully within their own control. The department's handling of this case was tactful, effective, and entirely appropriate. In my verbal comments at the final examiners' meeting I singled out for praise the work of the very recent appointee given the difficult task of remoderating that course. The students need have no concern about the justice of their final marks and comments. Two general points emerge from this case. The first is that the department needs to ensure that the method and criteria by which each module is assessed are not only clearly laid out from the beginning (described in the course materials, and on the supporting websites) but that they are known not to be subject to alteration if problems are perceived with (for example) the student performance on the first marked element. The second concerns the use made of presentation exercises as a marked component in the grading of modules. This seems to be (rightly) a minority practice in the department, but a permitted one. I am not convinced that in the case of literary critical work (as distinct from some forms of drama and film work) the student presentation should be allowed to contribute to the final mark, as opposed to being simply a requirement for participation in and completion of the module. If it is allowed to play a part in the marking it seems to me essential that the students’ own peer assessment plays no part in the formal mark, and that the tutor’s assessment is backed up by a clear record of how and why a mark was arrived at. My primary reason for doubting the value of these marks is that it seems to me (from experience) very difficult to devise a grading system for verbal presentations that is not unduly influenced by psychological factors—including the students’ and tutors’ awareness of the nervousness and social vulnerability of students unused to making formal oral presentations. It is certainly difficult to devise a marking scheme as clear as those that operate for written examinations, critical commentaries, essay proposals (the elements on other modules with which the presentation exercise ostensibly has parity).
Advisable: Areas of concern regarding threshold standards which, while currently being met, in your opinion, could be significantly improved.
The concentration of marks in the II.i band is unsurprising, and no doubt correct given the quality of the student intake at Exeter and the high quality of the teaching. That said, there seemed to me to be a need for greater numerical discrimination on at least one of the most popular second year modules this year. If a case arises where, on first marking, close on 90% of candidates have achieved II.is, only two have been given II.iis (at the highest possible II.ii mark) and the few firsts given rarely reach above 72 and never about 75, then the necessary discrimination is plainly not happening. If the department does not already have in place procedures by which markers for a long run of scripts meet early in the marking process, discuss the scripts read so far, and agree criteria by which to discriminate adequately, then I would recommend that it puts those procedures in place. If they are already in place, then they needed, in the case concerned, to be operating more effectively. I repeat that the detailed feedback given to work was in all cases admirably fair and supported by lucid explanations to the students. My concern here is that the written feedback should link up on all modules with adequate differentiation of achievement in the numerical form. The use of averaging as a means of reaching a final module mark derived from different exercises is certainly contributing to the levelling effect seen on several modules this year (and one of the retiring externals has suggested that you give some consideration to that effect). But even allowing for the influence of averaging, my sample reading indicated that there was an undue reluctance in this instance to match tough critical comments with a II.ii mark, and an equal reluctance (more generally) to reward the stronger work in ways that differentiated levels of achievement at the top. I support the critical comments made verbally by all the other external examiners this year concerning the Moderating Procedure handed down to the department from the Division. We were, I think, united in our view that reading the first marker's comments and checking them against the grade given is not adequate as a means of 'moderation' since it involves no reference to the written work being assessed. Some greater commitment to sample second marking seemed to all of us important, not least as a means of avoiding problems of the kind described above. A specific suggestion, if the Procedure is to re-examined with a review to revision, is that point 1, relating to the handling of discrepancies, should be clarified. At present it is unclear whether markers are being asked to attend to statistical discrepancies (only) between markers, and/or between modules, and/or between performances on a module in different years. An element of all three comparisons would be ideal, but the latter two kinds of comparison would have to be enabled by the examining board, or its Chair, since they will be affected by the size of the module and perhaps by perceptions of the relative strength of particular year groups. Individual markers will not, at the start of their work, have access to the synchronic comparison, and would need guidance on how to treat diachronic comparisons. The simplest directive, of course, would be that a long run of marks with very little differentiation from a mean, and that mean fairly high, needs to be looked at sceptically.
Desirable: Areas where, in your opinion there is potential for enhancement.
No further concerns.
The procedures contained in the TQA Manual look to a response normally within eight weeks after appropriate internal discussion within the College including an opportunity for input from the staff meeting and the College’s Education Strategy Group. However, as the External Examiner has raised an Essential recommendation, an immediate response is required to the specific issues raised in this section.
Please note that the University’s statement of procedures also requires that the College’s next annual main meeting of the Boards of Examiners for the programmes in question, at which an External Examiner is present, should include early in its agenda a copy of the External Examiner’s report and of the College’s response.
Yours sincerely
Professor Sir Steve Smith
Vice-Chancellor and Chief Executive
cc Jo Hatt
Susan Margetts