Appendix D: The Interim Assessment
1The current process
The Interim Assessment is currently a requirement of all research degree programmes. In effect, it is conducted on the model of the viva, but without involvement of External Examiners. The candidate submits a report of 4 000 words and then attends an oral examination before a panel of internal examiners. The panel then reaches decision about progression (continuation; transfer to another programme; termination) on the basis of the academic standard of the work presented. This is a significant difference between the Interim Assessment and the Annual Progress Report: the latter is partly concerned with academic standards but is also designed to ensure that certain procedural requirements have been met.
2Success of the current process
There is no clear evidence to suggest that the Interim Assessment does not work in practice. The recently established Postgraduate Research Awards Board now monitors completion of the Interim Assessment by a candidate and reports decisions to Postgraduate Office. However, there are a number of deficiencies in the current running of Interim Assessments, some of which have been identified by the Postgraduate Research Awards Board. These are as follows:
ithere are no clear regulations concerning membership of panels for Interim Assessment, and there are some problems with the category of member `appointed by the Director of Graduate Studies’
iithe forms that report the outcome of the Interim Assessment need to be re-designed to suit the range of research programmes and to clarify the decisions being made, as well as the identity of those who are making the decisions
iiipartly as a result of inadequate forms, paperwork recording Interim Assessments is often of substandard quality and provides little clear indication of what has been involved in the actual conduct of the Interim Assessment
ivthe Interim Assessment is also used to upgrade students to PhD status from MSc (by Research), MRes, or MPhil programmes. This introduces some confusion into the process and clarity is required on this matter
Some of the above points can be addressed through regulations and through re-designing the report forms for Interim Assessment. But there are other points that need to be addressed.
3Implications of the current process
As a result of the above, certain developments are threatening the value of the Interim Assessment process:
ithe process of reporting on Interim Assessments is not being used to its full benefit, but is being seen by some as a mere form-filling exercise. This renders the Interim Assessment merely a hurdle for Postgraduate Researchers and Supervisors/Assessors to get over, rather than the Interim Assessment being seen as something of benefit to the supervisory process
iiwhilst poor form design may be responsible for this reaction, it is also likely that greater clarity over the relevance of the Interim Assessment process would make reporting back on its results more relevant and of greater use, therefore leading to improvements in the quality of the report
iiipoor reporting of the Interim Assessment means that the Postgraduate Research Awards Board often finds itself speculating upon the academic standards of the research being undertaken. There is a danger that this will develop into centralised policing of research degree programmes delivered in Research Institutes that have been granted relative autonomy for the conduct of research within the University
4Reasons for improvement
The Interim Assessment should not be thought of as an intrusion into the real work of the PhD process – an irrelevancy that is merely there as a progression hurdle that requires the completion of forms that are largely seen as irrelevant. This is an indictment of the whole process that suggests it is in danger of wasting the valuable time of Postgraduate Researchers and of academic staff involved in the oral examination and who are required to complete the necessary forms. We may be missing important opportunities that could be realised through refinement of the Interim Assessment process.
More specifically, the Interim Assessment should be recognised as something that allows Research Institutes to make crucial assessments of the academic standards of work by their Postgraduate Researchers and also enables Research Institutes to demonstrate the good health of the research programmes that they effectively deliver on behalf of the University.
This document proposes amendments to the conduct of Interim Assessments and to the way in which they are reported. The proposals are made as a means of enhancing both aspects of the Interim Assessment (assessing the academic standards of the PGR and demonstrating the academic quality of the Postgraduate Research Programme), with a view to: improving the experience of the Interim Assessment for the PGR and removing any growing tendency to locate all quality controls for Postgraduate Research Programmes at the centre.
5Proposed Improvements
FIRST IMPROVEMENT
At present the Interim Assessment occurs as a single event: the PGR is given a single opportunity to defend progress on the research project (there are no clear options for ‘re-submission’ and no guidelines re what to do in the case of failure at that ‘event’). Rather than being a single event, the Interim Assessment should be re-formulated to become a process of academic achievement. That is to say, it is a process that should commence 15 months after registration on a full-time PhD and should be completed no later than 18 months after registration. During this 3 month period the PGR should demonstrate the academic standard of her/his work, and the Research Institute should be looking to demonstrate that it has brought the PGR to a satisfactory level of academic performance at this stage of the degree programme. This would work as follows:
-immediately following completion of 15 months on the programme, the PGR submits the 4 000 word report and the oral examination is held
-the panel conducting the Interim Assessment report on the following:
- Assessment of progress against generic criteria (eg, clarity of research focus; awareness of current literature; development of research methodology; presentation of data/findings to date; formulation of plan to bring project to completion; completion of required research training, as identified by the Learning Agreement). It may well be that this aspect of the report is produced in the form of tick-boxes against these criteria, on the understanding that those appointed to the panel by the Research Institute concerned are effectively given responsibility for upholding the RI’s understanding of these elements of research project management
- Evaluation of the project at this stage. This should be modelled on the Examiners’ Report following a viva (or, rather, it should be modelled on an improved version of that form). In this section the panel members record their assessment of the academic standard of the work produced by the candidate, and for this to work the comments must be extensive
- Diagnosis of future research training requirements (it is envisaged that a requirement to complete a range of training sessions of a general nature – provided centrally by SPoRT – will be integrated with a requirement to complete a range of generic and specific training sessions provided by the Faculty and/or the Research Institute, or provided outside the University by regional or national arrangements) These future requirements are then to be written into the Learning Agreement through a supplementary section that is completed by the panel and the candidate as part of the Interim Assessment process, and then signed off by supervisor and PGR
-NOTE: by holding this event immediately following completion of 15 months registration, 2 significant improvements are made:
isuccessful candidates are put upon a firm footing to proceed with their research on the basis of detailed feedback from the Interim Assessment panel (often including suggestions for new avenues of enquiry, improved modes of reporting findings) and also with the benefit of clear identification of research training suited to their particular needs in the second stage of their research
iiunsuccessful candidates receive constructive criticism in time for the Research Institute to address their deficiencies. By allowing the entire process to take up to three months (between 15 and 18 months after registration) this gives the RI a real opportunity not only to report on a candidate’s weaknesses/below par performance, but also to work with that candidate in a focused and informed manner to improve the standard of research and bring it up to the level expected of postgraduates by the Research Institute concerned. This could entail changes to supervision to enable the candidate to produce more effective demonstration of findings, and/or research training that is tailored to improve areas where the candidate is under-performing.
-NOTE: If the above process is adopted, there is no need to request ‘resubmission’ to the Interim Assessment. In effect, the RI will work with the candidate to raise his/her game to the required level (ie the level of research that the particular RI deems to be appropriate for its postgraduates) before then making a decision (at the end of the maximum period of 3 months if necessary) on whether or not the candidate should be allowed to proceed.
-ALL decisions made as a result of the Interim Assessment process must be submitted to the Postgraduate Research Awards Board via the Research Institute Board
In effect, these changes will mean that the Interim Assessment is something of real purpose and value for the Postgraduate Researcher and for the supervisory team involved in her/his research. All parties will be assured that a required level of academic achievement has been reached and that a clear line of progression to the eventual completion of the thesis has been identified. In the cases where a candidate did not initially achieve this level of achievement at the oral examination considerable support will have been provided in a positive and wholly legitimate manner to bring that candidate up to the required level of academic achievement before too great a period of registration has been exhausted.
SECOND IMPROVEMENT
Once the Interim Assessment can demonstrate its validity to all parties concerned, then the completion of necessary paperwork may well see improvements. However, it would be naïve to believe that this would automatically follow. For improvements in paperwork to occur, the actual form-filling stage of the Interim Assessment has to be seen to be of some value in and of itself (otherwise the view will soon develop that the process was done well, but the paperwork is just a subsequent use of valuable time that can be ill-afforded). Whilst there is a school of thought which says that if a job has been done well then it should be easy to record that success, realistically the reasons for ‘good paperwork’ have to be demonstrated (NOTE: the introduction of tick-boxes is intended as an economic use of time to record the candidate’s success at reaching certain academic levels required of its postgraduates by the Research Institute, but there is still an insistence upon production of the equivalent of an Examiners’ Report commentary). The key to this is to be found in the modification which stipulates that all reports following Interim Assessment must be forwarded to the Postgraduate Research Awards Board via the Research Institute Board. This requirement is introduced not simply so that the RI can get regular reports on what is going on in terms of postgraduate research (this in itself is significant) but also in order that, via Minutes of the Research Institute Board, the Interim Assessment can now be used to demonstrate (to internal and external audiences) the good health of the RI’s Postgraduate Research Programmes as follows:
ithere is evidence of high quality supervision in bringing the candidate to the level of academic achievement required by the RI and confirmed by the Interim Assessment
iithere is evidence of the benefits of support provided through required attendance at research training sessions delivered centrally (by SPoRT) and locally (by the Faculty/RI), and/or delivered outside the University (regional collaboration, or via national ‘Graduate Schools’)
iiiimmediate, and focused/tailored additional support is provided for struggling candidates (who fall short at the initial ‘event’) and this can be demonstrated to bring them up to the required standard of academic achievement
ivthe process is used to diagnose precisely the kind of research training that the candidate requires for the second stage of the research project, this being a combination of training provided by SPoRT, and by the Faculty/RI (or provided regionally and/or nationally)
vrather than being viewed simply as a progression hurdle, the Interim Assessment is seen as being of value to the candidate. Just as importantly, the Interim Assessment is used by the Research Institute to reflect upon the good health (quality) of the research programmes that it delivers. The report to the RI Board becomes a demonstration of the RI’s guardianship and maintenance of the academic standards it requires with regard to postgraduate programmes delivered under its name.
6Role of Postgraduate Research Awards Board
The Postgraduate Research Awards Board was introduced as a result of comments made in the Continuation Audit of 1999 which were critical of the fact that only one person appeared to be responsible for verifying the award of research degree qualifications. On the basis of the above it may be felt that if we went down this route then the Research Institute Board would be ideally placed to fulfil the role of the Awards Board. However, to agree to the above modifications of the Interim Assessment, and then to insist that its validation reside within the Research Institute Board, is to misunderstand the function of the Postgraduate Research Awards Board once these modifications have been made. The functions of the Awards Board are as follows:
iit is the University body that actually approves award on research programmes (it does so directly since it does not report to Research Committee)
iiwith its cross-institutional membership this means that a group of people outside the group immediately concerned with delivery of research programmes is making the final decision
iiithe Awards Board checks centrally that the process of delivery of research programmes is carried out accurately. The Board is not in a position to challenge the academic judgement of the Interim Assessment panel (just as it is not in a position to challenge the academic judgement of the Examiners at viva), because it does not have the expertise in that particular area of research (this is as a result of its constituency). But it is important to the University’s verification of the processes that are conducted by Research Institutes (eg, that appropriate people have been appointed to the panel, that there is a clear statement of the assessment of the academic standard of the research project, that required levels have been reached and that training requirements have been identified)
ivas such, the Awards Board reviews 2 key stages of any research programme (Interim and Final), and can therefore demonstrate the good health of the University’s delivery of Postgraduate Research Programmes as well as providing a broader perspective on achievement across the whole institution. This is a key aspect of the demonstration of the public accountability of Research Institutes in the area of postgraduate work because it is ongoing. It is therefore the necessary complement to the demonstration of the public accountability of Research Institutes that takes place periodically with Research Institute Review.
NOTE: it is suggested that the use of an Interim Assessment to upgrade someone from an MSc, MRes, or MPhil be allowed to continue in practice, but that the process be re-titled so as to distinguish it from an Interim Assessment for those registered for the PhD. This will simplify the paperwork and also the understanding of what is required. Since all research students are required to complete a Learning Agreement, the process can be same in terms of checking back, and looking forward, re Research Training Requirements. All decisions re whether to upgrade would come to the Postgraduate Awards Board.
Andrew Cooper Director of Graduate Studies (Dec 2002)
1
R:\DGS\Quality Review Group\APPENDIX D.INTERIM ASSESSMENT.doc