Application for funding from the Humanities
Teaching Enhancement and Student Success Fund 2010/11
Summarysheet
Title of project / Testing 1,2,3: Using formative online multiple-choice tests in the Humanities to enhance student learning, engagement and feedbackOutline of project / This project sets out to improve the processes of student learning, engagement and feedback in English and American Studies by embedding an online multiple-choice test in a popular Level 2 course, which recruits upwards of 100 students per year. Currently, the course is examined by coursework essay and written examination. However, some students have expressed dissatisfaction with the fact that there is no means of receiving early, formative feedback on their progress. In order to address this problem and enhance student engagement from the outset, I propose to use this grant to develop a Blackboard-delivered multiple-choice test that will be scheduled early in the course. A key feature of the test will be a component called ‘confidence-based marking’, whereby students not only select their answer from a set of alternatives but also rate their confidence in doing so on a three-point scale (high, medium and low). This feature is designed to deter random guessing and encourage students to reflect on the soundness of their knowledge and critically assess their reasoning.
If successful, my longer-term project aims would be:
(a)to build on this assessment tool by linking it to a summative multiple-choice examination at the end of the course;
(b)to develop a transferable model and guidelines for the application of this assessment method in other Humanities disciplines;
(c)to disseminate the benefits of this initiative and encourage cross-subject take-up on large team-taught courses, with a view to improving NSS scores across the School and Faculty.
The project has the full support of Dr Hal Gladfelder, EAS subject leader.
Name(s) of applicant(s) / Dr Liam Harte
Contact details / / ext. 53162
School / Arts, Histories and Cultures
Name of programme or unit and is it credit-bearing? / ENGL20492 Writing, Identity and Nation (credit-bearing)
Year of study / Level 2
Number of students to benefit? / There are currently 103 students registered on the course in 2010-11
Outputs
Background
Research shows that students not only vary in their preferences for different assessment methods but also that using a variety of methods helps them to achieve their full potential. While essays and examinations are valuable for assessing skills of comprehension, analysis, synthesis and evaluation, multiple-choice tests also have an important role to play in assessing students’ mastery of module content and providing them with immediate and timely feedback, especially in large cohorts. Having had direct experience of the benefits of paper-based multiple-choice exams during my time as Visiting Professor at the University of Toronto in 2008, I now wish to implement a more pedagogically and technologically advanced form of this assessment method at Manchester, beginning with the 2010-11 ‘Writing, Identity and Nation’ cohort. By developing a suitable model for an English and American Studies course, I hope to demonstrate both the usefulness and the transferability of such forms of assessment within the Humanities, where their suitability is often questioned or regarded with scepticism.
What will the development achieve?
The project will be designed with the following key learning objectives in mind:
a)to enable students to self-test their knowledge and understanding at an early stage of their studies;
b)to encourage students to think deeply and reflexively about their knowledge and understanding;
c)to provide students with immediate formative feedback that will give them a clear idea of how they are progressing during the early part of the course;
d)to motivate students to follow up on the feedback suggestions regarding further reading and research;
e)to help the course tutors to use the test results to inform lecture content and seminar discussion;
f)to improve student satisfaction with their programme of study, as measured by internal university questionnaires and the National Student Survey;
g)to develop a transferable model and guidelines for the application of this assessment method in other Humanities disciplines.
How do the outputs align with the Faculty or School’s teaching and learning priorities?
The outputs have been designed to fulfil the first two ‘Purposes of a Manchester Education’, namely, the development of ‘critical thinking and higher order conceptual reasoning and analytical skills’ and the promotion of ‘mastery of a discipline’. They are also tailored to meet a key objective identified in the Faculty of Humanities ‘Personalised Learning’ document (2008), which states that ‘All students should have access to suitable technologies and educational resources that allow flexible delivery of appropriate components of the curriculum at a time and a place of their choosing.’ At School level, the outputs are designed to fulfil the stated demand for initiatives that directly enhance student engagement and provide timely formative feedback to students, especially on large courses. It is envisaged that the provision of rapid, formative feedback, in electronic form, will positively affect levels of student engagement, understanding, interaction, performance and overall satisfaction.
How will the outputs enhance learning?
The outputs will enhance learning by:
a)helping students to evaluate better their progress towards meeting the intended learning outcomes of the course;
b)enabling students to improve their performance in the summative forms of course assessment;
c)facilitating the development of greater student self-reflexivity by means of the ‘confidence-based marking’ tool;
d)boosting students’ motivation and self-esteem;
e)providing course tutors with information that can be used to shape teaching;
f)promoting greater tutor-student dialogue around learning.
How many students will the outputs benefit, actually and potentially, and how are you going to ensure they complete the activities?
Over 100 students will benefit directly in 2010-11. ‘Writing, Identity and Nation’ has been a perennially popular Level 2 course since its introduction in 2005, and as EAS recruitment increases, there is no reason to doubt the course’s continuing popularity in the future. The mandatory nature of the online multiple-choice test means that all students on the course will derive some benefit from it, in line with the key learning objectives outlined above. Student completions will be monitored via Blackboard by the course unit director. If, as is anticipated, this project leads to the implementation of similar tests on other large team-taught courses in EAS, there is potential for every student in the subject area to benefit to some degree from the outputs of this project. I can also see ways in which a test of this kind would lend itself to cross-subject application, hence my emphasis on its transferable potential in other areas of the Humanities. Although designed for EAS in terms of content, I believe that it could provide a template for other subjects in terms of form and application.
How will the outputs be sustained after project funding ceases?
Once the multiple-choice questions have been devised and the Blackboard site enhanced to deliver the project outputs, no further funding will be necessary to sustain them. Indeed, a bank of re-useable questions will be built up by the course team over time. Any future modification or updating of the test in response to student feedback will be carried out by the course unit director as part of his or her normal administrative load.
How will the outputs be evaluated, by students and by others?
Student feedback will be actively sought by the course unit director and course tutors by informal and formal means. In addition to canvassing student opinion in seminars, I propose to devise an online questionnaire that will permit students to evaluate the merits the online test soon after they complete it. The University and departmental course questionnaires will provide additional channels for student evaluation. At the end of the course, this feedback will itself be evaluated by the teaching team and the results disseminated to colleagues with a view to encouraging them to integrate similar tests into their own courses.
A brief outline (no specifications needed) of the type of equipment needed and how it will support the project
One notebook/laptop computer and memory stick. These are needed to enable me to work more flexibly in developing and completing this project in a timely fashion, and to disseminate the outcomes among colleagues.