In the Throes of Change: an Insight Into the Impact of the EQAL Initiativeat Manchester

In the Throes of Change: An insight into the impact of the EQAL initiativeat Manchester Metropolitan University (MMU)

‘In the Throes of Change’ was commissioned to capturethe experience of implementing the Enhancing Quality and Assessment for Learning (EQAL)initiative at Manchester Metropolitan University (MMU). The aim of the report is two-fold: to capture evidence in a journalistic style of the impact of the “change experience” and to inform the work of other institutions embarking on their own curriculum transformation initiatives.

The EQAL initiative is not yet complete; written in spring 2012, the report captures the views of MMU staff and students caught up in a process of far-reaching curriculum change. Twelve in-depth interviews were held during February 2012 with staff ranging from members of the Executive to curriculum leaders and administrative personnel. In addition, the Vice-President of the Student Union was interviewed togain insight into the student perspective on EQAL.

The EQAL initiative has been informed by work undertaken by the JISC-funded Supporting Responsive Curricula (SRC) project and by MMU's new Strategic Framework for Learning Teaching and Assessment and Threshold Standards for the Student Experience.

Introduction

Manchester MetropolitanUniversity (MMU) is a multi-site university with over 35,000 students formed from a number of institutional mergers which left a legacy of different approaches to learning, teaching, assessment and information managementacross the institution. The size and diversity of the institution, combined with indifferent National Student Survey (NSS) results (in particular under the category of Organisation and Management)and looming changes in the relationship between student and university as a result of increased student feesprompted senior management to initiate an ambitious four-strand change programme designed to improve the student experience and increase the efficiency ofacademic processes. The initiative, Enhancing Quality and Assessment for Learning or EQAL, commenced two years ago.

A number of initiatives have come together under the EQAL banner; combined, they have produced significant changes tothe curricular, administrative and quality assurancefunctions of the university. Following developments in thetechnical infrastructure, there have also been fundamental changes to the way students interact with the university through its online learning environment.

Deputy Vice Chancellor, Kevin Bonnett, describes the EQAL initiative as‘a multi-faceted and medium-term project’. Thus the views articulated in this report reflect the beliefs, emotions and aspirations of individuals caught up in the throes ofa significant,but as yet, incomplete process of change. As a result, theirperceptions are inevitably coloured by their role, their ability to see the wider picture and the degree of impact EQAL has had on their working practices – for some, a sense of frustrationand loss of control predominate, for others there is a feeling of relief at havingachieved goals on the edge of possibility.Overall, there is broadunderstandingand acceptance of the aims of the initiative and satisfaction in working together to achieve an improved student experience.

Those involved in the EQAL initiative tell the story in their own words under four broad headings: the academic, IT systems, quality assurance and student-facing dimensions. Under each dimension, there is an outline of what has been achieved, and why, followed by a discussion of the benefits encountered and challenges met. Each section concludes withan outline of some of the unexpected outcomes produced by EQAL.

Academic dimension

Achievements

The EQAL initiative has standardised the curriculum in all disciplines (with occasional exceptions) to a consistent 30-credit unit model with a set number of learning outcomes and summative assessments per unit. In September 2011, thenew undergraduate curriculum structure was introduced at levels 3 and 4, the foundation and first-yearprogrammes. During 2012–2013, the revised structure is beingextended to level 5 (year two of an undergraduate degree) and to level 6 in 2013-2014 so that the new framework will be in force on all undergraduate programmes from September 2013.Postgraduate courses are not involved in the initiative.

“The process has been evolutionary as much as revolutionary - there was no big bang. The new curriculum was introduced for levels 3 and 4 initially and is now moving on to levels 5 and 6 so it unfolds as those first-year students progress on through their degree programmes.” Kevin Bonnett, Deputy Vice-Chancellor

The new structuredemands a more consistent approach to assessment planning and organisation;the way assessment data is collected has also undergone change. Academic teams have been asked to rethink and then revalidate their learning outcomes using a new programme modification and review process;they theninput information about themodified unit specifications directly into the university’s curriculum database using a template designed for the purpose. Following the introduction of a web-services approach to the university’s IT infrastructure, the data can then be re-used many times – for example, in strategic decision-making, marketing orrewriting the university’s prospectus.

Redesigning the undergraduate curricular structure has created a further opportunity to upgrade the university’s curriculum offer. The Employability Curriculum Framework, completed just prior to the commencement of the EQAL initiative, has been embedded into allnew assessment documentation: academics drawing up programme and unit modificationsare now asked to outline how learning outcomes will support the achievement of graduate outcomes; opportunities to demonstrate skills and attributes arethen flagged up forstudents in the description of each assessment and included onfeedback forms. Initiallya separate initiative, the integration of employability has been an example of the broad scope of the EQAL initiative and the momentum it has created. The employability initiative has been led by Penny Renwick, Pro Vice-Chancellor, Quality and Development:

“Without EQAL it would have taken a great deal longer to have arrived at this point. Although employability wasn’t the main thrust of EQAL, a lot has been achieved in terms of forging links between curriculum design and graduate prospects. I suspect that EQAL and the changes in documentation it has introduced have had a big part to play in raising awareness of employability at MMU.”Penny Renwick, ProVice-Chancellor, Quality and Development

Rationale for change

Pedagogic benefits have been the driver behind EQAL reforms. Prompted by the NSS and internal student surveys, the university recognised the need to focus more closely on the student experience, in particular in terms of assessment and feedback. Senior staff realised that a better balance needed to be struck between the demands of assessment and the workload associated with it; academic staff needed moretime to devise more frequent and achievable formative assessments, and students needed more time and opportunityto respond to feedback:

“… Reform was driven pedagogically; we needed to put a cap on summative assessment and increase formative assessment (assessment for learning) while still controlling the size and range of learning outcomes for each unit.”Kevin Bonnett, Deputy Vice-Chancellor

There was also an issue of parity across the university’s varied programmes of learning. The size and diversity of MMU had resulted in approachesto assessment driven by tradition and disciplinary culture. The result was an unequal and sometimes burdensome assessment experience which was impacting negatively on students’ perception of the university:

“We recognised that if we were to be one institution, albeit made up of different faculties and campuses with different origins and histories, we needed a greater degree of standardisation and consistency about the way the curriculum was assessed.” Peter Dunleavy, ProVice-Chancellor, Curriculum Innovation

The inequalityof MMU students’ assessment experiences was also mirrored in the workload of academic staff–a matter of concern for Peter Dunleavy who saw a link between excessive staff workloads and the university’s lower NSS ratings:

“… a student in one discipline could have a very different assessment experience to a student in another discipline. Not only was this inconsistent, it could also be unproductive since overworked academic staff were often unable to get feedback to students in time for it to be put to good effect. Unsurprisingly, students did not rate the organisation of their courses very highly as staff struggled to deliver such a complex curriculum. Complexity is difficult to organise.” Peter Dunleavy, Pro Vice-Chancellor, Curriculum Innovation

Care had to be taken when introducing the revised structure to MMU staff to differentiate between improving efficiency and consistency by means of standardisationand stifling the creativity of curriculum experts, a message that has not been easy to convey:

“We stress, however, that creativity remains important; the university is noted for the quality and variety of assignment types and we don’t want that to change. Academic staff are in charge of their own destiny and retain control over the planning and delivery of the curriculum but, at the same time, we have tried to encourage ways of demonstrating learning outcomes that do not entail unachievable or inconsistent workloads.”Peter Dunleavy, Pro Vice-Chancellor, Curriculum Innovation

Benefits and challenges

A standardised assessment structurehas thus become the academic priority of the EQAL initiative. Students can now expect parity in terms of workload from one unit to the next, and between their chosen programme and that of their peers; students should be able to manage their time and feedback more effectively as a result of taking fewer assessments, andstaff previously overburdened bycomplex summative assessments should have more time to refocus on formative assessmentand improving the quality of feedback. Staffefforts canalso be redirected towards enhancing the institution’s standing – for example, throughinnovative practice or research, overseas recruitment, commercial income generation or community engagement initiatives.

Along with these potential benefits, changes to the assessmentstructure havepresented some complex challenges. It was clear from the outset that to ensure coherence in terms of the student experience the new structure would need to ‘grow’ with the first-year students; this would necessitatea very strict schedule of events withwork occurring on several fronts simultaneously. Thus two years of intense activity have been experienced by academic and administrative staff as they deliver /support the first phase of the new curriculum, submit modifications for the next stage and, at the same time,maintain the status quo on courses unaffected by the EQAL initiative.

Unsurprisingly, the speed of change and initial additional workload have been cited by some staff as having a detrimental effect on teaching, with the burden falling unevenly on those most heavily committed to frontline activities:

“Lecturers who have high teaching loads, and consequently a large volume of marking, they would have a large number of units to re-write for EQAL at the same time as teaching and marking to undertake their normal workload. This has created a lot of pressure on tutors.” EQAL Faculty Lead

Academic staff havealso raised concerns that the reduction in the number of summative assessments to two per unit has placed too great a value on each assessment, while also reducing optionalityand choice for students. Some staff have found it necessary to combine learning outcomes that havelittle real connection in order to fit the standard:

“I don’t like the 30-credit unit size; it’s too big and prevents a diverse curriculum, reduces student choice and requires lumping things together which don’t naturally fit.” EQAL Faculty Lead

Students also worry that the higher value of each assessment could have a proportionate impact on the degree awarded if they underperform in several units, and both students and tutors have queried whether limiting the number of learning outcomes will conflict with the requirements of professional bodies:

“I am concerned about losing the optional units in courses. Without these, professional bodies might be less interested in students because they won’t have such a broad education.” Vice President, Student Union

Nonetheless, there is recognition among curriculum staff that EQAL represents a ‘fantastic opportunity to review and update our provision’and the process of change has not always been a painful one. In many faculties, good teamwork has enabled curriculum staff to meet the challenge of EQAL. Although at a very early stage, some staff arealso beginning to recognise the value of formative assessment, and the benefits to be gained from a richer variety of assessments:

“… reducing summative assessments to two, where some may not initially like it, it means using a better variety of assessment types, better use of formative assessments and different methods of assessment. The requirement is for two summative assessments so teams can be more creative with formative assessments.” EQAL Faculty Lead

“… there is a bit of a buzz about assessment at the moment. People are asking usif we are assessing students in the most appropriate way. [EQAL] helps us think about assessment as an ongoing rather than just a summative activity.” Rod Cullen, Centre for Learning and Teaching (CeLT)

Unforeseen outcomes

Central services such as the Centre for Learning and Teaching (CeLT) and the Centre for Academic Standards and Quality Enhancement (CASQE) have gained a wider profile across the university as a result of EQAL – an unforeseen but beneficial outcome of the initiative. Academic staff, for example, turned to CeLT for assistance in writing learning outcomes for the new undergraduate curriculum. Evidence of benefits gained from involving the CeLTteam inthe first round of unit modifications persuaded the EQAL Board to task the Centre with reviewing all learning outcomes for units at levels 5 and 6:

“I know this has been a leap of faith for many, but with the assistance of CeLT, we have made remarkable progress.” Peter Dunleavy, ProVice-Chancellor

The role of a small but dedicated quality assurance team has also claimed a wider significance:

“The university has for the first time grasped the nettle of making the fundamental changes to the student experience that today’s climate demands. I also believe the Centre for Academic Standards has gained a higher profile across the university. In that respect, we have moved forward significantly.” Peggy Cooke, Head of CASQE

IT systems dimension

Achievements

“The introduction of the new curriculum across the board for all students in that year was beneficial in that it forced us to do a lot of things differently.”Kevin Bonnett, Deputy Vice-Chancellor

One of the complexities of EQAL is that some projects already underway before the launch of the initiative have become absorbed under the EQAL banner. The common thread that united each project was enhancing the student experience, but sometimes thiscan occurin less obvious ways – for example, upgrading the IT systems and administrative infrastructure that support the student learning experience.

One of the most striking and challenging achievements of EQAL has been the new Programme and Modification Review Process. This includes a new online unit specification system, now completed by academic teams, which provides a single source of assessmentinformation for all units in undergraduate programmes. The system, created by a multi-disciplinary in-house group, has been used to record the details of all newly modified units during the last year.

The origins of this project lie before EQAL but coming to fruition at the same time as EQAL has enabled the role of IT systems and infrastructure in supporting assessment management to be more widely recognised.

EQAL has also enabled the development of other systems to move forward at a faster pace; notable among the work overseen by Student Services is the personalised timetables project. Students who know which sessions to attend, when their assignments are due, and can access this information from any location, are more likely to progress academically but, while desirable, such a project might have foundered for want of targeted investment. The timetabling projectnot only acquired additional financial support, once part of EQAL, but also broadened understanding of how the basics of student life can impact on academic success:

“There is a better understanding now of the connection between basic aspects of the curriculum and the student experience… it doesn’t matter how good teaching and learning is, it will be dragged down if the basics aren’t in place, like knowing where you are supposed to be, what assignments will be set and when the submission dates are. I think that understanding of this is beginning to grow across the institution now.” Mark Stubbs, Head of Department, Learning & Research Technologies

Rationalefor change

Completing changes to the infrastructure under the auspices of a cross-institutional initiative such as EQAL has offered a number of advantages: it has enabled a wider community to realise the interdependence of apparently disparate elements of the university experience; technical and investment issues are alsomore rapidly addressed with senior management involvement: