WGSSF Final Report to AC 6 March 2007
LONDON’S GLOBAL UNIVERSITY
ACADEMIC COMMITTEE
WORKING GROUP ON STUDENT SUPPORT FEEDBACK
Final Report to Academic Committee on Student Support Feedback
Executive Summary
The Working Group on Student Support Services met twice to consider how UCL could gain an overview of student feedback on its student support services. This report:
- presents the background information on the establishment of WGSSF, its remit and rationale [paragraph 1];
- outlines the key areas of focus considered by the WGSSF [paragraph 2];
- outlines the current provision for receiving feedback from students on the UCL student support services [paragraph 3];
- reports on the WGSSF discussions on the suggestions and proposals for creating an overview of the student feedback for the UCL Support Services [paragraph 4];
- informs the Committee of the conclusions of WGSSF [paragraph 5];
- reports on further developments of student feedback matters which emerged from the December meeting of AC and discussions of these matters by Council and Senior Management Team officers [paragraph 6];
- asks the Committee to note the future work of the new group which will succeed WGSSF in considering the Student Barometer and other matters relating to feedback on the student experience [paragraph 7].
Key to abbreviations:
AC Academic Committee
CERG Council Effectiveness Review Group
DTC Departmental Teaching Committee
MORI Market and Opinion Research International
NSS National Student Survey
QMEC Quality Management and Enhancement Committee
SEQ Student Evaluation Questionnaire
WGSF Working Group on Student Feedback
WGSSF Working Group on Student Support Feedback
1 Background and remit of the WGSSF
1.1 The Working Group on Student Support Feedback was commissioned by the Vice-Provost (Academic and International) in response to two developments:
(i) In 2005 the QMEC Working Group on Student Feedback, in the context of a review intended primarily to deal with feedback on teaching and learning at departmental level, raised questions regarding the mechanisms by which UCL could gain an overview of student opinion on its support services. The WGSF was not aware that there was currently any mechanism in place for such an overview, although it was clear that the individual services carried out their own surveys, focus groups etc to receive feedback. It advised that some thought should be given to how this feedback could be brought together overall, in order to help UCL identify common student concerns and share best practice between the support services.
(ii) The report of the Council Effectiveness Review Group (approved by Council at its meeting on 14 June 2006) recommended that Council should in future receive an annual report on the student experience, including information about the findings of student evaluation questionnaires.
1.2 The WGSSF membership included representation from all areas offering student support in UCL, in addition to the Chair of QMEC, the Dean of Students and representatives of UCL Union and the University of London Union. A list of the membership of the WGSSF is at Annex 1.
1.3 The WGSSF agreed that its terms of reference should be as follows:
(i) to ascertain that feedback mechanisms are in place in all areas of student support and to ensure fitness for purpose;
(ii) to explore ways to fill lacunae in existing feedback mechanisms within the student support services;
(iii) to identify and share examples of best practice;
(iv) to determine the need and scope for any adjustment or addition to existing lines and modes of information gathering and reporting;
(v) to make recommendations to Academic Committee on the above.
2 Introduction
2.1 WGSSF met on two occasions in October 2006. In order to facilitate its discussions, it considered two main areas of focus:
· The current provision made by UCL student support services in order to receive feedback from students;
· Suggestions and proposals for creating the overview of the student feedback for the UCL support services.
2.2 Further to these areas of focus, WGSSF also considered in more detail the evidence base for an overview of student feedback. It noted that the annual report on the student experience would need a strong evidential basis, and considered two main methods of providing this:
(i) a consolidated report of the existing UCL Services’ surveys and reports identifying common concerns, good practice and key matters of note;
and/or
(ii) the introduction of a pan-UCL services questionnaire developed specifically for the purpose of providing data for the annual report.
2.3 In order to facilitate the above, WGSSF examined in particular the use of data from already existing surveys of UCL students, such as the MORI Unite Living survey, as well as the possibility of the introduction of common questions for existing surveys in order to aid comparison.
3 UCL Student Support Services (and others) – Current Provision for Student Feedback
3.1 The WGSF report from 2005 collected information from a number of UCL support services on the mechanisms for receiving feedback from students. WGSSF members updated this information and gave more details on the rationale and methodology for the provision, the use made of the data and the reporting arrangements for discussion of the main findings.
3.2 WGSSF considered that the table provided useful evidence for the measures currently undertaken by student support services in eliciting feedback from students. It noted that virtually all the support services sought feedback from students, although there was much variation in the rationale for doing so, the methodology employed and the use made of the feedback received. The updated table is at Annex 2.
4 Suggestions and Proposals for Creating the Overview of the Student Feedback for the UCL Support Services
4.1 During the discussions of its remit and the matters raised by the WGSF and the Council Effectiveness Review Group, WGSSF considered the possibility of the creation of a consolidated report drawn from the various support services reports and surveys, in order to gain an overview of student opinion on its support services.
4.2 A number of concerns and objections were raised to this suggestion during the discussion. Many WGSSF members considered a consolidated report to be impractical, marrying up too diverse services and feedback mechanisms to produce meaningful overall information. Questions arose as to how the data produced by the various support services could be aggregated, with concerns that this could lead to misleading comparisons from the various student response data.
4.3 It was also noted that if a consolidated report were to limit the use of the raw data from the support services reports (in order for it not to be too unwieldy), so much detail would be lost that its value would be questionable. A consolidated report would of necessity create additional reporting lines and add to the administrative burden without necessarily producing value for money. In the light of these concerns, WGSSF agreed that an annual report on the student experience should not be based on the surveys or summaries of the student support services.
4.4 WGSSF also discussed the possibility of constructing a pan-UCL survey for all students incorporating questions on all support services, or introducing the common questions into all existing and future surveys in order to allow benchmarking of the services and for comparability.
4.5 WGSSF was not convinced that it would be useful to introduce generic questions, given, the different purposes and rationales of the surveys conducted within UCL. There was also some reluctance to introduce new questions to questionnaires which had evolved over a long period of time and were functioning well and a concern that the integrity of such surveys could be compromised.
4.6 WGSSF noted already existing surveys, such as the Ipsos MORI Unite living survey, a wide-ranging questionnaire conducted annually at various UK universities. The questions were highly detailed and covered wide aspects of the student experience including views of academic matters, accommodation, learning resources as well as financial and social matters. Interviews were conducted at random on campus and the survey, now in its fourth year of operation, also allowed for comparability within the data between years. UCL’s sample of 54 students, whilst small, was a significant proportion of the overall numbers of students completing the survey nationally, which was 1025. UCL was provided with the raw data on its own student responses, as well as with the overall response data to allow comparison. The survey was of no cost to UCL and permission was granted by the Dean of Students each year for Ipsos MORI to interview UCL students, in return for the report and the results of the data.
4.7 WGSSF agreed that it would be useful for UCL to make more use of the data provided by the survey and that Ipsos MORI should be approached to discuss the possibility (and costings) of two alternative options:
(i) That Ipsos MORI provide an analysis on the UCL respondees to the already existing survey summarising key findings. Should the analysis prove useful, the Ipsos MORI information could be presented to Council and/or AC annually, supplemented with data from the NSS (especially if the expanded form currently being piloted were to be adopted) and possibly from other surveys such as the Student Barometer Survey. Consideration could also be given to presenting high level outcomes from the academic annual monitoring process in order to give a more complete picture of feedback on the student experience.
(ii) That UCL commission a specially devised survey, incorporating some elements of the existing Ipsos MORI survey, targeting a cohort of students over the years of study and after graduation. It was envisaged that this option would be conducted as a pilot scheme to determine whether the quality of the information provided would add to information already available.
4.8 WGSSF suggested that whilst the second option might be in itself a valuable exercise which could provide much useful data on changing student perceptions, it would be more cost-effective in the first instance to attempt to make use of already existing information.
4.9 In the event, WGSSF made contact with Ipsos MORI and was informed that it was no longer involved in the 2007 Unite Living Survey. Ipsos MORI also advised WGSSF that, in their opinion, it was not practical to provide further analysis of the UCL data from the 2006 Unite Living survey. The rationale given for this was that the sample size of the students was too low to safely draw any meaningful conclusions. Statisticians generally accept that the minimum response figure for feasible analysis of survey data is twenty to thirty and the UCL data (with only fifty responses overall) would very likely go under this threshold, should more detailed analysis of the data be conducted. WGSSF did not pursue this line of enquiry further. Following the discussion at AC of the Student Barometer and decision for the WGSSF, (or a sub-group thereof to consider it further), as well as direction given by UCL officers to the group on student feedback matters (see paragraph 6.2 below), WGSSF also chose not to pursue further the option of commissioning a specially devised survey from Ipsos MORI.
5 Conclusions
5.1 WGSSF agreed with Council’s suggestion that the introduction of an annual report on student experience would be a good investment in principle, provided that it benefited UCL and its students. However, WGSSF did not believe (due to the difficulties outlined in paragraph 4.3 above) that it was appropriate to produce a consolidated report from the existing support services surveys and reports, nor that it would be practicable to create a pan-UCL survey.
5.2 WGSSF notes that the student support services are only one part the overall student experience and that students are far more concerned with academic matters. Any attempt to gain an overview of the student experience should ideally supplement feedback on support services with feedback on other aspects of student experience. WGSSF suggests that the evidence for the annual report should make use of existing data which incorporates the findings of surveys concerning academic matters, such as SEQ summaries from DTCs and from other sources (such as the NSS and the Student Barometer), in order to obtain a broader picture of student views.
5.3 Considerable progress had been made in identifying survey practice in the support areas. The chair and secretary of WGSSF discussed with each area the nature of their survey and reporting mechanisms in order to develop and complete the grid. The completed table is at Annex 2.
5.4 WGSSF suggests that where possible, student surveys should be refined to take note of different categories of students. This would help to improve the quality of the data received and allow services to focus on different student needs and concerns. It would also assist the work of the Graduate School, which was especially important in the light of the increased emphasis on post graduate students at UCL.
6 Further Developments
6.1 Subsequent to the submission of the WGSSF Interim Report to AC, a number of further developments emerged from the AC meeting and following additional discussion between UCL officers on student feedback matters more generally. The December meeting of AC also discussed the Student ` Barometer survey and resolved to accept recommendations which emerged from the report by the Director of Educational Liaison (at APPENDIX AC 1/06 (06-07). AC agreed that the WGSSF, or a sub-group thereof to be established, should examine the Student Barometer in depth and undertake the following:
· Consider the Barometer and other similar survey outcomes in detail and alongside the UCL Corporate Plan 2006-2012, the International Strategy, the Teaching and Learning Strategy and, as appropriate, supporting strategies;
· Working in consultation and collaboration with service heads, identify those areas where there is potential for developmental work which could realise a significant return;
· Consider whether there is scope for working with the other Barometer participants (for example, King’s College London) to address specific issues;
· Review how best to provide faculties with disaggregated Barometer and other survey data, and advice on how to make best use of it;
· Monitor the impact of developmental work undertaken on future survey performance;