Annual Programme Review Template, ACTION PLAN and GUIDANCE NOTES

06.1A

Annual Programme review Template, ACTION PLAN AND GUIDANCE NOTES

Academic Handbook 2017/18 – Volume 2 - 06.1A – Annual Programme Review Template, Action Plan and Guidance Notes – Introduced 28.10.14, modified 14.05.15, 04.08.15, 11.10.17

Cardiff Metropolitan University

Annual Programme Review

Programme(s) Active in the Academic Year

2016-2017

Cardiff School of

Title of Programme(s) and any pathways (please include previous programme title if this is still being taught out):

Name of Programme Director(s):

Link to Supporting Documents:

Summary of Measures of Success

Recruitment / Retention / Award outcomes / Career Destinations / Student Opinion / External Examiner
Red
Amber
Green

Summary of teaching quality (TEF) (home undergraduate programmes only)

NSS Teaching / NSS Assessment and Feedback
Red
Amber
Green
Date Considered by Programme Committee
Date Considered by Learning & Teaching Committee

Dated Signature of Programme Director:

Dated Signature of Student Representative:

Section 1 – Previous Action plan

Number / Issue / Source / Evaluation and Impact
(indicate evidence base)
16.01
Resolved?
YES/NO
16.02
Resolved?
YES/NO
16.03
Resolved?
YES/NO
16.04
Resolved?
YES/NO

Section 2 – Reflection on Last Year’s Action Plan

If your programme has not been through a Periodic Review or Validation, go to Section 4

Section 3 – Response to Recommendations from Periodic Review/Validation

If your programme has not made any modifications in the last academic year, go to Section 5

Section 4 – Modifications last academic year

If your programme does not have a PSRB, go to section 6

Section 5 - PSRB

Section 6

Data Population (by School Administrator)

6.1.Recruitment

6.2.Retention

6.3.Award outcomes

6.4.Career destination

6.5. Student Opinion

Section 7

Data Reflection & Commentary (1 Page max) (by Programme Director)

Strengths / Weaknesses
Opportunities / Threats

Section 8 – SWOT Analysis

Section 9

Number
KPI / Issue / Source / Actions taken/planned / By Whom / Timescale / Measurement of success

APR Guidance Document

Contents

Initial Section

Initial information

Link to Supporting Documents, i.e. Supporting Evidence

Summary of Measures of Success

Undergraduate Measures of Success

Postgraduate Measures of Success

Signatures and Dates

Section 1: Previous Action Plan and Evaluation and Impact of last year’s actions

Section 2: Reflection on last year’s action plan

Section 3: Response to Outcomes from Periodic/Elective Review

Section 4: Reflection on Modifications to Programme

Section 5: Reflection on PSRB activity

Section 6: Data Population

Section 7: Data Reflection & Commentary

Section 8: SWOT analysis

Section 9: Action Plan Going Forward

MIS - Origins of the APR data

Key Performance Indicators

Recruitment

Student Progression

Award Outcomes

Career Destinations

Student Opinion

Survey Data

UKES

SSS

PTES

Remaining data sheets

Criteria for Rating External Examiner Reports

Green

Amber

Red

Initial Section

Initial information

Designated School administrative colleagues complete the name of the School, the overarching title of the programme and the name of the programme director. If the programme contains pathways, then these should be listed in a separate WORD document and included as part of your evidence (02_ClusterList – see supporting evidence below).

Link to Supporting Documents, i.e. Supporting Evidence

Administrative colleagues set up the folder to host the evidence for the APR and ensure that the Programme Director (PD) and team and ASQU have permissions to access the folder. Administrative colleagues to include on the APR template the link to the OneDrive/SharePoint site. Administrative colleagues to populate the shared folder with the evidence documents using the following naming conventions:

• Please note for ALL file names there should be no spaces and the first letter of each word should be capitalised for ease of reading.
• Use the prefix of 01_CompletedAPR for the completed template for the programme once signed off by both PD and student representative.
• Use the prefix of 02_ClusterList for the file containing the list of programmes/pathways being included in the APR.
• Use the prefix of 03_ExternalExaminerReport for each external examiner report and add the external examiner’s initials at the end of the file name. For example, if you have external examiners Joe Bloggs and Ann Other, you should have 2 reports named:
• 03_ExternalExaminerReportJB
• 03_ExternalExaminerReportAO.
• Use the title 04_TeamResponseToExternalExaminerReport for the team’s response to the external examiner.
• Use the prefix of 05_ModuleEvaluationSummary for each module evaluation and add the module code at the end of the file name. For example, if you are including modules with codes MAE7001 and MAE7035, you should have 2 reports named:
• 05_ModuleEvaluationSummaryMAE7001
• 05_ModuleEvaluationSummaryMAE7035.
• Use the prefix of 06_ProgrammeEvaluationSummary for the summary of the programme evaluation and add the programme code at the end of the file name. If there are several pathways included in the APR, you may have separate evaluations. Append the pathway/programme codes to the end of the file name. For example, if you have two pathways, coded 10359 and 10771, you should have 2 reports named:
• 06_ProgrammeEvaluationSummary10359
• 06_ProgrammeEvaluationSummary10771

If you are including other programme type surveys such as NSS and PTES in your evidence body, add them to this section with the 06_ prefix, e.g.:

• 06_NSSSummary
• 06_PTESSummary.

• Use the prefix of 07_ProgrammeCommitteeMinutes for each set of programme committee minutes you are including. Append the dates of the meetings to the end of the file names. For example, if you had 3 meetings, dated 181016, 250317 and 240517, you should have 3 sets of minutes named:
• 07_ProgrammeCommitteeMinutes181016
• 07_ProgrammeCommitteeMinutes250317
• 07_ProgrammeCommitteeMinutes240517.
• If you have undergone an elective or period review, please include the response from the review panel. Use the filename:
• 08_ProgrammeReviewDocumentation.
• If your programme is professionally endorsed and you have a report from your PSRB, use the filename:
• 09_PSRBDocumentation.
• Each School should have a record of all external staff development undertaken, along with the staff member’s evaluation of the training. Members of academic staff should send details to the School Administrator of all internal staff development undertaken during the previous academic year. The details should be brief and recorded under the headings: Surname, Forename, Activity, Delivered by and Evaluation. The School Administrator should report the programme team’s staff development in a file named:
• 10_SummaryOfStaffDevelopment.
• If the programme team has undertaken any external engagement, for example, with employers, a summary of the activity should be recorded as:
• 11_ExternalEngagement.

If there are additional files deemed appropriate to add as evidence, then the numbering and naming conventions should be followed. For example, 12_AdditionalEvidence.

Your final folder in the SharePoint site should look something like:

Summary of Measures of Success

This section will be completed by administrative colleagues and PD. Several of the Measures of Success will have a Red or Green rating available in QlikView. This will differ for undergraduate and postgraduate programmes, so the guidance is a little different for each. Management Information Systems (MIS) colleagues have provided information on the QlikView application data and this is available at the end of this document. Loosely, most of the ratings are based on the 16/17 Measures of Success stipulated in the new Corporate Strategy. For Undergraduate programmes, these Measures of Success are: Retention (based on the HESA Performance Indicator tracking students from Year 1 to Year 2), Award Outcomes (% Good Honours), Career Destinations (DHLE data: Graduate Employment or Further Study), Student Opinion (NSS: Overall Satisfaction).

The rating for the external examiners’ reports, for both UG and PG programmes, will be provided by ASQU who will have reviewed the External Examiners’ reports and given them a Red/Amber/Green rating. (Details of the criteria for this are available at the end of this document).

Undergraduate Measures of Success

Administrative colleagues will be able to populate the Recruitment, Retention (i.e. Student Progression), Award outcomes, Career Destinations and Student Opinion Measures of Success. Administrative colleagues should place an X in the Red/Amber/Green area to reflect the information available in QlikView relating to the below categories:

• Green: Meets 2022/23 Measure of Success
• Amber: Meets 2016/17 Measure of Success but not 2022/23
• Red: Fails to meet 2016/17 Measure of Success

For example, with the following information available in QlikView for Student Progression (Retention), the administrative colleague would RAG code Retention Green:

With respect to the TEF metrics, PDs should select the TEF Metrics tab in QlikView and make a judgement based on the graphical information available for NSS Teaching and NSS Assessment and Feedback. They should then place an X in the Red/Amber/Green area to reflect their judgement.

Postgraduate Measures of Success

For postgraduate programmes the only measure of success RAG rated in QlikView is Career Destinations (DHLE data: Graduate Employment or Further Study). Administrative colleagues should place an X in the Red/Amber/Green area to reflect the information available in QlikView relating to this category.

To remain consistent with last year, the % of Distinctions and Merits is also presented. With the exception of the External Examiner category (where ASQU will provide the RAG rating), PDs should RAG the remaining measures of success based on their judgement about the information presented in QlikView and the evidence gathered in support of the APR and available in the SharePoint site.

Signatures and Dates

Administrative colleagues to add the date that the APR was considered at Programme Committee and at Learning and Teaching Committee.

Programme director needs to sign and date the form and ensure that there is student representation present at the programme committee meeting that considers the APR.

Section 1: Previous Action Plan and Evaluation and Impact of last year’s actions

The evaluation of last year’s APR actions needs to be completed at this point.

Administrative colleagues will add the numbers, issues and sources from the previous year’s APR.

Number: Each issue should be numbered sequentially, prefaced by the year the action plan was written, in the format: 16.01; 16.02 etc. Administrative colleagues should put the numbers in place.

Issue: A brief description of the issue should be provided, copied from the previous year’s APR.

Source: The source(s) of the issue should be identified, e.g. external examiner feedback.

The Evaluation and Impact of the previous year’s action plan should be completed by the PD.

Evaluation and Impact: PDs need to indicate whether the issue has been resolved by deleting YES/NO as appropriate. PDs, please note the phrase ‘impact’ in the final column. Although impact on student experience has always been implicit in the evaluation of actions taken, it has been included to emphasise that it is a primary focus. Please also identify the evidence base, e.g. external examiner report, improved module evaluations, programme statistics.

If, following PD’s evaluation, an issue has not been resolved effectively, it should be carried over to this year’s action plan. Administrative colleagues will carry it forward to the START of the new action plan in Section 9, ensuring that the number remains the same.

Section 2: Reflection on last year’s action plan

PDs need to reflect briefly on last year’s action plan, indicating any successes and noting why certain actions are being carried forward. Please try to limit the reflection to 250 words.

Section 3: Response to Outcomes from Periodic/Elective Review

If the programme underwent an elective/periodic review then the PD needs to reflect briefly on the event, in particular the recommendations from the panel. The panel’s final report should also be available in the OneDrive/SharePoint site discussed earlier. Please try to limit this to 500 words.

Section 4: Reflection on Modifications to Programme

If the programme underwent modifications then the PD needs to note them and reflect briefly on them. Please try to limit this to 500 words.

Section 5: Reflection on PSRB activity

If the programme is professionally accredited and there have been events during the academic year, the PD needs to reflect briefly on these events. Any supporting PSRB documentation should also be available in the SharePoint site discussed earlier. Please try to limit this to 500 words.

Section 6: Data Population

This section is populated by administrative staff using the data available in QlikView. This year, the data will all be available in Section 6 of the APR and the PD will comment in Section 7. Next year, we will revisit this format as some colleagues have queried keeping data presentation in a separate section to reflection. Where programmes are clustered together, the clustered selection should be made to create the graphs; otherwise, the graphs for a programme with several pathways will require several pages of the APR. However, if there are particular weaknesses in individual pathways, this should be highlighted.

The QlikView APR application is available and accessible to all PDs and designated School administrative colleagues. Those who do not have access to the data can email , copied to their DDLT, to request access. The application has undergone redevelopment since last year in order to include further data sets and simplify the aesthetics. Greater emphasis has been put on trends over academic years and comparisons against school and university populations.

Additional data for this year include:

• Survey data: UKES; SSS; PTES; PRES
• Franchise data: module completion rates

Data removed for this year include:

• Franchise data that are not relevant or reliable: disability; ethnicity.

Section 7: Data Reflection & Commentary

The PD reflects on the data included in the previous section. PDs encouraged to reflect on the three year trends in programme data. There is a limit of one page (Please try to limit this to 500 words).

Section 8: SWOT analysis

The PD identifies briefly the key strengths, weaknesses, opportunities and threats to the programme. There is a limit of one A4 page to include all components (bullet points are preferred). It is important to base these points on the evidence base included in evidence folder. Areas to consider (this is not an exhaustive list) include:

• Anything from the data set;
• Curriculum currency;
• Student experience;
• Student voice;
• On-going actions;
• Alignment to the institutional strategic plan – key theme for 2016-17 is Retention.

Strengths / Weaknesses
Opportunities / Threats

Section 9: Action Plan Going Forward

Number: Each issue should be numbered sequentially, prefaced by the year the action plan is written, in the format: 17.01; 17.02 etc. Administrative colleagues should put the numbers in place. Remember to start the list by including any items that need to be carried forward from previous APR action lists; the original numbering of the entry should be used so that it is clear the issue was raised in a previous year. Actions should be listed in order of priority, and be decided by the Programme Team, at Programme Committee, with the highest priority action being recorded first.

The Measures of Success associated with the issue should be identified by the PD (Recruitment/Student Progression (Retention)/Award Outcomes/Career Destinations/External Examiners/Student Opinion/Staff Opinion/PSRB/NSS Teaching/NSS Assessment and Feedback).

The remainder of the action plan should be completed by the PD.

Issue: A brief description of the issue should be provided.

Source: The source(s) of the issue should be identified, e.g. external examiner feedback.

Actions taken/planned: Ensure that action(s) already taken and those planned are clearly defined and are capable of addressing the issue identified.

By Whom: An indication of those responsible for undertaking the specified action(s) should be provided, e.g. Programme Director; Deputy/Assistant Dean Learning and Teaching. Please do not name individuals. You will need to discuss with your DDLT when assigning actions outside of the programme team.

Timescale: A realistic timescale for completion of each action should be provided.

Measurement of success: Indicate how the team will recognise/measure that an issue has been resolved (e.g. increased recruitment, fewer unfair practice cases).

MIS - Origins of the APR data

Key Performance Indicators

Recruitment

For the chosen programme or cluster of programmes, this is the number of new students enrolled in 16/17.

Undergraduate Degree student numbers do not include students who have progressed from the Foundation Leading to… programmes.

Student Progression

This replicates (within the limits of the current Qlikview application) the Higher Education Statistic Agency (HESA) Performance Indicator which measures the proportion of first year 15/16 Undergraduate Degree students who re-enrol in 16/17. Students who withdraw before 1st December in 15/16 are not included. Students who withdraw after 1st December in 15/16 but gain an exit award are classed as ‘retained’.

Award Outcomes

The proportion of students who graduated in 16/17 with a First Class or Second Class Upper degree.

Career Destinations

The results of the 15/16 Destination of Leavers from Higher Education (DLHE) survey, which questions students who left in the 15/16 Academic Year.

Student Opinion

The results of the 16/17 National Student Survey (NSS), which questions students who were in their penultimate year of study in 15/16.

Survey Data

UKES

The United Kingdom Engagement Survey (UKES) is run by the Higher Education Academy and questions undergraduate students in their first year of study in 15/16 (threshold or 5 or more students).

SSS

The Student Satisfaction Survey (SSS) is a Cardiff Metropolitan Survey and questions undergraduate students in their second year of study in 15/16 and third year top-up students (threshold of 10 or more students).

PTES

The Postgraduate Taught Experience Survey (PTES) is run by the Higher Education Academy and questions postgraduate students (threshold of 5 or more students).