Using RAISEonline to improve our schools

Workshop delivery notes: Secondary

Contents

Purpose

Aims

The audience

Background resources

Before the workshop

Workshop specifications

Workshop facilitators

Workshop delegates

Workshop length

Technical requirements

Facilitator resources

Workshop ‘learning journey’ overview

Outline of workshop content

Setting the scene

Extending our understanding of RAISEonline

Using data for school improvement

Working with your senior leadership team

Key messages – using RAISEonline effectively

Participant workbook: analysis answers

References

Purpose

The purpose of this training is to develop the capacity of governors to challenge and support their school’s senior leadership in their efforts to bring about school improvement;to raise standards of achievement in their pupils; and to provide them with opportunities to test out their understanding of the use of data and to hone their skills in interpreting it.

Aims

The aim of the training is to develop governors’:

•knowledge of the data available to them from RAISEonline

•understanding key terms and concepts used in RAISEonline

•skills in the interpretation of key data

•awareness of how RAISEonline can be used to ask challenging questions about the performance of their school

•understanding of the contribution of RAISEonline to school self-evaluation and school improvement

•understanding of the proper use of RAISEonline by governors and the protocols that they should follow

The audience

It is highly likely that the audience will include governors with a widerange of experience in the use and interpretation of data. It is very easy for individuals who are not comfortable with numbers to feel de-skilled and switched off if there is a heavy focus on statistical methodology and terminology.

It is recommended that a balance is struck between developing understanding of essential terminology and desirable terminology. A list is suggested below.

Essential terminology:

•attainment

•significance

•trends

•average

•sample size

•value added

•expected progress

Desirable terminology might include:

•confidence intervals

•relative performance indicator

It is suggested that more advanced concepts such as ‘validity’ and ‘reliability’ are not covered.

Statisticians in the audience?

It is probable that some groups will include participants with a background in statistics. On occasions these participants may ask questions about the methodology and statistical techniques used.

Where this is the case, avoid allowing the session to be hijacked into a discussion or debate of the methodology used, but direct interested participants to the RAISEonline website which includes a methodology library.Where participants question the methodology used, accept that there are different opinions, but point out that this is the approach currently employed, and that external groups, Ofsted in particular, regard the data as valid and reliable and will use it to inform their judgements on individual schools.

Background resources

RAISEonline website

The RAISEonline website: this has a variety of resources, a number of which are referred to below:

•Training materials: includes anonymised RAISEonline reports

•Methodology library: provides a comprehensive breakdown of the methodology used for each data set. Facilitators should ensure that they are familiar with key concepts, definitions and measures – there is no need to know the formulas. It is a good idea to have a copy at the event for reference.

•Progression guidance: provides guidance on measuring the progress of children with special educational needs (SEN). This will be particularly valuable for audiences made up of governors of special schools.

•RAISEonline newsletter: RAISEonline continues to change, and a number of alterations are planned for 2012–13. The programme refers to the changes currently planned. Facilitators should read this newsletter and check the RAISEonline website regularly for updates.

•Key stages 2 to 4: Ebacc value added: the first few sections are helpful in summarising how the calculation is achieved.

•Defining low, middle and high attainers: this is a one-page explanation of the definitions used.

National Governors’ Association

•NGA, 2012, Knowing your school: RAISEonline for governors of secondary schools

•NGA, 2012, Knowing your school: RAISEonline for governors of primary schools

These publications provide excellent, concise overviews and include some useful exemplar material.

National College for Teaching and Learning

National College, 2012, Good governance

Before the workshop

The materials in this guide are designed to provide workshop providers with a flexible framework for the delivery of its key messages.In planning your workshop you should:

•ensure that you have read and understand the guidance notes for each slide

•access the various websites and materials referenced in the ‘Background resources’section to familiarise yourself with their general contents

•consider any supportyou will offerfollowingthe workshop and make this information available to delegates at the event

•familiarise yourself with the support available to governing bodies from:

— National Governors’ Association: refer to its website

— National Leaders of Governance (NLGs): refer to the NCTL and provide contact details for local NLGs

— Local authority governor service: check what is available from local authorities and provide relevant contact details

— NCTL Chairs of Governors’ Leadership Development Programme

— NCTL resources

—school improvement partner or equivalent

Workshop specifications

Workshop facilitators

Ideally, the workshop will be delivered by two presenters. Facilitators should have excellent facilitation skills, a good knowledge of how to interpret data and use itto inform school improvement, and be familiar with the needs of governors.

Providers should give serious consideration to including a local governor or headteacher as part of the day: they could work alongside the facilitators as an ‘expert witness’, illustrating what effective practice looks like, while the facilitators ensure that key messages are being addressed.

Workshop delegates

It is for the workshop organiser to determine the target group for their workshop. It is recommended that workshops are delivered to phase specific groups. is recommended that workshops have a limit of 24 delegates.

Workshop length

It is recommended that the workshop is scheduled to last three hours (including breaks) however providers may wish to increase its length if circumstances permit.

Technical requirements

The presenters willneed a digital projector and laptop. If the video clip is used facilitators will need audio facilities.

Facilitator resources

a)A high level outline of the workshop is provided below.

b)A set of presentation slides is provided for the workshop. Facilitator notes accompany each slide – the slides should be printed off using the ‘Print notes page’ function.The notes are not a script;they expand upon the material included on the slides, and on occasions make suggestions for activities.Two sets of presentation slides are provided: one for primary and one for secondary.

c)An anonymised 2013RAISEonline Summary Report is provide for participants to refer to.

d)A participant workbook contains activities for participants to complete plus appendices with background information.

Workshop ‘learning journey’ overview

Outline of workshop content

The workshop is divided into six sections, but these are not of equal length.Suggested timings are provided, but it is for facilitators to determine how much time to devote to each section.

Indicative
timing / Content / Resources
00:00 /

Setting the scene

Introduction
Welcome participants to the event.
Go through housekeeping in line with venue policy.
Introduce the purpose of the event and programme for the day.
Emphasisethe ‘mixed-experience’ of working with RAISEonline and the need for participants to support one another.
Using data to drive improvement
Use the slides to bring out:
•the importance attached by policy-makers to the effective use of data in bringing about school improvement (a two-minute video clip featuring Emma Knights, Chief Executive of the National Governors’ Association, discussing the importance of data is provided)
•Ofsted’s expectations – discussion activity
•why governors should use data
•how governors evaluate their school’s performance – discussion activity
•the uses of data and the concept of ‘useful data’; distinguish between merely collecting data and analysing data in order to make strategic plans
The (optional) video Emma Knights: Data for the governing body can be downloaded from or streamed via the College’s ‘Good Practice for Leaders’ library:

You will need to be a member of the College website to be able to access the video.
Use and abuse of data
Briefly point out that it is possible to draw lots of conclusions from data, but they may not all be valid. Emphasise the importance of understanding what a particular data set is telling us if we are to use the data responsibly.
What is RAISEonline?
Distribute the anonymised RAISEonline provided. These slides explain:
•purpose of RAISEonline
•use of RAISEonline by Ofsted
•National Governors’ Association guidance on access to RAISEonline
Getting started with RAISEonline
These slides explain some of the concepts that governors need to understand to assist them in interpreting the data provided in RAISEonline:
•significance
•trends
•sample size
•levels and points scores
•attainment and progress / PPTs 1–3
PPTs 4–8
Video:Emma Knights: Data for the governing body
PPTs 9–12
PPTs 13–19
PPTs 20 –29
00:50 /

Extending our understanding of RAISEonline

Note: This section forms the core of the workshop and it is essential that all the issues raised are covered.
Understanding RAISEonline
This session provides participants with an opportunity to explore the data available to them in RAISEonline.
At the start of the session, distribute copies of the RAISEonline analysis workbook.
Explain to participants that we will work through each of the sections of RAISEonline as follows:
•input by the facilitator explaining what information is provided in that section
•participants checking their understanding by working through the analysis activitiesin the workbook
•discussion and feedback
Detailed facilitator notes are provided on each of the slides.
Using the RAISEonline analysis workbook
It is suggested in the introduction to the workbook that participants work through the analyses on their own or in pairs, but facilitators may wish to vary this.
Five separate sets of analysis are provided: facilitators may decide not to use all of them. On occasions facilitators may wish to carry out the analysis with the whole group rather than in small groups.
There are a large number of tables; we have focused on those with a (although not all of these tables are referenced).
When participants have completed the 5 sets of analyses they should compete the ‘Synthesis’ activity which asks them to look at the big picture that is emerging from their detailed analyses. / PPTs 30-59
RAISEonline
analysis
workbook
02:25 /

Using data for school improvement

It is recommended that this session is delivered by a headteacher or governor from a local school who can illustrate how their school usesRAISEonline, and data in general, to secure improved outcomes for children.
Other sources of data
The first slide emphasises the importance of triangulating the evidence from RAISEonline with other pupil performance data that the school may have, before coming to judgements about a school and any improvement strategies that may be necessary.
The second slide reminds participants that focusing on the hard statistical data provided by RAISEonline can cause us to lose sight of the softer and less easily measured desirable outcomes of schooling – discussion activity.
Using data for school improvement
These slides remind participants that the primary purpose of collecting data is to bring about improvement: constantly weighing the pig doesnot make it heavier. / PPTs 60-64
02:40 /

Working with your senior leadership team

Introduce this session by explaining the importance of the:
•governing body agreeing how it will use RAISEonline to help them carry out its role of ‘challenge and support’ and the challenges of carrying out this role
•governing body and senior leadership team establishing agreed protocols for the use of the data available from RAISEonline
•governing body assuring themselves that the school is making full use of RAISEonline
Challenge and support
Use the slide to start a whole-group discussion about how governors might use RAISEonline to challenge and support the senior leadership of their school – discussion activity.
Establishing protocols
Use the slide to introduce the group activity: groups to draw up a flip chart outlining the:
•arrangements for governor access to RAISEonline and reporting arrangements by the senior leadership team to the governing body
•protocols to be followed by governors in their use of data from RAISEonline
Making full use of data
The slide provides a list of possible uses that schools can make of the data they generate or are given. Do participants’ schools use data for all the purposes suggested on the slide? / PPTs65-69
02:55 /

Key messages – using RAISEonline effectively

Use this session to bring the event to a close, summarising the key learning for the day and revisiting key points you wish to emphasise with the group.
Key messages
The slide is self-explanatory. Add other messages to reflect issues that have arisen during the event. Invite participants to share their key learning.
Support and advice
Remind governors of the support and advice available to them (some governing bodies may wish to obtain an independent analysis of their school’s RAISEonline), and in particular what is available through the National College. Provide them with details of any local sources.
Close the session
Thank participants for their attendance.
Distribute evaluation forms. / PPTs70-73
03:00 / Close

Participant workbook: analysis answers

Analysis 1: Basic Characteristics, Attendance & Exclusions – Tables 1.1.1, 1.1.2 and 2.1.1 (pages 7, 8, 11)

  1. What do you consider to be the most significant features of table 1.1.1? What impact might this information have upon the school’s long term planning?

In general the school’s basic characteristics place it around national averages except for % of pupils at ‘school action’ which is much higher than the national average.

The school’s leadership will need to consider any resource implications e.g. deployment of TAs and the shape of the curriculum e.g. support for ‘basics’, ensuring there is an engaging curriculum at KS 4, pupil groupings within year groups.

  1. Are there any differences in the composition of the different year groups that might impact on performance levels (table1.1.2)?

Ratio of boys to girls in Y8 and 9 – very different. Could this bring about a significant difference in the school’s overall performance when these year groups sit their GCSEs?

  1. How would you describe attendance at the school? Are absence levels above or below the national average (table 2.1.1)? Absence levels have been above the national average for the last 3 years; at the same time absence levels have fallen.
  2. Are the levels of exclusions above or below the national average (refer to 2012 data in table 2.1.1)? Permanent exclusions below; fixed term above.

Analysis 2: Prior Attainment– Table 3.1.1 (page 16)

  1. Which year group’s prior attainment is stastically significantly below the national average? None (no blue indicators)
  2. Which year group has the greatest proportion of higher ability children? Year 9
  3. Why do governors need to be aware of prior attainment data when evaluating pupil performance in KS4 examinations?Prior attainment is an indicator of future performance.
  4. What should governors and senior leaders guard against when considering prior attainment data? Low expectations viz that pupils with below average prior attainment will achieve below average results at the end of KS4.
  5. How might governors use this Prior Attainment data to inform the school’s strategic planning?If the PA data indicates that a particular year group is ‘very weak’, for example, the school may wish to consider deploying additional support to pupils.

Analysis 3a: Attainment at Key Stage 4: Key performance indicators – Tables 4.1.2, 4.1.4 (pages 17 and 18)

  1. What percentage of children achieved 5+ A*-C grades including English and maths? 43%
  2. Did the school achieve the national floor target in 2013?Yes, it is over 40%.
  3. Using the ‘Average capped point score’ measure was the school above or below the national averagein 2013 (refer to table 4.1.4)? Was the difference between the school’s performance and the nationally average statistically significant? How do we know? Slightly below the national average (-1.6 pts). Not statistically significant – no green or blue indicator.

Analysis 3b: Attainment at Key Stage 4: Individual subjects –Table 4.1.11 (page 22)

1.Test your understanding. Refer to the results for Additional Science in Table 4.1.11. What do the following indicate or mean:

a)‘58’ Number of pupils who sat the Additional Science exam

b)‘64.4’ % of pupils nationally who gained an A*-C grade

c)’62.1’ % of pupils at the school who gained an A*-C grade

d)Roughly what GCSE grade does a score of ’38.3’ equate to? Just below a grade C (grade C is 40 points)

2.Which subjects’ performance were statistically significantlyabove the national average on the ‘% A*-C’ measure? Which subjects’ performance were statistically significantlybelow the national average on the ‘% A*-C’ measure?

Above: English Literature (green); Below: Geography, Mathematics (blue)

3.In general does the school perform better on the ‘% A*-A’ measure or the ‘% A*-C’ measure? What is the basis for judgement?

Table suggests worse on ‘A*-A’ measure: 8 subjects are blue only 1 subject is green.

Analysis 3c: Attainment at Key Stage 4: Ebacc measures - Table 4.1.15, 4.1.17 (pages 29, & 32)

  1. In which of the 5 Ebacc subject areas was the school’s performance not statistically significantly below the national average for ‘all pupils’? (Note column 1 ‘English Baccalaureate’ indicates the % of pupils who achieved an A*-C grade in each of the 5 Ebacc subject areas.) Languages
  2. Analyse the performance of ‘SEN pupils without a statement’.

How many pupils are in this category? 25

How many of them sat a mathematics examination? 25