Attainment Training Materials

An approach to judging attainment

Training materials

Age group: 0–19

Published:January 2011

Reference no: 090190

Introduction

1.The evaluation schedule requires inspectorsto consider a wide range of data, initially from RAISEonline.[1]There is a particular focus on the statistical significance of test and examination results over three years. As well as headline data, inspectors must also take account of the attainment of different groups and in key subjects. One way of managing this process is explained below.

2.A series of examples show the steps which inspectors might follow to reach a judgment on attainment. The examples are based on analyses of RAISEonline data for actual schools. Each sets out the pattern of attainment, hypotheses arising from analysis of those patterns and a series of comments and further questions for consideration. It is important to remember that while RAISEonline gives an initial indication of the general standard of attainment, inspectors are required to exercise profession judgement and consider all the available evidence before reaching a final view about attainment.

3.Each example demonstrates the same basic method which ensures that all the data is considered systematically. Inspectors should:

consider the pattern of attainment over the last three years as revealed by significance tests on the main points scores and grade or level indicators and make an initial hypothesis based on this analysis

consider any significance data on the attainment of groups in the most recent year referred to by RAISEonline and modify the original hypothesis and/or consider what further data, if any, it would be helpful to see

consider any data in RAISEonline about attainment in key subjects and further refine the original hypothesis and/or identify any other data required.

4.In using these materials inspectors may wish to:

read the examples to familiarise themselves with the approach and consider the questions in the order they appear

apply the approach to schools with which they are familiar, perhaps from a recent inspection, using blank templates

apply the approach using unknown schools drawn from RAISEonline, using blank templates.

5.The examples illustrate the approach in secondary schools, primary schools and an infant school, with some thoughts about the implications for first and middle schools. The examples are followed by the answers to some frequently asked questions.

Attainment summary: secondary school (1)

Table 1: Attainment indicators at the end of Key Stage 4 over the last three years

Key to abbreviations used in table 1

CAPS:Capped average points score
APS:Average points score
5ACEM:Percentage of pupils passing five GCSEs at C or above, including English and mathematics
5AC:Percentage of pupils passing five GCSEs at C or above
5AG:Percentage of pupils passing five GCSEs at G or above
APSMA: Average points score for mathematics
APSEN:Average points score for English

Table 2: Attainment indicators (average and capped average points scores) for groups at the end of Key Stage 4 as shown in RAISE

Key to abbreviations used in table 2

FSM:Free school meals

EAL:English as an additional language

Table 3: Attainment indicators for key subjects at GCSE (subjects with large numbers being entered)

Key to abbreviations used in table 3

MA:Mathematics
SC:Science
DT:Design and technology
EL:Electronics/ systems and control
EN:English/ English literature

Initial analysis and hypothesis about attainment

Over the last three years, attainment indicators have generally not been significantly different from average, initially suggesting a judgement of ‘grade 3, average’ for attainment. The school is above the National Challenge floor target.

Attainment in English as shown by the average points score for English has been significantly below average for three years which may suggest a judgement of ‘grade 4, low’.

For sizeable groups as shown by RAISEonline, there are none significantly below average. There are only very small numbers of pupils from minority ethnic backgrounds and so no significance tests were carried out.

Design technology, a key subject with almost the whole cohort entered was sig- in 2009 for A*−C pass rate.

Taking into account the previous points, an initial hypothesis might be that attainment is low because English average points score, a key subject has been sig- for the last three years. Note that an attainment grade of ‘4’ is being considered even though the school has met the National Challenge floor target.

Issues for discussion with the school and comparison with the school’s evaluation in its self-evaluation form

Does the school present any data which might change the initial hypothesis? For example, an analysis of current attainment in English or information on the attainment of any groups identified by the school but not shown in RAISE online?

Inspectors would need to assess current attainment, particularly in English and look at attainment of key subjects in the last three years.

Commentary and further questions

In this school, inspection evidence based on analysis of the school’s assessment data for English and observation of the current standard of work suggested that attainment in English was now above, but not significantly different from, average. There was no pattern of key subjects being significantly below average.

Would the broad picture of attainment as found in RAISE and from the inspection evidence as set out above, result in a grade 3 or grade 4 for attainment?

To what extent would inspectors need to take account of the date of publication of the RAISEonline data?

Attainment summary: secondary school (2)

Table 1: Attainment indicators at the end of Key Stage 4 over the last three years

Key to abbreviations used in table 1

CAPS:Capped average points score

APS:Average points score

5ACEM:Percentage of pupils passing five GCSEs at C or above, including English and mathematics

5AC:Percentage of pupils passing five GCSEs at C or above

5AG:Percentage of pupils passing five GCSEs at G or above

APSMA: Average points score for mathematics

APSEN:Average points score for English

Table 2: Attainment indicators (average and capped average points scores) for groups at the end of Key Stage 4 as shown in RAISE

Key to abbreviations used in table 2

FSM:Free school meals

EAL:English as an additional language

Table 3: Attainment indicators for key subjects at GCSE (subjects with large numbers being entered)

Key to abbreviations used in table 3

EL:Electronics/ systems and control
GG:Geography

MA:Mathematics

RE:Religious education

EN:English/ English literature

Other SC:Other science

Initial analysis and hypothesis about attainment

Well over two thirds (a large majority) of indicators over the last three years have been significantly above average, suggesting the possibility of ‘grade 1, high’ attainment.

Some groups are not sig+ for 2009 so this may suggest a grade 2 rather than a grade 1 for attainment and one group is below average and not significantly different from average. However, RAISE shows that none of these groups consisted of more than three pupils. Given that the cohort was over 200, they were not deemed to be sizeable groups.

Key subjects are mainly sig+ for A−C and A*−A.

Taking the previous points into account the initial hypothesis is that attainment is a grade 1.

Issues for discussion with the school and comparison with the schools’ evaluation in its self-evaluation form

Is the pattern of high attainment being maintained by pupils currently in the school?

Commentary and further questions

The school was able to demonstrate that pupils on the gifted and talented register also attained highly.

The school’s records showed that in the last three years all of the key subjects and others had A*−A as sig+, including English in 2007 and 2008.

Inspectors found no evidence that standards of attainment were slipping in the current year.

Would the broad picture of attainment as found in RAISE and from the inspection result in a grade 1 or grade 2 for attainment?

Attainment summary: secondary school (3)

Table 1: Attainment indicators at the end of Key Stage 4 over the last three years

*Above National Challenge floor targets

Key for abbreviations used in table 1

CAPS:Capped average points score
APS:Average points score
5ACEM:Percentage of pupils passing five GCSEs at C or above, including English andmathematics
5AC:Percentage of pupils passing five GCSEs at C or above
5AG:Percentage of pupils passing five GCSEs at G or above
APSMA: Average points score for mathematics
APSEN:Average points score for English

Table 2: Attainment indicators (average and capped average points scores) for groups at the end of Key Stage 4 as shown in RAISE

Key for abbreviations used in table 2

FSM:Free school meals

SEN:Special educational needs and/or disabilities

Table 3: Attainment indicators for key subjects at GCSE (subjects with large numbers being entered)

Key to abbreviations used in table 3

DT:Design and technology

SC:Science

EN:English/ English literature

MA:Mathematics

Initial analysis and hypothesis about attainment

Indicators over three years are generally not significantly below average.

Indicators for English and mathematics, key subjects, have been sig- for the last three years.

No groups were below average in 2009. All pupils were White British.

Despite some indicators being sig+, the indicators for English and mathematics over the last three years suggest a hypothesis that attainment is likely to be judged ‘grade 4, low’.

Issues for discussion with the school and comparison with the schools’ evaluation in its self-evaluation form

RAISE shows that pass rates for five A*−C with functional English and mathematics have been significantly above the average for the last three years.

Commentary and further questions

The attainment data is polarised, with some indicators sig+ and others sig-. Inspectors would need to reconcile the apparent contradictions, for example in the results for English and mathematics and the results for GCSE passes with functional English and mathematics.

What evidence would inspectors need to gather in order to establish whether the attainment grade was a three or a four?

Attainment summary: secondary school (4)

Table 1: Attainment indicators at the end of Key Stage 4 over the last three years

Key for abbreviations used in table 1

CAPS:Capped average points score
APS:Average points score
5ACEM:Percentage of pupils passing five GCSEs at C or above, including English and mathematics
5AC:Percentage of pupils passing five GCSEs at C or above
5AG:Percentage of pupils passing five GCSEs at G or above
APSMA: Average points score for mathematics
APSEN:Average points score for English

Table 2: Attainment indicators (average and capped average points scores) for groups at the end of Key Stage 4 as shown in RAISE

Key for abbreviations used in table 2

FSM:Free school meals

SEN:Special educational needs and/or disabilities

Table 3: Attainment indicators for key subjects at GCSE (subjects with large numbers being entered)

Key to abbreviations used in table 3

EN:English/ English literature
EL:Electronics/ systems and controls
GG:Geography
MA:Mathematics
SC:Science

Initial analysis and hypothesis about attainment

A large majority of indicators over three years are significantly above average, suggesting an initial hypothesis that the judgement for attainment may be ‘grade 1, high’.

In 2009, the attainment of groups varied, with pupils eligible for free school meals, who make up ten percent of the cohort, significantly below average for attainment.

The key subjects in 2009 were all sig+ at A*−A and A*−C.

Issues for discussion with the school and comparison with the schools’ evaluation in its self-evaluation form

Inspectors will need to gather further information on the attainment of different groups, in particular those pupils eligible for free school meals and check the attainment of key subjects in previous years and in the current year.

Commentary and further questions

The school did not provide any further information which changed the picture already gleaned from RAISE, other than to note that the pattern of attainment seen for groups and for key subjects was typical of the last two years and seemed similar for the current year.

Given this scenario, what is the likely judgement for attainment?

Attainment summary: primary school (1)

Table 1: Average points scores over three years for all subjects, English, mathematics and science at the end of Key Stage 2

Key for abbreviations used in table 1

MA:Mathematics

EN:English/ English literature

SC:Science

Table 2: Attainment thresholds at the end of Key Stage 2 for 2009

Key for abbreviations used in table 2

EN:English/ English literature
MA:Mathematics
SC:Science

Initial analysis and hypothesis about the judgement on attainment

The general pattern ofattainment indicators are mostly significantly below average for the last three years, including two key subjects.

The attainment thresholds for 2009 are generally sig-.

The data strongly suggests a judgement of ‘grade 4, low’ for attainment.

Issues for discussion with the school and comparison with the schools’ evaluation in its self-evaluation form

RAISE does not provide any significance data on attainment for different minority ethnic groups. Inspectors will need to note the level of attainment of different groups. In this case, however, most pupils belonged to a minority ethnic group and inspectors may reasonably apply the overall figures to the main minority ethnic group.

Inspectors may want to check on attainment thresholds for previous years.

Commentary and further questions

Is there any further information the school could provide, either in its self-evaluation form or that inspectors could gather on inspection which could deflect the judgement of ‘grade 4, low’ for attainment?

Attainment summary: primary school (2)

Table 1: Average points scores over three years for all subjects and English, mathematics and science at the end of Key Stage 2

Key for abbreviations used in table 1