November 2017 Agenda Item 08 Addendum - Meeting Agendas (CA State Board of Education)

California Department of Education
Executive Office
SBE-004 (REV.01/2011) / addendum-nov17item08
ITEM ADDENDUM
Date: / November 6, 2017
TO: / MEMBERS, State Board of Education
FROM: / TOM TORLAKSON, State Superintendent of Public Instruction
SUBJECT: / Item 08–English Language Proficiency Assessments for California: Approve the Operational Summative Assessment Threshold Scores and Composite Weights for the English Language Proficiency Assessments for California and Approve the Local Educational Agency Apportionment Rates.

Summary of Key Issues

This item addendum adds data and information related to the standard setting process for the English Language Proficiency Assessments for California (ELPAC). The standard setting process was detailed in an Information Memorandum provided to the State Board of Education (SBE) in October 2017 and can be found at The data used in the standard setting process came from an administration of the ELPAC field test to 41,763 English learners and 5,226 English only students in schools selected to represent the diversity of the state of California. Assuming the SBE takes action on the proposed threshold scores, they should be considered preliminary threshold scores until after the validation study that is scheduled to be conducted in 2018-19.

(Note that Attachment 4 was not previously listed in Item 8 as an attachment.)

Attachment(s)

Attachment 1: Educator Panel Proposed Threshold and Composite Weight Recommendations will be provided as an Item Addendum (2 pages)

Attachment 2: State Superintendent of Public Instruction’s Proposed Threshold and Composite Weight Recommendations (2 pages)

Attachment 3: Impact Data for the Composite Weight Recommendations (2 pages)

Attachment 4: Impact Data for the Threshold Score Determination (15 pages)

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 1

Page 1 of 2

Educator Panel Proposed Threshold and Composite Weight Recommendations

Overall Score Kindergarten through Grade Twelve

Table 1. Standard-Setting Panel’s Judgments for the Thresholds for the Performance Levels on the Summative English Language Proficiency Assessments for California (ELPAC), Overall Score

Oral /Written
Weight / Grade / Level 1
% of Students / Level 1
% at or above / Level 2
% of Students / Level 2 Standard-Setting Scale Threshold Score / Level 2
% at or above / Level 3
% of Students / Level 3 Standard-Setting Scale Threshold Score / Level 3
% at or above / Level 4
% of Students / Level 4 Standard-Setting Scale Threshold Score / Level 4
% at or above
90/10* / K / 10.6 / 100 / 20.4 / 338 / 89.4 / 35.8 / 380 / 68.9 / 33.2 / 428 / 33.2
70/30* / 1 / 9.1 / 100 / 14.0 / 381 / 90.9 / 21.3 / 411 / 76.9 / 55.5 / 441 / 55.5
50/50 / 2 / 5.1 / 100 / 10.2 / 389 / 94.9 / 33.6 / 424 / 84.7 / 51.1 / 475 / 51.1
50/50 / 3–5 / 8.1 / 100 / 21.4 / 441 / 91.9 / 52.8 / 490 / 70.5 / 17.7 / 569 / 17.7
50/50 / 6–8 / 6.5 / 100 / 24.6 / 451 / 93.5 / 40.7 / 516 / 68.9 / 28.3 / 577 / 28.3
50/50 / 9–10 / 16.7 / 100 / 25.6 / 484 / 83.3 / 31.9 / 544 / 57.8 / 25.9 / 607 / 25.9
50/50 / 11–12 / 13.4 / 100 / 24.3 / 486 / 86.6 / 36.4 / 547 / 62.2 / 25.8 / 618 / 25.8

*In preparation for standard setting calculations, Educational Testing Service set the weights at 90/10 for Kindergarten and 70/30 for grade one rather than have 3 different sets of data for panelists to review. At the end of standard setting, the three different options were reviewed and educator selected the 70/30 weighting option for Kindergarten, and 50/50 weighting option for grade one. Additionally, the educator panel was in support of the 50/50 weights for grades two through twelve.

Key
% of Students / Estimated percent of students statewide who would be placed at this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.
Standard-Setting Scale Threshold Score / Minimum standard-setting scale score needed to achieve this performance level on the basis of the results of the 2016–17 field test administration. Threshold scores were generated solely for the standard-setting process. Reporting scales will be developed to report scores on the Student Score Report and public reporting. The report scale will be a 4-digit vertical scale that does not overlap with the scale used for Smarter Balanced Assessments.
% at or above / Estimated percent of students statewide who would be at and above this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.

Oral Composite Score Kindergarten through Grade Twelve

Table 2. Standard-Setting Panel’s Judgments for the Thresholds for the Performance Levels on the Summative ELPAC, Oral Composite Score

Grade/Grade Span (Weight) / Level 2 / Level 3 / Level 4
K (90) / 343 / 382 / 431
1 (70) / 379 / 410 / 437
2 (50) / 372 / 409 / 458
3–5 (50) / 413 / 459 / 531
6–8 (50) / 402 / 469 / 545
9–10 (50) / 430 / 497 / 555
11–12 (50) / 430 / 481 / 545

Written Composite Score Kindergarten through Grade Twelve

Table 3. Standard-Setting Panel’s Judgments for the Performance Levels on the Summative ELPAC, Written Composite Score

Grade/Grade Span (Weight) / Level 2 / Level 3 / Level 4
K (10) / 289 / 362 / 398
1 (30) / 387 / 414 / 450
2 (50) / 405 / 438 / 491
3–5 (50) / 469 / 520 / 607
6–8 (50) / 499 / 562 / 609
9–10 (50) / 537 / 591 / 659
11–12 (50) / 542 / 613 / 691

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 2

Page 1 of 2

State Superintendent of Public Instruction’s Proposed Threshold and Composite Weight Recommendations

Overall Score Kindergarten through Grade Twelve

Table 1. State Superintendent of Public Instruction (SSPI) Recommendations for the Proposed Preliminary Thresholds for Performance Levels on the Summative English Language Proficiency Assessments for California (ELPAC), Overall Score

Oral/ Written
Weight / Grade / Level 1
% of Students / Level 1
% at or above / Level 2
% of Students / Level 2
Standard-Setting Scale Threshold Score / Level 2
% at or above / Level 3
% of Students / Level 3
Standard-Setting Scale Threshold Score / Level 3
% at or above / Level 4
% of Students / Level 4
Standard-Setting Scale Threshold Score / Level 4
% at or above
70/30 / K / 9.6 / 100 / 25.4 / 327 / 90.4 / 35.0 / 376 / 65.0 / 30.1 / 421 / 30.1
50/50 / 1 / 9.0 / 100 / 14.6 / 378 / 91.0 / 32.9 / 409 / 76.3 / 43.5 / 453 / 43.5
50/50 / 2 / 4.6 / 100 / 10.7 / 386 / 95.4 / 40.9 / 424 / 84.7 / 43.8 / 484 / 43.8
50/50 / 3–5 / 8.1 / 100 / 19.0 / 441 / 91.9 / 49.6 / 486 / 72.9 / 23.3 / 556 / 23.3
50/50 / 6–8 / 8.8 / 100 / 22.3 / 462 / 91.2 / 40.7 / 516 / 68.9 / 28.3 / 577 / 28.3
50/50 / 9–10 / 15.9 / 100 / 26.4 / 481 / 84.1 / 35.5 / 544 / 57.8 / 22.3 / 616 / 22.3
50/50 / 11–12 / 12.5 / 100 / 23.2 / 483 / 87.5 / 41.4 / 543 / 64.3 / 22.9 / 626 / 22.9
Key
% of Students / Estimated percent of students statewide who would be placed at this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.
Standard-Setting Scale Threshold Score / Minimum standard-setting scale score needed to achieve this performance level on the basis of the results of the 2016–17 field test administration. Threshold scores were generated solely for the standard-setting process. Reporting scales will be developed to report scores on the Student Score Report and public reporting. The report scale will be a 4-digit vertical scale that does not overlap with the scale used for Smarter Balanced Assessments.
% at or above / Estimated percent of students statewide who would be at and above this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.
Oral Composite Score Kindergarten through Grade Twelve
Table 2. SSPI’s Recommendations for the Proposed Preliminary Thresholds for Performance Levels on the Summative ELPAC, Oral Composite Score
Grade/Grade Span (Weight) / Level 2 / Level 3 / Level 4
K (70) / 343 / 382 / 431
1 (50) / 374 / 404 / 446
2 (50) / 372 / 409 / 467
3–5 (50) / 413 / 452 / 518
6–8 (50) / 408 / 469 / 545
9–10 (50) / 424 / 497 / 572
11–12 (50) / 423 / 473 / 560

Written Composite Score Kindergarten through Grade Twelve

Table 3. SSPI’s Recommendations for the Proposed Preliminary Thresholds for Performance Levels on the ELPAC, Written Composite Score
Grade/Grade Span (Weight) / Level 2 / Level 3 / Level 4
K (30) / 289 / 362 / 398
1 (50) / 382 / 414 / 459
2 (50) / 400 / 438 / 501
3–5 (50) / 469 / 520 / 594
6–8 (50/50) / 515 / 562 / 609
9–10 (50/50) / 537 / 591 / 659
11–12 (50/50) / 542 / 613 / 691

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 3

Page 1 of 2

Impact Data for the Composite Weight Recommendations

This attachment provides various impact data for the recommendation outlined in this item.

Table 1 displays impact data for kindergarten and grade one weighting options on the Summative English Language Proficiency Assessments for California based on State Superintendent of Public Instruction’s Recommended Threshold Scores. Note that the recommendation of Option 2 was based on educator panel feedback received at the ELPAC standard setting meeting.

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 3

Page 1 of 2

Table 1. Impact Data for Kindergarten and Grade 1 Weighting Options on the Summative English Language Proficiency Assessments for California, based on State Superintendent of Public Instruction’s (SSPI) Recommended Threshold Scores

Grade / Oral / Written Weight / Leve 1 % of Students / Level 1 % at or above / Level 2 % of Students / Level 2 Standard-Setting Scale Threshold Score / Level 2 % at or above / Level 3 % of Students / Level 3 Standard-Setting Scale Threshold Score / Level 3 % at or above / Level 4 % of Students / Level 4 Standard-Setting Scale Threshold Score / Level 4 % at or above
Option 1
K / 90/10 / 10.6 / 100 / 20.4 / 338 / 89.4 / 35.8 / 380 / 68.9 / 33.2 / 428 / 33.2
1 / 70/30 / 7.6 / 100 / 13.2 / 376 / 92.4 / 31.6 / 407 / 79.2 / 47.6 / 450 / 47.6
Option 2*
K / 70/30 / 9.6 / 100 / 25.4 / 327 / 90.4 / 35.0 / 376 / 65.0 / 30.1 / 421 / 30.1
1 / 50/50 / 9.0 / 100 / 14.6 / 378 / 91.0 / 32.9 / 409 / 76.3 / 43.5 / 453 / 43.5
Option 3
K / 50/50 / 9.6 / 100 / 31.0 / 316 / 90.4 / 31.0 / 372 / 59.4 / 28.4 / 415 / 28.4
1 / 50/50 / 9.0 / 100 / 14.6 / 378 / 91.0 / 32.9 / 409 / 76.3 / 43.5 / 453 / 43.5
Key
% of Students / Estimated percent of students statewide who would be placed at this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.
Standard-Setting Scale Threshold Score / Minimum standard-setting scale score needed to achieve this performance level on the basis of the results of the 2016–17 field test administration. Threshold scores were generated solely for the standard-setting process. Reporting scales will be developed to report scores on the Student Score Report and public reporting.
% at or above / Estimated percent of students statewide who would be at and above this performance level on the basis of the results of the 2016–17 field test administration. Each level percentages may not sum to 100 due to rounding.

* SSPI Recommended

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 4

Page 1 of 15

Impact Data for the Threshold Score Determination

This attachment provides various impact data for the recommendation outlined in this item.

Tables 1 through 7 display student performance on 2017 California Assessment of Student Performance and Progress (CAASPP) English language arts/literacy (ELA) assessment by English Language Proficiency Assessments for California (ELPAC) performance levels 1 through 4, based on educator panel judgments, including performance of English only students (EOS) that participated in the field test. The tables illustrate how the ELPAC performance increase corresponds to increases in ELA performance on the CAASPP. The tables also allow for a comparison of English only student performance with that of the level 4 ELPAC student performance.

Tables 8 through 14 display student performance on 2017 the CAASPP ELA assessment by ELPAC performance levels 1 through 4, based on SSPI recommendations, including performance of EOS that participated in the field test. These tables match the preceding tablesthe only difference being that tables 1 through 7 display data based on the panel judgements, while tables 8 through 14 are based on the SSPI recommendations.

12/17/2018 9:08 AM

addendum-nov17item08

Attachment 4

Page 1 of 15

Table 1. Grade Three Impact Data Based on Panel Judgments

Table 2. Grade Four Impact Data Based on Panel Judgments

Table 3. Grade Five Impact Data Based on Panel Judgments

Table 4. Grade Six Impact Data Based on Panel Judgments

Table 5. Grade Seven Impact Data Based on Panel Judgments

Table 6. Grade Eight Impact Data Based on Panel Judgments

Table 7. Grade Eleven Impact Data Based on Panel Judgments

Table 8. Grade Three Impact Data Based on SSPI Recommendations

Table 9. Grade Four Impact Data Based on SSPIRecommendations

Table 10. Grade Five Impact Data Based on SSPI Recommendations

Table 11. Grade Six Impact Data Based on SSPI Recommendations

Table 12. Grade Seven Impact Data Based on SSPI Recommendations

Table 13. Grade Eight Impact Data Based on SSPI Recommendations

Table 14. Grade Eleven Impact Data Based on SSPI Recommendations

12/17/2018 9:08 AM