Ten Steps to Writing a Successful Program Report

Audrey Church, Longwood University, Farmville, VA

ALA Midwinter NCATE Program Reviewer Training, January 7, 2011

Updated June 8, 2011

  1. Become thoroughly familiar with the standards.
  2. 2003 ALA/AASL Standards for Initial Programs for School Library Media Specialist Preparation (can be used through Spring 2012 reports)
  3. 2010 ALA/AASL Standards for Initial Preparation of School Librarians (must be used for reports beginning Fall 2012)
  1. Examine the assignments you currently use in your courses through the lens of the standards. Do this by looking at your syllabi, any narratives about the assignments that you provide your candidates, and the scoring tool that you use to evaluate the assignments. Which standards align?
  1. Realize that you are working within NCATE’s framework:
  2. Assessments #1 and #2 must demonstrate candidates’ content knowledge;
  3. Assessment #3 must show candidates’ ability to plan to meet program needs;
  4. Assessment #4 must be the assessment of the practicum or internship;
  5. Assessment #5 must demonstrate candidates’ effect on student learning;
  6. You must submit a sixth assessment;
  7. Assessments #7 and #8 are optional.
  1. Select the six to eight assessments that you feel best demonstrate that your candidates master the standards. If your current assessments miss the mark, create new ones. For sample assessments, explore NCATE’s SPA Assessment Library
  1. Writing to the 2010 standards: collect data at (at least) the standard level. You may collect it at the element level if you choose. See note below regarding “preponderance of evidence.” (For the 2003 standards, data should be presented at the element level.)
  1. By current NCATE guidelines, for an initial report, you must report at least two applications of data for each assessment
  1. Familiarize yourself with the Program Report for the Preparation of School Librarians American Library Association/American Association for School Librarians(ALA/AASL) (2003 standards)

(2010 standards)

  • Cover Sheet
  • Section I Context;
  • Section II List of Assessments
  • Section III Relationship of Assessments to Standards
  • Section IV Evidence for Meeting Standards

(1) A two-page narrative that includes the following:

a. A brief description of the assessment and its use in the program (one sentence may be sufficient);

b. A description of how this assessment specifically aligns with the standards it is cited for in Section III. Cite SPA standards by number, title, and/or standard wording.

c. A brief analysis of the data findings;

d. An interpretation of how that data provides evidence for meeting standards, indicating the specific SPA standards by number, title, and/or standard wording;

and

(2) Assessment documentation

e. The assessment tool itself or a rich description of the assessment (often the directions given to candidates);

f. The scoring guide for the assessment; and

g. Charts that provide candidate data derived from the assessment;

  • Section V, Use of Assessment Results to Improve Program.
  1. Attend as many program report writer/reviewer training sessions as possible.
  1. Write a draft of your report, sections I through V, a year before your program report is due. This will allow you to identify any gaps and have time to address them.
  1. Consider becoming a program reviewer yourself!

******************************************

Preponderance of Evidence

a.“Preponderance of evidence” means an overall confirmation of candidate performance on the standards in the strength, weight, or quality of evidence.

b.Programs are required to submit data at the standard level, but not at the element level, and they may disaggregate data by elements to better make their case, but that is not required.

c.Program reviewers weigh the evidence presented in SPA program reports, and when there is a greater weight of evidence in favor, they should conclude that a standard is met or that a program is recognized.

d.The elements are used by programs and reviewers to help determine how standards are met. This means that a standard could be met, even though evidence related to one or more elements is weak.

e.Reviewers make judgments that “overall” there is/ is not sufficient evidence that the standard is met.

f.Rubrics and guidance for reviewers developed by the SPA for use by program reviewers must reflect this policy. Reviewers must be trained to review evidence and make judgments based on the preponderance of evidence that standards are met. (SASB policy on preponderance of evidence, Policies and Procedures Handbook, II.C)

1