Pennsylvania Commission on Crime and Delinquency (PCCD)

Grantee Outcomes Report Template

Guidelines for Reporting on the Incredible Years

As a requirement of funding under PCCD’s Research-based Programs Initiative, all grantees are required to submit a cumulative outcomes report. Grantees awarded funding in 2010 and 2011 should submit this report in the 3rd quarter of the 2nd year of funding.

The purpose of the Outcomes Report is to conveythe experience of the grantee in implementing the program, and to summarize the program’s reach, implementation quality, and impact. Beyond meeting the reporting requirements of PCCD, the Outcomes Report canalso serve as a valuable tool to the grantee for communicating the program’s impact to local stakeholders.

We recommended that prior to completing the report, grantees print copies of their grant application and quarterly E-grants reports or other program data. These resources should be used to respond in narrative format to all of the outcome report questions. Please answer using complete sentences.

Please also include the numerator and denominator and calculation explanations for any data or percentages provided. For example, rather than a) reporting that 60% of participants demonstrated change or b) 60 participants demonstrated change in a specified outcome, report that 60 of 100 surveyed participants (60%) demonstrated change.

Please only report on data that is reflective of participants and services funded by PCCD. If multiple grants have been funded, separate outcome reports are required for each grant.

The EPISCenter has provided guidance and clarification for responding to the required questions in the gray, shaded boxes below the questions. Also, please pay special attention to Notes included in the template. If you need assistance completing this report, please contact your assigned EPISCenter Prevention Coordinator. You are strongly encouraged to submit your draft report to your assigned EPISCenter Prevention Coordinator for feedback prior to submitting the report to PCCD. The final report should be attached in E-grants with your quarterly report.

Grant ID #:

Grantee name:

Person completing this report (name, phone, email):

Evidence-basedprogram being implemented:

Initial amount awarded:

Geographic location/county/school(s)served:

Is this grant a new implementation or expansion of an existing program?

Period covered by this report

Grant start date:

Current funding year & quarter:

Report submission date:

Describe any major changes to the project plan from what was originally proposed, and why those changes were necessary. If a Project Modification Request (PMR) was submitted, please indicate so. You may simply copy and paste the description and justification from the PMR as your description here.

Response Guidance:

At the time of writing a grant application, it is impossible to foresee all the influences that may lead to implementation barriers and challenges. These challenges can lead to changes to the envisioned project plan. Discuss challenges you encountered and any resulting changes to your proposed implementation.

SECTION 1 – DESCRIPTION OF POPULATION SERVED

In this section, you will describe the population that has actually been reached by the program since the beginning of the grant period.

  1. Total number of youth served directly by the program to date:

Response Guidance: Indicate the number of youth who received the IYS DINA Classroom and/or IYS DINA Small Group curriculum at your site at the time of this report. This is not applicable for sites implementing IYS BASIC and/or IYS ADVANCE Parent program(s).

  1. Total number of adults served directly by the program to date:
  1. Of the total number of program participants served, how many successfully completed the program?
  2. Total number of youth who completed the program
  3. Total number of adults served who completed the program

Response Guidance:

IYS DINA Classroom or IYS DINA Small Group:

Indicate the total number of youth who completed 45 of the 60 Year 1 lessons or 30 preschool lessons (Aggregate data from PM: Youth that Completed: 45 of the 60 Year 1 lessons or 30 preschool lessonsfrom EPISCenter Spreadsheet Tracking Tool)

IYS BASIC or ADVANCE Parent Program:

Indicate the number of adults served who attended at least 75% of the sessions of the Incredible Years BASIC or ADVANCE Parent program (Aggregate data from PM: Adults that attended at least 75% of IYS BASIC or ADVANCE sessions from EPISCenter Spreadsheet Tracking Tool)

  1. Of the total number of program participants, how many did not complete the program?

Response Guidance:

IYS DINA Classroom or IYS DINA Small Group:

Indicate the number of youth who completed less than 45 of the 60 Year 1 lessons or 30 preschool lessons

IYS BASIC or IYS ADVANCE Parent Program:

Indicate the number of adults served who attended at less than75% of the sessions of the Incredible Years BASIC or ADVANCE Parent Program

Note:3.a and 4.a. should equal the total number of youth served (Section 1, Question 1)

3 .b and 4.b should equal the total number of adults served.(Section 1, Question 2)

  1. Describe the overall demographics of the population served.

Age range served:

Gender ratio (percent male/percent female), if known:

Races/ethnicities served, if known:

Explain the risks and other characteristics of the population you are serving.

Response Guidance:

Share any demographic information collected or available through observation for the youth and adults served by the Incredible Years programs. BASIC and ADVANCE Parent Programs and/or DINA Classroom or Small Group Curriculum

  1. Provide a listing of all the zip codes and schools served by the program. (This information will be used by PCCD to measure the saturation of each program type across the Commonwealth.)

Zip Codes Served:

Response Guidance: Use the addresses of the youth and adults served to list all zip codes served

Name of Schools District(s) Served:

Response Guidance: Indicate all the school districts targeted by your project, if applicable.

Name(s) of Individual Schools Served:

Response Guidance: Indicate the schools that the youth participants attend, for your site if you are implementing the IYS DINA Classroom or IYS DINA Small Group curriculum.

  1. Was the program universally implemented or was it targeted at a specific at-risk or referred population? If targeted, what criteria were used to select participants and what was the target population? (Programs targeting an entire grade of students are considered universal).

Universal or targeted:

Criteria for referral or participation:

Referral process:

Referral sources:

Did you encounter any barriers to recruitment or referral?

Response Guidance:

IYS DINA Classroom and IYS DINA Small Group Curriculum:

The Incredible Years DINA curriculum was developed to be used as a universal and/or selective intervention. Describe how your site used the IYS DINA Curriculum and make sure to include what criteria were used if a specific population was targeted.

IYS BASIC and IYS ADVANCE Parent Programs:

The Incredible Years BASIC and ADVANCE parent programs target parents of high-risk children and/or those youth displaying behavioral problems. Describe in this section how your site implemented the IYS BASIC and/or ADVANCE Parent Programs and how your site determined the criteria for targeting families for intervention.

If specific eligibility criterion was used, such as income requirements, age, or geographic residency, please explain this criterion

  1. Describe the dosage or amount of service provided. Depending on the program you are implementing, this would be the number of lessons delivered, number of mentoring hours, or number of sessions participated in. Please include the level of participation by all program participants, as well as the percentage of participants who received the full amount of service (e.g., percentage of youth served who received all of the lessons of a curriculum).

SECTION 2 - INDICATORS OF PROGRAM IMPACT

In this section you will describe how you measured the impact your program had on its target audience including what data you collected, how it was analyzed and what the data tells you.

Response Guidance: In order to report these numbers meaningfully to your community stakeholders it will be important to reference the program’s logic model and particularly the risk and protective factors targeted for change. You can access the IYS Program Logic Models from the following links:

IYS BASIC Parent Program

IYS ADVANCE Parent Program

IYS DINA Classroom Curriculum

IYS DINA Small Group Curriculum

Notes:

In this section, please indicate the total number of participants surveyed or about whom data was collected and included in the reported outcomes.

In this section, please include the numerator and denominator for all calculations.

  1. Indicate change in baseline community level indicators that were identified in your grant application that you have tracked over the length of the grant (i.e., PAYS data, child abuse rates, juvenile court or probation statistics, school dropout rates, etc.).
  1. Other than the required PCCD evaluation tools, describe any instruments or methods used to measure the impact of your program. Also, please describe plans for continued data collection and analysis after the end of the grant period.
  1. Indicate the total number of participants surveyed or about whom data was collected, and explain any challenges you encountered in collecting or analyzing the survey data?
Given the objectives and benchmarks identified in the grant application and the required Performance Measures for your program, please summarize any quantifiable changes in attitudes, skills, knowledge, intentions, or behaviors among the population served. At minimum, please include cumulative totals (across the entire grant period) for all Outcome Performance Measures for your program in table form (see below for an example). Additionally, if applicable, please describe any other peripheral benefits realized by the community, families, etc.
Outcomes Performance Measures / Total Number with Positive Change / Total Number Surveyed / Total Percent
IYS BASIC Parent Program
Outcome # 1 / Adults with Decreased Inconsistent Discipline
Outcome # 2 / Adults with Increased Appropriate Discipline
Outcome # 3 / Adults with Increased Positive Parenting
Outcome # 4 / Adults with Increased Clear Expectations
IYS ADVANCE Parent Program
Outcome # 1 / Adults with Decreased Anger/Aggression
Outcome # 2 / Adults with a Reduction in Depression Symptoms
IYS DINA Classroom
Outcome # 1 / Youth with Decreased Antisocial Behavior
Outcome # 2 / Youth with Improved Concentration/Attention
Outcome # 3 / Youth with Increased Emotional Competence
IYS DINA Small Group
Outcome # 1 / Youth with Decreased Antisocial Behavior
Outcome # 2 / Youth with Improved Concentration/Attention
Outcome # 3 / Youth with Increased Emotional Competence
  1. If any outcome goals of the project were not met, please describe the reason(s) if known:
  1. Please describe any ways in which you exceeded the expectations of the project as proposed?
  1. Describe how you measured your clients’ satisfaction with the program and explain any available indicators of client satisfaction, quantitative or qualitative.

Response Guidance:

IYS DINA Classroom and IYS DINA Small Group Curriculum:

Not applicable

IYS BASIC and IYS ADVANCE Parent Programs:

Use data collected from the Weekly Parent Evaluation form to describe the participants’ satisfaction with the program.

SECTION 3 – INDICATORS OF IMPLEMENTATION QUALITY AND FIDELITY

In this section, you will describe how you assessed the quality and fidelity of implementation, and the results you found.

  1. Describe how you assessed implementation quality and fidelity to the developer’s proven program model:
  • What specific tools did you use and who provided them to you? (please list all tools used)
  • What did you assess? (e.g., adherence to curriculum/manual, dosage, quality of delivery, participant responsiveness, program reach/targeted population, etc.)
  • How often were assessments conducted? (e.g., weekly, 2x/cycle, quarterly, etc.)
  • How were assessments conducted? (e.g., observations, self-reports, visit from developer, etc)
  • Who completed the assessment? (could be more than one person)
  • What barriers did you encounter during the assessment process and how did/will you overcome them?
  1. Summarize the results of implementation quality and fidelity monitoring:
  • Describe the information you gathered using the tools and processes described above (Section 4, item 1). Report on specific numbers (rates, percentages) when possible. Note any areas of strength or weakness in implementation fidelity. If you have developed specific processes for providing feedback to implementers to support continuous quality improvement, please describe. Also note any changes you made in your implementation in response to areas of weakness identified.
  • Indicate how you will change your implementation to improve fidelity.
  • Indicate whether/how you anticipate continuing to monitor implementation quality and fidelity after grant funding ends.

Response Guidance:

In addition to answering the questions, be sure to include:

Other fidelity measurement tools beyond what was required by PCCD.

The number/percentage of sessions/lessons with a minimum fidelity score of .75 aggregated from your PM calculation spreadsheet tool.

Recommendations from developer from Quality Assurance process related to implementation quality/fidelity.

SECTION 4- LESSONS LEARNED

  1. With the experience and knowledge you now have, please describeanything you would have done differently to improve the program’s implementation or effectiveness?
  1. What lessons have you learned that would benefit other communities who are considering implementing this program?

SECTION 5 - PROGRAM SUSTAINABILITY

  1. Who has the responsibility of sustainability planning for your program?
  1. To whom and in what ways have you communicated the impact of the program (e.g., your Collaborative Board, School Board, CTC Board, County Commissioners or other officials such JPO or CYS, United Way, PTO, news outlets, other local prevention efforts, etc.)?
  1. What specific steps have you taken to sustain the program beyond PCCD funding (e.g., detailing the budget, meeting with stakeholders, securing local investment, applying for additional grants)? If you have applied for additional funding from any source to support the program, please list the sources and status of the application.
  1. What sustainability funding have you secured, and what other accomplishments have been made to sustain or institutionalize the program?
  1. Please describe any new relationships have you created, or existing relationships that have been strengthened, during the course of this grant, that have or could be beneficial to the program’s effectiveness or sustainability.

THANK YOU FOR YOUR TIME!