Project Narrative – Boys & Girls Clubs of Palm Beach County – Marjorie S Fisher Club- Elementary – RFA 2015-2016

Boys & Girls Clubs – Marjorie S. Fisher(Elementary)

Children Achieving & Reaching Educational Success (CARES Program)

5.1 Project Abstract:

At the Marjorie S. Fisher Boys & Girls Clubin West Palm Beach, Florida, the Boys & Girls Clubs of Palm Beach County (BGCPBC) will provide the free 21st CCLC Children Achieving and Reaching Educational Success (CARES) to 90 children in grades K-5 from five Title I schools during the school year; 20 students from Wynnebrook elementary, 20 students from Seminole Trails elementary, 15 students from Hope-Centennial elementary, 15 students from Benoist Farms elementary, and 20 students from Grassy Waters elementary, meeting 3.75 hours from 2:30-6:15pm (one later ending target school 3:15-7:00pm), Monday through Friday for 36 weeks, and 40 children during the 7-week summer break, 8:00-2:00pm, Monday through Friday.Children will receive a variety of reading/language arts, math, science, and enrichment programs that complement the regular school day. Students will be provided 60 minutes of PBL programming in a 1:15 ratio. Family services will target community needs. Students are provided free snacks and meals during program hours after school and during the summer.

5.2 Continuing Improvement

Now in its first year of funding for grant #5PCC4, the BGCPBC’s 21st CCLC program at the Marjorie S. Fisher Boys & Girls Club has not yet undergone a formative or summative evaluation. A February 2015 site visit report identifiednumerous programmatic strengths, including average daily attendance at 156% of proposed number of students (141 ADA, compared to 90 proposed), rich variety of PBL and enrichment activities; provision of an exceptionally safe environment; and daily meals and snacks at no-cost—all attributes that the BGCPBC works hard to sustain. It was recommended that the BGCPBC increase its collection of baseline data of 21st CCLC members at/near the beginning of the funding period to afford accurate and timely assessment of outcomes and to refine scheduling systems to produce daily schedules that accurately reflect daily activities. The BGCPBC has initiated programmatic adjustments to comply with these recommendations, as well as adheres to its own rigorous internal and external evaluationactivities, as described in 5.3.a, below.

5.3 Evaluation

5.3.a Evaluation Plan

Independent Evaluator and Qualifications: The BGCPBC has enlisted the services of an external evaluator for each of its 21st CCLC grant projects. Since the 2012-2013 program year, the external evaluation for BGCPBC 21st CCLC grant projects has been conducted by The Center for Assessment, Strategic Planning, Evaluation and Research (CASPER), a highly experienced firm specializing in evaluation of out-of-school programs using a continuous improvement and developmental evaluation model to assess progress towards the stated objectives. CASPER will continue to oversee all aspects of program evaluation, including formative, summative, and data reporting to both the Florida Department of Education and the US Department of Education. The evaluator fully understands the 21st CCLC program and will personally conduct all evaluation tasks. The lead evaluator for the BGCPBC grant projects is the vice president of CASPER, who is a former director of evaluation for the Statewide Florida 21st CCLC Administrative Team. Led by a professional evaluator and licensed psychologist, CASPER has overseen the evaluation of over 400 educational programs. As an active member of the American Evaluation Association and American Psychological Association, all evaluations will be conducted under the ethical codes and established procedures of these professional organizations.

Evaluation Activities: The evaluator will provide any necessary data analysis, interim reports, professional services, and assistance in training staff on collecting appropriate data which is reasonable and necessary. Formal reports will include a review of accomplishments and challenges, while also providing recommendations for program refinement based on both quantitative and qualitative data. Individual and/or group interviews with providers, staff, students, and parents will be conducted to collect additional qualitative data to help inform evaluations. Objective performance data will be collected an estimated minimum of four times per year (summer, baseline, mid-year, end-of-year). Specifically, the following activities will be included in the evaluation of the program:

  • Continuous Improvement: The evaluator will provide assistance and oversight of the ongoing evaluation using the Continuous Improvement Model (CIM). This will include individualized assessment using both formal (e.g., surveys) and informal (e.g., meetings) techniques to guide incremental changes within ongoing services, adopt new ways to improve and measure outcomes, discontinue or adapt activities that have no value, and increase emphasis on objectives and outcomes. The immediate and individualized feedback provided through CIM is particularly important for implementation of this 21st CCLC model to help guide and ensure the highest impact for each student. The overall evaluation process is designed to provide a structure to (1) generate information for ongoing program/activity refinement, (2) assess progress in meeting outcomes, (3) document/analyze how the model works in practice, and (4) document/analyze changes in targeted students’ actions, attitudes, knowledge, and performance.
  • Evaluation Support: The evaluator will provide evaluation training and assistance regarding baseline and follow-up data collection, selection of data elements, data storage, evaluation planning, development of recommendations under CIM, and use of evaluation findings for planning and revising program activities.
  • On-Site Visits: As part of the developmental evaluation model, the evaluator will provide at least two multiday, agency-wide visits during which each site of each BGCPBC 21st CCLC grant project will be visited at least once and many will be visited twice, depending upon the need of each individual site. Feedback will be provided to program-level and site-level staff as appropriate, and a formative site visit report will be completed for each grant project within 30 days of its initial on-site visit based on observations and data gathered during the project’s visit.
  • Formative Evaluation: The evaluator will provide either a formative evaluation report and/or the mid-year evaluation objective assessment tool (currently utilized by the FLDOE for 21st CCLC programs), depending on what is specifically required by the FLDOE. Regardless of whether the program is required to submit a formative report or assessment tool, the evaluator will assist and ensure all mid-year data reporting requirements are met as part of the evaluation process (e.g., initial objective tool development, summary of formative findings, formative recommendations, and responses to recommendations).
  • Summative Evaluation: The evaluator will provide an end-of-year summative evaluation report and/or annual evaluation summary (depending on specific requirements from the FLDOE). The summative evaluation report will be focused on program operations and program outcomes at the end of the operational year based on the current US Department of Education and FLDOE operational year (prior summer and current academic year). The purpose of the summative evaluation is aimed at recording and developing a model that can be applied in other settings.
  • Annual Data Reporting: The evaluator will provide assistance and oversight of the federal online submission of 21st CCLC data. The evaluator will also provide assistance in submitting any necessary data to the Florida Department of Education, if required.

Using and Sharing Evaluation Results: The process for sharing evaluation findings is integral to a strong evaluation method and overall evaluation plan, as it is the primary method for using results to refine, improve, and strengthen program outcomes. Sharing and distributing information will occur at three levels: (1) administrators, (2) staff members, and (3) stakeholders. Feedback will include written evaluation reports (as above), as well as teleconferences, when requested, with the evaluator, program director, and any staff wishing to participate. During these meetings, current data trends and operations will be reviewed with a focus on program improvement and immediate refinement. The evaluator will assist in refining and addressing any identified concerns. The evaluator will also assist, if requested, with revisions of objectives based on future needs assessments and proposed activities aligned to objectives and student needs.

Data will be utilized by the program director and teachers to help tailor program offerings to the needs and progress of individual students. For formative and summative evaluations, the evaluator will provide written reports to the program director and will help guide BGCPBC in refining and addressing any identified recommendations. In addition, on-site feedback will be provided to 21st CCLC staff to: (1) engage staff in addressing challenges; (2) promote “buy-in” into evaluation as a “living” process; and (3) promote discussion, cross-training, and support. Finally, evaluations will be shared electronically with all stakeholders (e.g., administrators, parents, and partners) to share information about the program and encourage feedback.

Reporting Outcomes: As detailed above, the evaluation process includes all elements of the required Reporting Outcomes detailed in the Request for Application. The Evaluator will submit and validate all data to the US Department of Education. The program and the external evaluator are aware there could be changes to the online method of collecting data through the US Department of Education, as well as changes with data collection by the FLDOE. The program and evaluator are committed to submitting all required data as instructed and through whatever system required.

Assessment of Progress/Performance Measures/Data Sources and Timeline: Each objective is fully explored within a continuous improvement model. For each program objective, a series of specific evaluation questions will be answered to enable project leaders to assess progress toward goals, objectives, and benchmarks. The evaluation questions have been tailored to each objective, but limited space necessitates more general questions herein. Questions will include: (1) Was the objective met? (2) Were the baseline and quarterly data sufficient to show change? (3) Did the program follow the planned interventions and activities? (4) Which activities provided the highest impact? (5) Which strategies did not have impact (should they be removed, revised, or strengthened)? (6) How can the program improve general operations to meet this objective?

Measures and data sources include a combination of quantitative data (e.g., test scores), stakeholder perceptual data (e.g., parent surveys), and qualitative data (e.g., student interviews). Objective performance data will be collected an estimated five times per year: baseline from the prior year and/or immediately upon enrollment in the program, and then once each quarter of the academic year. In addition to quarterly assessments, the program will implement a series of pre-post knowledge assessments for project-based units and lessons. All data will be analyzed with research-based statistical procedures, such as t-tests, MANCOVAs, and multiple regression.

5.3.b Measurable Objectives and Assessments (online table)

5.4 Partnerships, Collaboration & Sustainability

5.4.a Community Notice

While the 21st CCLC CARES program at the Marjorie S. Fisher Boys & Girls Club is now in its first year of the current grant, the program was funded during five years. As such, the local community is aware of the program and supportive of the BGCPBC’s intention to vie for year-two funding to sustain the program. To ensure that constituents are informed about its intention to submit this proposal, BGCPBC posts a notice on itswebsite and that which is dedicated specifically to the 21st CCLC programs( of which will provide a link to a pdf of the submitted proposal until notice of an award is received. Within 30 days of receiving notice of funding, both websites will include a link to the approved grant narratives, basic program logistics (existing hours, location of services, and activities), and contact information for the site coordinator. Formative and summative evaluations will be uploaded to the site as they become available. The BGCPBC’s 21st CCLC website will be updated at least once a month to highlight student progress and showcase special projects. Parents will be encouraged to check the website as well as the Club’s Facebook and the Clubs’ bulletin boards for information and continuous updates on 21st CCLC program activity.

Parents will be encouraged to check the website as well as the Club’s Facebook and the Clubs’ bulletin boards for information and continuous updates on 21st CCLC program activity.

This application has also been discussed in meetings with school district personnel at the school and district levels. At regular meetings with the Palm Beach County Infant, Child, Youth, and Young Adult Symposium, the BGCPBC’s intention to apply for 21st CCLC funding andits ongoing operations of 21st CCLC programs have been discussed at length.

5.4.b 21st CCLC Advisory Board

The 21st CCLC advisory board associated with grant #5PP4 (serving elementary schoolmembers) at the Marjorie S. Fisher Boys & Girls Clubmeets twice each year, unitingrepresentatives of the United Way of Palm Beach County, the Criminal Justice Commission, the Education Commission, the Children’s Services Council, the Palm Beach County School District, the local Career Source, at least two parents, one school-day teacher from each feeder school, and principals. Feedback from these and other regular stakeholder meetings, including those with the external evaluators, state evaluators, and other community sources, are reviewed with action steps taken for improvement and program development. Funders of other Boys & Girls Clubs programs that provide many of the personal enrichment services to 21st CCLC students are also involved in the planning process and as participants in the advisory board. Minutes are taken and attendance is recorded at each advisory board meeting. Upon notice of funding award, the BGCPBC 21st Century project administration, project director, and family service coordinator will work together to recruit new advisory boardmembers and solicit a commitment for continued participation from existing members.

5.4.c Collaboration with Private Schools

Private schools within the service area of this program have been provided information by email, personal visits by the site coordinators, and in print by way of flyers delivered to them for distribution. The program as described herein is made available to them. The family program is made available to the families of private school students and their teachers and other school personnel as well by the same methods. On-going community discussions, including discussion of 21st CCLC applications include private school administrators and teachers.

5.4.d Partnerships

BGCPBC is a long-standing fixture in Palm Beach County’s human service and youth development community, and enjoys strong relationships with organizations and funders across the county. BGCPBC’s nonprofit community partners offer students a broad array of in-kind activities which are aligned to at least one of the personal enrichment objectives and designed to support the academic achievement of students. Recent partners have included the Palm Beach County Sheriff’s Office (prevention programs), the Florida Health Department-Palm Beach County (free dental screenings, health promotion and wellness nutrition classes), the Center for Creative Education (a variety of free art classes), Prime Time Palm Beach County (afterschool accreditation, quality improvement systems, and staff professional development), the Norton Museum of Art (free admission and classes), the Juvenile Transition Center (summer and afterschool food programs), and the Legal Aid Society of Palm Beach County (workshops presenting information about medical insurance, fair housing, immigration domestic violence prevention, and parenting skills).

BGCPBC has a long history of support from these partners and, through regular engagement at all organizational levels, will continue to meet and strengthen the relationships so that the services provided to students will be as powerful and effective as possible. The value of the in-kind services provided by these agencies throughout the year varies but all in-kind donations are captured throughout the year and reported in year-end financial reports.

5.4 e Collaborations with the Regular School Day

BGCPBC’s 21st CCLC programs are the catalyst for the very strong working relationship with the School District of Palm Beach County (District) with which it maintains an inter-local agreement allowing for the sharing of student data, including report card grades, standardized test scores, and behavior and attendance records. Over the past 12 years of operating its 21st CCLC program, BGCPBC has successfully collaborated with the principals and teachers of feeder schools and will maintain the same level of communication in the proposed year.

Whenever possible, BGCPBC hires teachers who are also employed by feeder schools as regular school-day teachers and bring a good deal of knowledge of students and their needs. At the beginning of the school year, 21st CCLC teachers review the grades, standardized test scores from the prior Spring, and behavior and attendance records of students. They identify and group students as ‘gainers’ or ‘sliders’ and assess their particular academic and social needs. This information is communicated to regular school day teachers and parents, frequently in writing in the case of teachers and in person in the case of parents, as well as to inform the student’s 21st CCLC academic plan. As the year progresses, 21st CCLC teachers and regular school day teachers meet to compare and discuss student progress, demonstrate and discuss lessons and develop strategies to further engage their students in academic enrichment.