Michigan Department of Energy, Labor & Economic Growth

Office of Adult Learning

Adult Education and Family Literacy Act Funding

Independent Reviewer Rating Instrument

Applicant: / Application Number:
Reviewer Number: / Date of Review:

Prior to attending the Review Meeting on June 12, the Reviewers will read and evaluate each application to determine the score that will be assigned for the Narrative Application. Reviewers are asked to rate each of the required components in the Narrative Application for a maximum of 480 points. Each component and sub-component of the Narrative Application indicates the possible points that can be given. Reviewers should utilize the Application Narrative Application Instructions and Review Criteria in the Reviewer package to determine what the applicant was requested to provide for each sub-component.

Reviewers are asked to provide appropriate commentary in the spaces provided, noting strengths and areas of weakness in a constructive format that will be useful to the application authors. The score given to each component should reflect the review criteria and the commentary provided. Please make all entries in ink so that they may be copied later, upon request of the applicant.

A. Regional Adult Education and Literacy Needs (Maximum of 70 points)

If the application includes:

1.A description of how a needs assessment was conducted relating to moving adults with low basic skills to postsecondary education and employment and the specific results of the needs assessment. Additionally, a description of the proposed program and activities addressing the identified needs and priorities.

2. A description of how the program identifies and supports those individuals in need of adult basic education and literacy services who are low-income, individuals with disabilities, single parents, displaced homemakers, individuals with multiple barriers to educational enhancement, and individuals with limited English proficiency or minimal literacy skills.

3. Offers a detailed reasoning for program offerings and gives viable options for individuals who need services outside of what they offer.

4. Indicates the process for continually assessing and addressing the future needs of participants and the applicant’s working relationship with the WDB and EAG.

Points Awarded: ______

Comments:

B. Program Design(Maximum of 160 points)

If the application includes:

1. Curriculum and Instruction

  1. Description of the specific program offerings and how they are aligned with real opportunities for employment, postsecondary education and training.
  2. Description of the core curriculum materials used. Identified measurable program goals that enhance participant learning, reflects the community and learner needs, and incorporates socio-economic realities such as employment opportunities and postsecondary attainment.
  1. For each program offering, describes the various instructional methods used in the program, how multiple forms of service delivery modes are utilized, and how your agency determines their effectiveness. Description of Instructional methods used in the program, various modes of service delivery and how program effectiveness is determined.
  2. Description of the use of a wide variety of multi-level texts, current and contextualized reading materials for persons with learning and/or physical disabilities, and limited English proficiency.
  3. Description of how the agency aligns its curriculum with ABE, GED, and or ESL state recommended content standards and guiding principles in Section 2.

2. Program Intensity and Duration

  1. Provides the program’s days and hours of operation, including number of weeks in the semester or term and explain how this scheduling allows participants to achieve substantial learning gains and attain their outcome goals.
  2. Identifies the enrollment schedule for learners and the process for enrolling new learners throughout the program year.
  3. Describes how the program’s course schedule accommodates adult learners’ various schedules.

3. Alignment with State Performance Objectives

  1. Describes any services offered to enable individuals to secure employment within a specific industry or occupational sector and/or to advance to higher levels of education.
  2. Describes how the program aligns exit and entry requirements so that expected learner outcomes in one program match the skills, knowledge and abilities required to enter a subsequent program.
  3. Describes how the program accelerates the advancement of learners to transitions to employment or postsecondary education and training.

4. Technology

  1. Describes how technology is used to enhance the instructional program or to deliver adult education services and provide specific examples. How are on-site learning labs or other on-site uses of technology used to increase the effectiveness and efficiency of learning? Is technology being used to meet the needs of learners who cannot participate in regular on-site coursework via distance learning approaches?
  2. Provides information about the current practices, skills and knowledge of staff and participants in the adult education program that relate to the use of technology, including computers in the classroom.
  3. Provides information on how student technology skill levels and needs are assessed and how the curriculum or instruction is or can be designed to address those needs.

5.Tuition/Fees (Program Income)

  1. If applicable, describes program’s tuition/fee structure and the methodology used to determine any tuition and/or fees charged to an eligible adult education participant, and how they are used to augment the program.

6.Recruitment & Retention

  1. Describes the marketing and recruitment strategies utilized to inform potential participants, referral agencies, special targeted groups and the general public of available services. Describe how the agency’s marketing and recruitment plan enables the prospective learners in identified target populations in the community to find out about the program, the learning opportunities it offers, and how to take advantage of them.
  2. Describes the retention policy and how community resources are utilized to promote continued learner participation until goal attainment.

Points Awarded: ______

Comments:

C. Staff Development(Maximum of 30 points)

If the application includes:

  1. Describes how the agency ensures that instructors, counselors, and administrative staff are well-qualified and certified based on professional criteria.
  2. Describes any pre-service training provided for new teachers and/or tutors.
  3. Describes how the agency assesses the effectiveness of instructional staff.
  4. Describe the professional development opportunities available to nurture, sustain, and support quality teaching and learning through the adult education and family literacy program.
  5. Describe how the information and material obtained from staff participation in training or attendance at conferences is shared with other staff and used to improve program performance.

Points Awarded: ______

Comments:

D.Coordination of Efforts, Partnerships and Collaborations (Maximum of 80 points)

If the application includes:

  1. Describes the linkages, agreements and/or coordination of activities between adult education and family literacy program and other educational community and/or workforce support services (e.g., K-12 school districts, postsecondary institutions, workforce development agencies, literacy councils, libraries, employers, community-based organizations, faith-based organizations, job training programs (integrated with occupation training), regional economic development agencies, proprietary schools, local office of state government, local foundations/United Way) that support individuals transitions to postsecondary, employment or training.

Consider the following elements:

  1. Describes what experience the partnering organizations have had in delivering the proposed range of services.
  2. Identifies the governance structure for the partnership and the role of the applicant for this grant.
  3. Attaches to the application letters of commitment identifying the roles and responsibilities and resource commitments of each of the partner organizations.
  1. Describes how your program solicits ongoing input and feedback from your partners to continuously improve programming. Provides how shared responsibility is promoted to collect data on program performance.
  2. Identifies any limitations that partnerships may have to serving participants including capacity and funding restrictions, and how these limitations are being addressed.

Points Awarded: ______

Comments:

E. Performance and Evaluation(Maximum of 140 points)

If the application includes:

  1. Adult Learning Plans (ALP)
  1. Describes the process for using the DELEG-approved ALP form for each participant and how adjustments to the plan will be appropriately incorporated.
  1. Assessment
  1. Participant Level:
  2. Identifies the DELEG-approved assessment utilized for each program of enrollment.
  1. Describes how the program will track and monitor hours of instruction to ensure continued eligibility and testing timelines for each participant.
  2. Addresses how common learner assessments are being used and shared with all agency partners.
  3. Describes the process for placing students in the appropriate program.

b. Program Level:

i. Describes how the program is evaluated using specific data and comparative analysis.

3. MAERS

  1. Describe how your agency will ensure quarterly data entry into MAERS that accurately reflects the information in the participant’s file. (Note: All eligible participants enrolled must be entered into MAERS regardless of the number of instructional hours they received).
  2. Describe how your agency will use the Participant Enrollment Report, the Missing Data Report, and the Upcoming Soft Exit management reports from the MAERS system to monitor your program performance and ensure timely and complete participant data entry into MAERS.

4.Follow-Up

  1. Describe the local process for goal setting and reporting all measurable goal attainments, which align with State Performance Measures (Educational Goals: GED, HSD, Enter Postsecondary; Employment Goals: Obtain a Job, Retain a Job, Improve a Job), as per State follow-up requirements.
  2. Describe the process for completing and having on file a follow-up survey form for each participant.
  3. Describe the process for contacting participants within the required timelines, and completing and having on file a contact log sheet. (Educational Goals: throughout the year as appropriate; Employment Goals: quarterly).

Points Awarded: ______

Comments:

State Negotiated Projected Targets for 2009-2010

The performance goals for all of Michigan’s state and federally funded adult education and family literacy programs have been negotiated between DELEG and USDOE. Upon entry into the adult education program, a participant designates his or her program of instruction and expected learning goal. The EFL gains and the Goal Related Outcomes are determined by standardized assessments and DELEG follow up process/survey appropriate to the participant’s goal. The assessments used must be those approved by the USDOE and DELEG. The 2009-2010 USDOE projected levels of performance for Michigan are as follows:

State Performance Measures for Michigan
Performance Measures / 2009-2010 - Projected
Beginning ABE Literacy / 38.00%
Beginning Basic Education / 35.00%
Low Intermediate Basic Education / 36.00%
High Intermediate Basic Education / 31.00%
Low Adult Secondary Education / 34.00%
High Adult Secondary Education / NA
ESL Beginning Literacy / 56.00%
ESL Low Beginning / 67.00%
ESL High Beginning / 65.00%
ESL Low Intermediate / 54.00%
ESL High Intermediate / 57.00%
ESL Advanced / 60.00%
High School Diploma & GED / 55.00%
Entered Employment / 60.00%
Retained Employment / 55.00%
Placement in Postsecondary Education or Training / 60.00%

After assigning points, check for consistency between the score assigned to the sub-category and the comments made in the comments section. Comments and scores must reflect the same rating. Record the score for each sub-component on the Independent Review Rating Summary Sheet.

If the applicant has indicated they will be providing any of the following program components they are to have completed a “Program Supplement” for that component and included the supplement(s) in the application. Please review the program supplements for further information.

Check if a program supplement is included:

______Least Literate Set-Aside Funding

______Corrections/Institutional Set-Aside Funding

______Family Literacy

______English As A Second Language (ESL)

______Workplace Literacy Services

All instructional services provided for any of these programs must be:

measurable by DELEG approved assessments;

integrated within adult basic education, ESL, or high school completion classes;

have all participating students placed within an Educational Functioning Level and recorded in MAERS; and,

provided only to students eligible for this federal funding.

Michigan Department of Energy, Labor & Economic Growth

Office of Adult Learning

Adult Education and Family Literacy Act Federal Funding

GENERAL INSTRUCTION

Narrative Application Components

/ Reviewer
Score /

Comments

A. Regional Adult Education and Literacy Needs and Priorities 70 points maximum
Question #1 20 points total
  • Description of needs assessment -4pts.
  • Mentions moving low basic skills individuals to postsecondary -3pts.
  • Mentions moving low basic skills individuals to employment -3pts.
  • Specific results of needs assessment-4pts.
  • Description of programs -2pts.
  • Description of how programs meet identified needs -4pts.
Question #2 31 points total
  • Describes how the program identifies low income, individuals with disabilities, single parents, displaced home workers, individuals with multiple barriers, individuals with limited English proficiency or minimum literacy skills- 10 points
  • Identifies support services for subpopulations 3pt. for each support service/maximum points -21 pts.
Question #3 10 points total
  • Describes how program offering was determined -5pts.
  • Describes what happens to individuals who need Adult Education programs not offered -5pts.
Question #4 9 points total
  • Identifies a plan for continually assessing program needs -5pts.
  • Identifies a relationship or plans to build a relationship with WDB and EAG. -4 pts.

Applicant:Application Number: Reviewer Number:

B. Program Design 160 points maximum
1. Curriculum and Instruction 80 points
A.
  • Description of programs -5pts.
  • Demonstrates program alignment with opportunities for employment, postsecondary education and training -8pts
B.
  • Description of curriculum -5pts.
  • Identifies measurable program goals -5pts.
C.
  • Description of instructional methods- 10 pts.
  • Identifies multiple service modes-5pts.
  • Determination of program effectiveness-5pts.
D.
  • Description of multilevel texts with person of learning disabilities, physical disabilities and limited English proficiency -10pts
E.
  • Description of how the curriculum is aligned with content standards -6pts.
  • Alignment with guiding principles -3 pts. For each guiding principle that is being used in the program -21 points total
2. Program Intensity and Duration 25 points
A.
  • Provides the program’s schedule -5 pts.
  • And how this schedule allows for substantial learning -5 pts.
B.
  • Provides and enrollment schedule -5pts.
  • Describes a process for enrolling students throughout the year -5pts.
C.
  • Describes how the program schedule meets learners’ scheduling needs -5pts.
  • Alignment with State Performance Objectives 20 points
A.
  • Describes services offered to secure employment
Describes alignment with postsecondary and state negotiated performance goals -5 pts.
B.
  • Describes aligned exit and entry requirements-5pts.
C.
  • Offers accelerated programs-5pts; Describes accelerated programming- 5pts.
4. Technology 15 points
A.
  • Description of how technology enhances instruction-3pts; Offers Onsite or learning labs-2pts.; Offers distance learning opportunities -2pts.
B.
  • Technology skills of staff and use in the classroom-2pts. Describes technology skills of students-2pts.
C.
  • Assessment of student technology skills -2pts; Describes how the curriculum is designed to address technology needs-2pts.
5. Tuition and Fees0 points
A.
  • Structure and methodology used to determine.
How fees are used to augment the program.
6. Recruitment and Retention 20 points
A.
  • Describes marketing strategies-5pts; How these strategies reach out to target populations-5pts.
B.
  • Describes retention policy-5pts; and how community resources are utilized to ensure goal attainment-5pts.

C. Staff Development—30 points maximum
1. Staffed by well-prepared instructors, counselors, and administrators-5pts.; Staffed by certified staff -5pts.
  1. Describes pre-service training-5 pts.
  2. Describes staff evaluation procedures that include classroom observation-5pts.
  3. Describes continued professional development -5pts
  4. Sharing of training to improve program performance-5pts

D. Coordination of Efforts, Collaboration and Partnership—80 points maximum
1. Describes linkages with Adult Basic Education (literacy council), Adult Education, Post Secondary Institute, Employers or Training (10 points for each mentioned- 40 total possible)
2. Describes how your program solicits feedback form partners 10 pts. Describes how responsibility is shared for data collection on program performance -10 pts.
3. Identifies any limitations to the partnerships 10 -pts. and how these limitations may be addressed -10 pts.
E. Performance and Evaluation 140 points maximum
1. Use of DELEG Approve Adult Learning Plan -5 pts. and how adjustments are made and incorporated-15 pts.
2. Assessment
a. Participant Level
  1. Identifies DELEG approved assessment for each program of enrollment-5pts.
  2. Describes how hours will be tracked and monitored-10pts.
  3. How learner assessments are being used with agency partners-5pts.
  4. Describes the process for placing students in appropriate programs according to their Educational Functioning Level -10 pts.
b. Program Level
  • Describes how the agency evaluates its program-10pts.
  • Provides specific data and comparative analysis to support findings-10pts.
3. MAERS
  1. Process for entering data on a quarterly basis and describes a process that insures accuracy in data entry -20pts.
  2. How Participant Enrollment, Missing Data Report, and Soft Exit Report will be used to evaluate program-10pts.
4. Follow up
  1. Local process for goal setting and process for reporting all measurable goals-20pts.
  2. Process for completing follow up for all participants.-10pts.
  3. Process for contacting participants and maintaining contact log-10pts.

Total Points

Michigan Department of Energy, Labor & Economic Growth

Office of Adult Learning

Adult Education and Family Literacy Act Federal Funding

INSTITUTIONAL

Independent Reviewer Rating Instrument Summary Sheet

Applicant:Application Number: Reviewer Number:

Narrative Application Components

/ Reviewer