Milwaukee Area Technical College (MATC) P116W090125
Evaluator: Rosa Castro Feinberg, 305-226-1668, ,
2660 SW 119 Ct. Miami, FL 33175
FINAL EVALUATION REPORT
March 29, 2013
Pathways to Construction Trades (PaCT)
MATC School of Technology and Applied Sciences
Funding Period from 01/01/2010 to 12/31/2012, Special Focus Competition:
Innovative Strategies in Community Colleges for Working Adults and Displaced Workers
Funds Requested from FIPSE: $745,555. Funds Awarded: $547,000.
Additional Funding: $651,447 from the Milwaukee Area Workforce Investment Board (MAWIB)
FIPSE Program Officers: Frank Frankfort
FIPSE Education Research Analyst: Susan Lehmann
Project Summary
The Milwaukee Area Technical College (MATC) Pathways to Construction Trades (PaCT) Project increased the number of minorities entering the industry through the development of a short-term technical training program of 16 weeks duration in basic construction and green skills. Home language instruction in the construction trades and math was integrated with delivery of contextualized teaching in Adult Basic Education (ABE) and English as a Second Language (ESL) instruction for low-skilled displaced workers. The program provided participants with basic skills development in the construction trades and related soft skills, team building and leadership opportunities, information on career pathways, and assistance in planning their educational and career goals.
Program design features included a 511 hour course for Hmong and Spanish language origin participants and a 420 hour course for English language origin participants. A bridge program linked Adult Basic Education (ABE) and English as a Second Language (ESL) instruction to transition dislocated workers with linguistic and academic barriers into postsecondary training and facilitate exploration of multiple career options as students engaged in occupational training. The additional time for English Language Learners was devoted to instruction in English as a Second Language (ESL). All students received the reinforcement provided by contextualized instruction and had the opportunity to use the resources of the Learning Labs at MATC and at the HIRE Center on an individual basis. Field trips to industry sites and campus tours fostered incorporation into the college and industry communities.
Project PaCT was administered by the School of Technology and Applied Sciences (T&AS) of MATC. The primary partner in this project was the Milwaukee HIRE Consortium. The HIRE Center managed recruitment and case management and provided an array of support services to the project participants, including follow-up services for one year following the completion of the program. Consortium members included the Milwaukee County Labor Council, Wisconsin Job Service, AFL-CIO Labor Education and Training Center, MATC, Wisconsin Regional Training Partnership (WRTP), and the Milwaukee Area Workforce Investment Board (MAWIB).
The Project PaCT Advisory Committee was comprised of faculty and administration representatives from the MATC advisory committees for Carpentry, Electricity and Masonry; members of the MAWIB Board, the HIRE Center, the Wisconsin Regional Training Partnership/BIG STEP, PieperPower, the Southeast Wisconsin Carpentry Joint Apprenticeship and Training Committee, and the Allied Construction Employers Association. It also included representatives from community based organizations. The PaCT Advisory Board advised on issues affecting the industry, helped define needed workforce skills, reviewed curriculum, and visited training classes.
Project Goals
The goal of Project PaCT was to develop curriculum and implement a program in basic construction skills and green skills resulting in skills certificates, postsecondary credit, employability plans, and jobs for project participants who are dislocated workers with academic deficiencies or linguistic barriers.
The purpose of the evaluation was to produce information on outcomes useful for crafting project improvements and as a basis for decision making by funding sources, program staff, partners and advisors; and potential replicators. The collaboration on evaluation processes by PaCT participants, MATC’s administrators, faculty, staff, and Institutional Research Division; and HIRE Center administrators and caseworkers is appreciated. The assistance of MATC Workforce and Economic Development Assistant Dean and Project PaCT Co-Director Mona Schroeder-Beers and HIRE Center Employment and Training Specialist Roger Hinkle with data collection and of MATC Instructional Research and Strategic Planning Research Analyst Colleen Wagner with statistical tests is noteworthy.
Focus of this Report
This fourth and final evaluation report summarizes evaluation findings, and describes who benefited from Project PaCT. Unanticipated outcomes and advice to project replicators, including suggested modifications to the project, are included in a description of what was learned from project implementation.
Summary of Evaluation Findings
Summaries of prior findings are integrated with updated information. From the beginning of the Fall 2012 semester through the end of January 2013, MATC faculty and HIRE staff did a final roundup of participant employment and enrollment information. A PaCT Reunion held December 4th provided opportunity for confirmation and expansion of the data through informal interaction between the evaluator and the participants. Data sheets were completed for each participant and are on file at MATC and in the evaluator’s office. The results were tallied by the evaluator and are reported below.
Evaluation Question 1: What is the employment and enrollment status of Project PaCT participants after project completion?
The expectation that project participants will gain employment or continue their studies is considered in this section.
Data updated as of Feb. 1, 2013 are available for 36 of the 43 participants who successfully completed their technical courses, accounting for a total of 84% of the total group. Two participants were hospitalized and are disabled; their data is not included in the following summary.
Employment Status
The recession and weak recovery greatly impacted career prospects in the construction industry. The impact of the crisis on the African American students who constitute the majority of the project’s participants may have been exacerbated by the effect of racial barriers to employment. According to Levin (Jan. 2012), no metro area has witnessed more precipitous erosion in the labor market for black males over the past 40 years than has Milwaukee, a city that shows the largest racial disparity in employment rates for males of any metropolitan area in the country.
Nevertheless, thirty, or 88% of the thirty-four participants under consideration, are employed or enrolled in additional classes. Thirteen of the PaCT graduates (including two who are self-employed), or 44%, were employed full time. Equipped by their project participation with added skills, confidence, and contacts, six, or 18%, earned higher salaries after participation in the PaCT Project than they did before they were displaced from employment. It should be emphasized that these results are skewed by the extent and duration of the economic crisis and its effect on the construction industry. Construction work and employment opportunity was limited and there was also no growth in green jobs as originally projected.
Enrollment Status
Over a third of the project graduates under consideration had enrolled in classes to prepare for the GED or the Accuplacer exams or in short term or semester training opportunities leading to a certificate of completion, a diploma, or an AA degree. Twelve students, or 35% of the group, enrolled in ESL, Communication Skills, and GED classes; or in Heating, Ventilation, and Air Conditioning (HVAC), Appliance Technician, Automated Manufacturing, and Food and Beverage. Three students earned Commercial Drivers Licenses. One participant was taking classes designed to help him pass the Accuplacer. Three Project PaCT graduates successfully completed Appliance Repair requirements.
Three others passed the Accuplacer. Of these three, one enrolled in a Carpentry Program at Waukesha County Technical College; one enrolled in the MATC Cabinet MakingDiploma Program, and another was on the waiting list for an MATC Associate Degree granting Culinary Program. Enrollment is pending for students accepted into Health Technician, Asbestos, and Carpentry.
These accomplishments indicate that Project PaCT graduates were able to overcome multiple challenges to continued study identified in previous evaluation reports. The data for these 34 members of the total group of PaCT participants are consistent with employment and enrollment rates of from 94% to 75% for Cohorts 1 through 4 previously reported. Prior evaluation reports are available at http://www.matc.edu/pact/resources.cfm .
Conclusion
The expectation that project participants will gain employment or continue their studies has been met for a large majority (88%) of the participants for whom updated information is available.
Evaluation Question 2: Do Project PaCT participants demonstrate learning gains in their technical and academic courses?
Of the 51 participants to start the training, 42 (82%) completed the academic training and 43 (84%) completed the technical training. Completion rates by cohorts are presented in Table 1, below.
Technical Courses
Summaries of data from faculty records on student performance on the Worldwide Instructional Design System (WIDS) Course Assessments, the evaluator’s observations of students' shop work and finished products displayed during Community Night, and students' self reports on their skill acquisition are the basis for an affirmative answer to this part of the evaluation question.
To pass technical courses, students completed all required assignments and earned at least 70% of the possible points for each competency. Forty-three participants (84%) completed the technical training. They earned certifications in Occupational Safety and Health Administration (OSHA), Cardiopulmonary Resuscitation/ Automated External Defibrillator (CPR/AED), First Aid and a Certification of Completion of the PaCT training. Several participants received a certificate of Perfect Attendance or Near Perfect Attendance. All individuals who completed the training received one MATC program credit for completing Carpentry 302, OSHA and First Aid.
Table 1. Number and Percent Passing Academic and Technical Courses, by Cohort
Cohort Number/Language / Number Enrolled / Number Completing Courses / Percent completing coursesAcademic / Technical / Academic / Technical
1. English / 16 / 14 / 14 / 88% / 88%
2. Spanish / 14 / 10 / 11 / 71% / 79%
3. English / 11 / 9 / 9 / 82% / 82%
4. English / 10 / 9 / 9 / 90% / 90%
TOTAL / 51 / 42 / 43 / 82% / 84%
Academic Courses
Analysis of participants’ pre-and–post-test scores for Math on the Wide Range Achievement Test 4 (WRAT4), Forms Blue and Green, and for Reading on the Tests of Adult Basic Education (TABE), Forms 9&10 provide the data that responds to this part of the evaluation question. For each cohort, the number and percent of students who advance at least one functional level (the equivalent of two grade levels) or remain at the highest level were identified and are reported by Cohort in Table 2, below. The standard set by project staff was that 80% of the participants would demonstrate these measurable gains in academic skills.
The participants in Cohorts 1-4 demonstrated functionally significant learning gains in math, exceeding the 80% standard by three percentage points. Slightly less than half of the participants demonstrated functional gains in reading.
Table 2. Percent Meeting Expectation of Increase in Functional Level from Pre-to Post-Test, by CohortCohort and Number of Participants / Advanced in Math Functional Level from Pre-to Post-Test on WRAT4 Math / Advanced in Reading Functional Level from Pre-to Post-Test on TABE 9/10 Reading
1. N=14 / 71% / 29%
2. N=10 / 90% / 50%
3. N=9 / 78% / 56%
4. N=9 / 89% / 66%
TOTAL N=42 / 83% / 47%
The results of the Wilcoxon signed ranks test show that post-test Math scores were higher than pre-test scores with the statistical significance of the difference between pre-and post-tests at the .023, .012, .028, and .012 levels for Cohorts 1, 2, 3, and 4, respectively, confirming that cohort pre/post scores differ to a greater degree than would be expected by chance. Pre-test Reading scores were higher than post-test scores but not to a significant degree for any of the four cohorts.
The Kruskal-Wallis test was used to analyze the changes in pre-post-test scores in the four groups.The Mann-Whitney test was then applied to compare the significance in changes among the cohorts. These statistical analyses show that Cohort 4 is the most improved in Reading and in Math. The change in Math scores shows the second and third groups improved more than the first group. The score change in Reading is not significantly different among the four cohorts (.187) while the difference in Math is significant at the .000 level.
Results of statistical analysis for Cohorts 1 through 4 are presented in the appendices for the Year 2 (for Cohorts 1 through 3) and Year 3 (for Cohort 4) Evaluation Reports. Both reports can be accessed on-line at http://www.matc.edu/pact/resources.cfm .
Conclusion
Three sources of data support the conclusion that project participants in the four cohorts demonstrated measurable gains in technical skills.
The participants in all four cohorts demonstrated functionally and statistically significant learning gains in math.
Slightly less than half of the participants met the functional level standard for Reading. There was no significant difference in pre- to post-test results in Reading for any of the cohorts.
Evaluation Question 3: Do project PaCT participants report indicators of improved quality of life after program participation?
Data from semi-structured open-ended interviews with project participants provide insights into students’ perceptions of the project’s impact on their lives as students and labor force, family, and community members.
Here are some examples of the students’ comments.
-I am now more confident than I have ever been in my life. In construction I have to read a tape. Now I can.
- It opened up other possibilities. l have a career goal: go to MATC and get an Apprenticeship in carpentry. Before I would never have thought of getting into a field like that. It's awesome.
- I’m more caring to other folks’ feelings. If I can help someone, I will help them.
-I’m registered to vote now. I’m going to find out about Neighborhood Watch.
- The program gave me more confidence to continue with a career as a residential and building inspector.
- I’m still unemployed but I learned quite a bit. I’m doing jobs for the family.
-At the moment I'm working on my own house. I'll be signing up for courses and on track to pass the Accuplacer.
- I want to be an appliance technician. I decided during the Campus Tour. I'm not getting no younger. Other programs take too many years. I'm on track for a part-time job while training.
-I want an MATC degree in construction. PaCT classes give an advance view of what’s coming ahead. The program doesn’t last that long but it gets your foot in the door.