Collecting Data for Special Populations:
VTEA Supplemental Data Collection Study - Early Results
Collecting Data for Special Populations:
VTEA Supplemental Data Collection Study
Early Results
Wednesday, August 14, 2002
Please be aware this is a draft document.
Comments are welcome.
Send comments to Dr. Chuck Wiseley
Table of Contents
Introduction: 1
Background: 1
Study Methodology: 2
Early Results: 2
Impact on Economically Disadvantaged Counts 4
VTEA Survey Administration Process 4
Sample Questions: 6
Introductions - Directions: 6
Economic Disadvantagement: 7
1. BOG fee waiver questions: 7
2. Forms of public assistance: 8
Single Parent / Single Pregnant Women: 9
Displaced Homemaker: 10
Summary: 11
Appendix A - Email Survey Sent to RP Group Listserv 12
Appendix B – Students Identified Using SV03 Only 13
Appendix C – Respondent Colleges - Students Identified Using SV03 Only 19
Appendix D - Envelope Instructions for Faculty 21
Appendix E – Letters to Faculty 22
Letter #1 22
Letter #2 23
Letter #3 24
Collecting Data for Special Populations:
VTEA Supplemental Data Collection Study - Early Results
Introduction:
The Vocational Education Services Team (VEST) recently undertook an informal study of how colleges collect the data to report the special population categories required for the Carl D. Perkins Vocational and Technical Education Act (VTEA) reporting. The intent of the study is to provide an understanding of some common practices in the VTEA data collection effort, the effect of supplemental collection on funding and possibly provide exemplary practices and samples.
Although the study is in an early stage, some preliminary findings are available. As part of the study, some exceptional classroom and registration surveys were made available for VEST review. A sampling of questions to determine special population status are included in the report. Some of the highlights from this preliminary report are:
· Most of the information needed for VTEA can be asked in four questions
· Questions at registration:
o are least obtrusive to faculty and student learning
o remove administration logistics nightmares of surveys in all vocational classrooms
o facilitates expansion to all students for research purposes
· If the data is for special population status only, simple questions work well
· Surveys often included questions for both VTEA and Matriculation
· If the data is used to determine services needed by students or other research, more detailed information is often required
· Eligibility for BOG fee waiver can be used as a financial aid marketing tool
· Statewide nearly 16% of economically disadvantaged vocational students are identified only by SV03 (as self-reported)
Background:
Although this report discusses the student supplemental data surveys used for collecting data that is reported in the VTEA accountability reporting system, the VEST recognizes that the intent of the collection and the benefits to students and colleges is in the local use of the data collected. The benefits come out of local colleges identifying specific needs that might be addressed to support students in their educational pursuits. Often, identifying inclusion in certain special populations can provide insights into unmet needs when outcomes for those groups suffer. The VTEA supplemental data collection surveys are one of the mechanisms for more accurately identifying inclusion in special populations.
A number of the special population categories required for the VTEA Core Indicator reports are temporal in nature. The status of students may change from term to term in categories such as Single Parent / Single Pregnant Women, Displaced Homemaker, and Economically Disadvantaged. Additionally, colleges were provided an expanded set of criteria for identifying Economically Disadvantaged students in the approvedCalifornia Perkins State Plan that allows for an expanded set of criteria. The plan included a “Request for Waiver of Section 132 Funds” to allow the state to use more than just the Pell grant to identify economically disadvantaged students as specified in the law in Section 132, Postsecondary and Adults. Thewaiver[1] includes the criterion below for identifying economically disadvantaged students:
#8. An adult who is eligible for economic public assistance or student fund aid and/or an annual income below the poverty line as defined by the county of eligibility
The ability to identify whether a student is “eligible for” rather than “in receipt of” financial aid or public assistance requires information typically not available from information collected at registration or through other administrative data. Many colleges began collecting this information in vocational education classroom surveys in the early years of Perkins II (1990) when the collection and reporting by special populations was implemented. Colleges also use these supplemental collections to augment their economically disadvantaged counts in order to maximize their Perkins funding.
More recently, as documented in this study, colleges have begun to move to a more expedient method of asking a few additional questions at registration where the student has already identified themselves and their course requests. More importantly, however, as colleges used the information to identify where special population students might need additional services to help them succeed in their educational pursuits, some colleges expanded the administration of the survey questions to include all students rather than only those enrolling in vocational courses.
Study Methodology:
The first round of questions were sent via email to the Research and Planning Group listserv. That email with the survey questions is included in attachment A. The first round is complete at this point in the study and the results of the review of those responses follow. The early results will be shared with the research community.
Since college research staff are often not involved in the vocational education classroom survey development or administration, a second round will consist of sending the survey and the preliminary results to the AOE-ALL list for confirmation and updating.
For those colleges responding, some analysis of impact of funding will be provided. At each point in the study that new information is added, the analysis will be updated. The final step will be to develop a guidance paper on collecting supplemental data for VTEA. That paper is planned for release in fall 2002.
Early Results:
Of the 36 colleges responding in the first round, 86% of the colleges (31) used some form of survey or supplemental collection to identify some special population students. Twenty colleges, 56% of those responding, used classroom surveys administered in vocational courses each semester. Eight colleges, 22%, had moved to (or were implementing in the coming term) surveys administered at registration. Five of the eight collecting the data at registration were collecting it for all students. Three colleges were collecting special population data through supplemental surveys within student services and financial aid.
Colleges Responding: 36
Percent of
Colleges Respondents
Vocational Classroom Survey: 20 56%
Vocational Students at Registration: 3 8%
All Students at Registration 5 14%
Supplemental Data collected in Student Services: 3 8%
Total Collecting Supplemental VTEA data 31 86%
All of the responding colleges used some form of electronic or automated data capture for their questions. For those who supplied their surveys or the questions (20 colleges), the most common special population statuses being collected were: economically disadvantaged (20), single parent/single pregnant women (10), and displaced homemaker (10). The economically disadvantaged status, however, was determined differently on nearly every survey. Additionally, each of the surveys had a statement to the student and all but one included, along with a purpose and confidentiality statement, the importance of accuracy for continued VTEA funding.
All of the colleges supplying their surveys collected some additional information beyond VTEA needs. Nearly all collected either matriculation goal or major. A few colleges collected information to identify Tech Prep students. Others continued to collect information no longer collected by the state such as Dislocated Worker status and Expected Hours of Employment. Whether or not the data not requested by the state is used locally or for some other reporting or research effort is not apparent from this study.
Interestingly, some college surveys asked questions that were marketing services as well as supplying eligibility information. One creative phrasing of a family size and income combination question, to determine BOG fee waiver eligibility, ended with a referral to financial aid if the student met the criteria. A number of colleges not only collected student services usage in a number of areas but also provided for information requests on the survey. One survey even alerted the respondent to employment placement services in a question about plans after completion of the program.
The apparent trade off being weighed in each survey (as in most all survey instruments) is the time to complete the survey and the value of more detailed data being collected. The more information requested the more reticent the respondent is to complete the survey (and the faculty to administer it). However, the more detailed data can provide a better understanding of the needs of those special populations that we serve.
Questions asked at registration seemed the least obtrusive to the educational processes of the college and many colleges already ask matriculation questions at registration. The move from classroom surveys to supplemental data collection at registration involves a number of pieces (i.e. online, telephone and hardcopy forms) and has impact on the district information technology staff. The workload and logistics of doing classroom surveys every semester, however, far exceed the workload of adding 3 or 4 VTEA items to the data already collected at registration for matriculation and other college research efforts.
Impact on Economically Disadvantaged Counts
The impact on “Economically Disadvantaged” counts of the various methods of collecting data can not be clearly drawn from MIS data at the college level but some impact on the numbers is evident generally. This section of the analysis assumes that data collected through supplemental data collection is reported in the MIS data element “STUDENT-VTEA-ECONOMICALLY-DISADV-STATUS” (SV03). The difficulty in determining impact at the college level lies in the marketing and informational aspects of the collection instruments provided by the respondents. Some colleges providing high numbers of economically disadvantaged status in SV03 (over 3,000 per term at one college) did not identify many economically disadvantaged students beyond those identified through financial aid and other sources. Most of those colleges, however, consciously marketed financial aid and other services in their survey instruments.
Generally, the impact of collecting data through the various methods is however dramatic. Statewide, 15.92% of the economically disadvantaged vocational students were identified by SV03 only and would not have been identified by other criteria. For the group of colleges collecting data at registration, 22.3% of the economically disadvantaged students were identified by SV03 only. For those collecting data using classroom surveys, 15.8% of the economically disadvantaged students, identified by SV03, would not have been identified by other criteria. Those not collecting supplemental data and those collecting supplemental data only in student service areas identified only 0.25% of students through SV03 alone. The table below summarizes those results:
Type of Supplimental Data Collection / Total Identified w/ SV03 Only / College Average Identified w/ SV03 Only / Average Increase Using SV03 / Minimum Increase Using SV03 / Maximum Increase Using SV03Registration / 22.33% / 21.71% / 34.23% / 0.00% / 155.43%
Classroom Survey / 15.84% / 14.25% / 19.83% / 0.00% / 67.73%
Neither / 0.25% / 0.33% / 0.33% / 0.00% / 2.00%
*Note: Includes data only for those responding to the email survey. See Appendices B and C for complete listing.
VTEA Survey Administration Process
Additional pieces of information provided by some folks in the email survey responses included comments on the survey administration process, logistics, and letters to faculty and deans. Although only a few of the respondents included comments on the process, a summarization of the comments and a sampling of a few merit inclusion here if just to elicit further discussion. Also in the samples are key parts from a few of the letters to instructors.
1. Buy-in of faculty was necessary in all cases for classroom surveys. A few responses mentioned return rate from instructors as a continuing problem.
2. Faculty seemed less opposed to short surveys they would administer than an outsider doing anything in their classroom.
3. Faculty seemed to elicit higher quality responses than “outsiders” administering the survey in the classroom.
Sample comments on administration:
One respondent had tried two administration methodologies and included the following appraisal.
1. Student ambassadors visiting classroom - high rate of compliance (95% of classes surveyed)
-quality of responses poor (instructors resented interruptions - students
picked up on this, treated survey carelessly)
2. Delivered to faculty via campus mail, relied on faculty to administer to class, then return via campus mail.
- lower rate of compliance (70%)
- higher quality of responses (more complete, compliance with SSN)
Another respondent included commentson faculty administration of the survey.
Best to have the faculty administer it due to time and logistics. Also, some faculty are a wee bit sensitive about someone else coming into their class. I would suggest that each college needs to approach their Academic Senate and get their feedback on the process.
Another respondent included commentson getting buy-in from the instructional deans as well as the faculty.
In the initial terms of this process we told the deans informally each additional survey that meets the criteria of a correct ssn and at least one yes answer means an additional $100 for the district (based on our increased funding and the increased number of students we were able to identify).
A number of respondents also included their letter to the instructors which often provided a short script for the faculty (see appendix B for sampling of letters). The letters are designed to help get both faculty and student buy in.
The following is a suggested script to offer your students.
The purpose of collecting the information on this survey is to provide the State of California with data that supports important funding for occupational programs at CCSF. In the past, these funds have been used for purchasing new computers and software to update labs; audio-visual equipment; healthcare materials, new technology for numerous programs; support for classroom assistants, tutors and much, much more. By completing this survey you are helping to improve the quality of education at CCSF…