University of South Carolina Beaufort
Institutional Effectiveness Report 2001-2002
Introduction
This report includes: General Education, Achievement of Students Transferring from Two to Four-Year Institutions and Policies and Procedures to Ensure that Academic Programs Support the Economic Development Needs in the State by Providing a Technologically Skilled Workforce.
A reporting schedule for USC Columbia and Regional Campuses can be found at http://kudzu.ipr.sc.edu/IEReports/a629schd.htm.
Assessment of General Education
USCB is examining new ways of measuring effectiveness on its objectives, including standardized measures (please see attachment.) It is also refining its objectives. Indeed, USCB’s recent mission change to baccalaureate-degree granting status will open new opportunities for assessment and interventions:
· Off-the-shelf Measures Now More Appropriate: USCB initiated a regional campus system-wide review of “off the shelf” general education assessment instruments. USCB Academic Dean and Institutional Research Director, along with David Hunter of the USC regional campuses office, presented the findings at a national conference featuring, in particular, three “outcome” measures (ACT’s CAAP, Missouri’s College Base, and the ETS Academic Profile) as well as a “process” measure on student engagement, the NSSE. A summary of these instruments (prepared for these sessions) is attached.
· Other Summative Measures Now Available: USCB will now be able to institute consistent assessment at appropriate times and make possible other means of assessment, such as: electronic portfolios, capstone exams and courses, GRE, senior papers (and other summative writing exercises), graduation (exit) requirements, etc. This expanded range of approaches now available to USCB (everything from off-the-shelf measures to capstone courses) is made possible by opportunities for course embedding. For example, the extensive effort in collecting and compiling portfolios will be motivated as requirement for baccalaureate degree.
· Opportunities to Measure Four-Year Growth: The truest picture of general education development comes at the end of the four years of study. As a regional 2-year campus, USCB was able to get only rough idea of basic instructional quality. Ability to examine 4-year students will enable the campus to more accurately pinpoint areas of weakness and identify courses for intervention.
Indicators: Multiple Measures
USCB garners feedback on its program effectiveness through multiple measures of success, including such things as: Transfer Student Success: USCB continues to track GPAs of students who transfer to other colleges and universities. This serves as a measure for the basic liberal arts courses: writing, research, and other liberal arts basics are developed in coursework on which USCB has traditionally focused. (Findings: USCB transfer students outperform students from all other regional campuses and even students who began their college careers at the main USC campus – confirming the reputation for rigor among the regional campuses.)
· Alumni Tracking, in which USCB surveys its alumni every two years for feedback on general education instruction; and MFAT test of Business Majors. (Business majors among USCB students continue to score as high or higher than their peers at sister institutions on this evaluation of academic skills.)
In this report, we discuss the most recently-administered device, University of Kentucky Community College System Self-Report Survey, through which USCB explores statistically significant correlations between semester hours at USCB, on the one hand, and (perceived) gains in general education objectives. The instrument has rough correlation with USCB objectives determined in a recent faculty workshop, and also indicates effectiveness on additional (maturation) dimensions. (Note attachment.)
The Results Chart (attached) shows analysis of the data through three filters:
o Total Semester Hours at USCB/Elsewhere: Correlations between time spent at other institutions, on the one hand, and learning achievement attributed to USCB, on the other, will indicate strengths/weaknesses in the USCB program.
o Discipline-by-Discipline Semester Hours: This gives us a real sense of WHERE student development gains take place. If the USCB program is truly effective, key general education competencies must be instilled in students at multiple points. (In a gratifying way, teaching of certain desired competencies did seem to appear “across the curriculum.”) Also, we measured impact this way for reasons of discern ability: If genuine gains are taking place, these gains – which students generally attribute to USCB in this wording -- must correspond to actual involvement in disciplines (as measured in number of semester hours in that discipline) for USCB to know if survey responses are meaningful.
o Military Status: Students also develop skills in the Military. Those students may feel their achievement is best attributed elsewhere.
For this self-report measure, the campus used a previously-administered instrument, as stated above. This questionnaire was administered to all students at all campuses at one (“snapshot”) class time.
This sampling method yielded the following number of valid survey responses:
Stratification of Students Surveyed: By Campus Spring 2002
Valid Surveys / frequency / percent / Valid Percent / Cumulative PercentDowntown
/ 76 / 55.9 / 55.9 / 55.9Hilton Head / 43 / 31.6 / 31.6 / 87.5
Military Bases / 17 / 12.5 / 12.5 / 100.0
Total / 136 / 100.0 / 100.0
This sampling is roughly proportional to the student populations on each satellite campus location: More importantly, though, the “snap shot” sample represents a cross-section along a variety of course-levels and student profiles.
Findings:
The attached table displays results of the Pearson Correlation (P) and Multivariate (MR) analyses which correlated response on the general education component with SEMESTER HOURS:
· simple (bivariate) correlations between variables represented in the first two columns (measures of college wide semester hours); asterisks and
· multivariate analyses enabled us to look at the independent contribution of each discipline on each of the general education objectives.
· multivariate analyses were used to determine the mitigating impact of military service. (This confirmed general assumption on campus that – especially with “maturation” objectives – students who are active duty do not attribute learning in some areas to USCB.)
Interpretation of Results:
Student-identified strengths correspond to traditional liberal arts emphases of USCB:
Strengths: Students agreed that their “college experience at the University of South Carolina-Beaufort has strengthened [their] ability to:
· utilize my abilities, resources and talents in community activities.”
· better understand international issues.” (USCB faculty, including those who teach Government and International Studies, probably do a much better job than those encountered in the prior instructional experiences of our students.)
· enhanced my concern for the environment.” (USCB has a strong marine studies program, including field trips around its own barrier island.)
· understand the contribution of science and technology to society,” and,
· relate to people from different cultural backgrounds.” (Both of these are strongly associated with higher education missions uniquely emphasizing the liberal arts.)
Three major findings emerged as we more deeply “mined” the data:
1. Active-Duty military status made students less likely to attribute growth to USCB in some areas: primarily in student (e.g. emotional) development and technology training. (See Chart below.)
2. The comparison of ratings between transfer students, on the one hand, and those whose college experience was exclusively USCB, on the other, was illustrative. It appears even transfer students agree that USCB has strengths in several areas of traditional liberal arts objectives, including “concern for the environment,” and in a “usefulness to the community” item. Potential weaknesses include: technology skills, developmental issues, values instruction, and certain writing skills. (Please see chart attached – weaknesses perceived by transfer students marked with asterisks.)
3. Disciplines: Findings of correlations between hours in specific disciplines and various general education components indicated:
· Gains in certain objectives not usually associated with specific disciplinary areas provide evidence of a sort: these correlations in unpredictable yet explainable disciplinary areas -- along with gains in predictable areas (such as math semester hours and perceived gains in math proficiency) – tend to validate the instrument itself as they reveal a consistency of institutional mission of liberal arts: revealing “across the curriculum” efforts to reinforce specific general education skills.
· The attached chart (Attachment 1), showing the “independent” contribution of individual disciplines to specific learning objectives, were reviewed by faculty to direct recommendations for course-level intervention.
·
Use of Assessment Findings
Faculty leaders have reviewed the findings from the Spring 2002 administration of the University of Kentucky instrument. These data will continue to be analyzed by faculty in the weeks and months ahead.
Also, the faculty will continue to review and refine objectives by which to select a primary instrument for the general education program at USC Beaufort, with these concerns in mind:
· Compatibility with USCB Objectives.
· Usability of data: Division chairs must be able to focus improvement on levels of individual interactions where change can best be made.
· Credibility with faculty.
USCB will continue the process of selecting among measures currently under review.
In turn, the new instruments will themselves inform development of General Education objectives at USCB. While faculty will not adjust objectives to established tests if this means that they 'teach to the test,' the availability of reliable data has such tremendous value for ensuring quality that the deference to well-researched choice of indicators and basic skills is defensible. Simply put, the campuse maintains that national “standards” will be useful for determining weaknesses and directing improvement efforts.
In sum, the campus will soon develop a matrix comparing various instruments on assessed learning objectives. From this analysis, the faculty will select a measure for future implementation on campus.
Achievement of Students Transferring from Two to Four-Year Institutions
Transfer reports for the USC Regional Campuses can be found at http://kudzu.ipr.sc.edu/IEReports/transfers/tran2002.doc
Policies and Procedures to Ensure that Academic Programs Support the Economic Development Needs in the State by Providing a Technologically Skilled Workforce
This year, USCB gained re-accreditation through an “alternative” SACS review process which emphasized Information Technology. USCB’s emphasis is in evidence in other ways, as well:
· Routine general education assessment at USCB has always included special analyses of IT skills taught on campus. (Please note links under “computer” on http://www.sc.edu/beaufort/facts/assess97.htm)
· General Education objectives state special emphasis on IT objectives (excerpt below taken from campus-wide effort to coalesce USCB general education objectives:)
ELECTRONIC COMMUNICATIONS
GOAL: AA/AS graduates will demonstrate effective use of computers across the curriculum.
LEARNING OBJECTIVES / OUTCOMES:
Students will demonstrate:
o Ability to access electronic information and databases
o Mastery of basic computer operating system functions
o Utilization of basic software programs (e.g., word processing, spreadsheets, etc.)
o Sending and receiving electronic communication
o Creating and editing written texts and integrating graphical and statistical information into them
· Special inventory was undertaken on campus in 1999 of IT objectives taught on campus. Full results of this inventory are seen at: http://www.sc.edu/beaufort/facts/teamobjs.htm#elec printed here.)
ATTACHMENT 1: Results Chart of Survey of 136 USCB students in 2002
find, evaluate, and organize information logically. / * / * / * / * / * / * / * / * / * / * / *
intellectually solve mathematical problems. / * / * / * / * / * / * / * / 0.4 / * / * / *
understand the contribution of science and technology to society. / .23 / .24 / * / * / * / * / * / * / * / * / * / * / *
reach my own conclusions after evaluating information. / .27 / * / * / * / * / * / * / * / 0.2 / * / * / *
keep my choices and behavior consistent with sound moral principles. / * / * / * / * / * / * / * / * / * / * / * / *
consider ideas different from mine. / * / * / * / * / * / * / * / * / 0.2 / * / * / *
write using logical organization. / * / .26 / * / * / * / * / * / * / * / 0.2 / * / * / -0.2
apply new knowledge. / * / .24 / * / * / * / * / * / * / * / * / * / * / *
define my own values and beliefs. / * / * / * / * / * / * / -0.2 / * / * / 0.3 / * / -0.3
think in new ways about familiar subjects. * / .26 / * / * / * / * / * / * / * / * / * / * / *
relate to people from different cultural backgrounds. / .23 / * / * / * / * / * / * / * / * / * / * / * / *
better understand international issues. / .2 / .26 / * / 0.9 / * / * / * / * / * / 0.7 / * / * / *
be accountable for my actions. / * / * / * / * / * / * / * / * / * / * / *
speak using correct grammar. / .21 / * / * / * / * / * / * / * / * / * / * / *
reason and think logically. / * / * / * / * / * / * / * / * / * / * / *
read with increased comprehension. / * / * / * / * / * / * / * / * / * / * / 0.0
set goals for myself and pursue them. / * / * / * / * / * / * / * / * / * / 0.4 / *
demonstrate my creativity through writing. * / * / * / * / * / * / * / * / * / * / * / -0.4
listen more effectively. / * / * / * / * / * / * / * / * / * / * / 0.5 / *
develop the knowledge and skills required for professional growth. * / * / * / * / * / * / * / * / * / * / 0.2 / *
be more open to new ideas. / * / * / * / * / * / * / * / * / * / * / * / *
interact in small groups. / * / * / * / * / * / * / * / * / 0.2 / * / * / *
recognize moral conflicts and identify means of resolving them. * / .25 / * / 0.7 / * / * / * / * / * / 0.5 / * / * / *
speak more confidently. / .28 / * / * / * / * / * / * / * / * / * / * / *
develop the computer knowledge and skills required for professional growth. * / .46 / * / * / * / * / * / * / * / * / * / 0.4 / *
enhanced my concern for the environment. / .21 / .34 / * / * / * / 0.2 / * / * / * / * / * / * / *
originate new ideas on my own. / * / * / * / * / * / * / * / * / * / * / *
better understand social issues. / .19 / * / * / * / * / * / * / * / * / * / * / *
intelligently solve problems. / .21 / * / * / * / * / * / * / * / * / * / * / *
appreciate the values and beliefs of others. / * / 0.2 / * / * / -0.3 / * / * / * / * / * / -0.3
write using correct grammar and punctuation. * / .26 / * / * / 0.4 / * / * / * / * / * / * / * / *