Monographs in Engineering Education Excellence
University of South Carolina College of Engineering and Information Technology
Gateway Engineering Education Coalition
Edward Ernst, University of South Carolina, Monographs Editor
A Continuous Quality Improvement System: An On-going Assessment Process within the College of Engineering and Information Technology at U.S.C.
Susan D. Creighton
Edward W. Ernst
Joseph H. Gibbons
Charles W. Brice
Francis A. Gadala-Maria
Jed S. Lyons
Anthony Steve McAnally
University of South Carolina
Number 4, December 2000
Monographs in Engineering Education Excellence
Edward Ernst, University of South Carolina, Monographs Editor
A Continuous Quality Improvement System: An On-going Assessment Process within the College of Engineering and Information Technology at U.S.C.
By:
Susan D. Creighton
Edward W. Ernst
Joseph H. Gibbons
Charles W. Brice
Francis A. Gadala-Maria
Jed S. Lyons
Anthony Steve McAnally
Published by the College of Engineering and Information Technology, University of South Carolina, Columbia, SC 29208. Address editorial correspondence to Edward Ernst, 3A12 Swearingen Engineering Center, University of South Carolina, Columbia, SC 29208; (803) 777-9017; .
Contents
Page
Preface 4
Background 6
College Assessment Infrastructure 8
College-Wide System 10
Assessment Plan 13
Assessment Methods 15
Quality Review Process 23
Program Assessment Structures and Processes 25
Mechanical Engineering Program 29
Chemical Engineering Program 35
Civil Engineering Program 42
Electrical Engineering Program 50
Computer Engineering Program 57
Appendices
AAssessment Plan 64
BSenior Survey 69
CSenior Survey reports (sample) 75
DCourse Survey 118
ECourse Survey reports (sample) 121
FAlumnae/Alumni Survey 128
GAlumnae/Alumni Survey reports (sample) 135
HFaculty/Staff Surveys 173
IFaculty/Staff Survey reports (sample) 181
JEntering Student Survey 196
KEntering Student Survey reports (sample) 201
LPerformance Assessment Instrument 223
MMid-Course Evaluation 230
NEducation Outreach Survey 234
OProfessional Communication Center Assessment Report 236
PLongitudinal Student Tracking Report (sample) 242
QBates House Project Report 251
RTemplate for Documenting Assessment Progress 260
Preface
Monographs in Engineering Education Excellence is a series of publications dealing with innovations in engineering education introduced at the University of South Carolina, with the support of the Gateway Engineering Education Coalition. The series seeks to make the information and ideas in the reports more accessible to engineering educators. It is hoped that other institutions will find the reports useful and adaptable to their own educational mission.
The Monographs in Engineering Education Excellence series includes a variety of genrestheses, dissertations, and technical reports, but all have the common objective of rethinking, reshaping, and revitalizing engineering education. This monograph, A Continuous Quality Improvement System: An Ongoing Assessment Process within the College of Engineering and Information Technology at U.S.C., discusses the college-wide assessment and CQI system developed to ensure the educational programs of the college are achieving the expectations held for them. The monograph presents examples and details regarding the tools, policies, processes, and procedures that have been developed and implemented in the college. These assessment/CQI efforts have evolved with support from the Gateway Engineering Education Coalition.
A broad agreement on the need for systemic educational reform exists within the engineering education community so those programs can provide the activities necessary to develop graduates who meet the new standards for the 21st century. The reform movement encourages more diversity in classroom practices that move instruction from a traditional lecture to structured activities reflecting what engineers do in the workplace. These initiatives promote changes in classroom practices to reflect the knowledge, skills, and abilities required by engineers to conceptualize, articulate, and implement a solution for engineering problems. The reform movement also advocates that engineering curricula incorporate a variety of teaching methods to involve students in active learning, design projects, technology use, and multidisciplinary teams. Outcomes-based assessments, in the form of design projects, portfolios, and model construction, enable faculty to link student competencies with the expectations of the workplace.
Believing in the need for change and recognizing that engineering is part of the growing national trend toward increased accountability, many accrediting organizations as well as national and state funding agencies, such as the National Science Foundation, have taken leadership roles in defining new parameters for engineering education. The paradigm shift is clearly evident in the new criteria adopted by the Accreditation Board for Engineering and Technology (ABET) which promote the use of outcomes assessment as the measuring tool for institutional and program evaluation. The stated goals of the ABET accreditation include: (1) providing graduates of accredited programs who are adequately prepared to enter the engineering profession; (2) stimulating the improvement of engineering education; and (3) encouraging new and innovative approaches to engineering education.
To achieve these objectives, the ABET Engineering Criteria 2000 stipulate that individual programs must have and have published educational objectives consistent with the mission of their institution. Programs must evaluate the success of students in meeting program objectives using appropriate assessment methodologies. The ABET criteria also require engineering programs to include a continuous quality improvement process. In this model, the program evaluation process documents progress towards achievement of objectives established by the engineering program and uses this information to improve the program.
Moreover, the criteria require that programs demonstrate student outcomes of such complex skills as the ability to design and conduct experiments, as well as to analyze and interpret data, the ability to design a system, component, or process to meet desired needs and an ability to communicate effectively. Types of evidence advocated by ABET to document these student outcomes include portfolios, design projects, nationally normed subject content examinations, and alumnae/alumni and employer surveys.
Criterion 2 of the ABET Engineering Criteria 2000 mandates a system that continually evaluates the programs to determine if program objectives are met and if they meet the needs of the program’s constituencies. The college developed and implemented a college-wide infrastructure with supporting policies, procedures, personnel and assessment tools to ensure the permanency and effective operation of the system.
The college-wide assessment system is linked with the continuous quality improvement processes initiated within each USC engineering program - Chemical, Civil, Computer, Electrical and Mechanical engineering. Together, the college-wide assessment processes and the program assessment processes comprise the USC COEIT Continuous Quality Improvement System.
The college-wide infrastructure provides the coordination and collaboration efforts needed to facilitate: (1) continuous cycles of program improvement; (2) the attainment of college goals and objectives; and (3) the achievement of state-level and accreditation agency performance indicators. The structure supports the personnel and resources necessary to maintain the flow of data, information and evaluation results through the system. It also serves as the focus for the triangulation and synthesis of data from different constituencies and various reports.
Background
Numerous reports over the past ten years have outlined the attributes that engineering graduates need to possess in the 21st century workplace [1]. The engineering education culture is shifting from one emphasizing individual specialization, compartmentalization of knowledge and a research-based faculty reward structure to one that values integration and specialization, teamwork, educational research and innovation. Institutions of higher education now focus on student outcomes or performance-based models of instruction that strive to measure what students have learned and what they can do [2]. Outcomes assessment examines the results of the education process by asking to what extent students have accomplished the objectives of their discipline.
There is broad agreement of the need for systemic educational reform within the engineering community so those programs can provide the activities necessary to develop graduates who meet the new standards for the next century. The reform movement encourages more diversity in classroom practices that move instruction from a traditional lecture to structured activities reflecting what engineers do in the workplace. These initiatives promote changes in classroom practices to reflect the knowledge, skills, and abilities required by engineers to conceptualize, articulate, and implement a solution for engineering problems. The reform movement also advocates that engineering curricula incorporate a variety of teaching methods to involve students in active learning, design projects, technology use, and multidisciplinary teams. Outcomes-based assessments, in the form of design projects, portfolios, and model construction, enable faculty to directly link student competencies with the expectations of the workplace.
Believing in the need for change and recognizing that engineering is part of the growing national trend toward increased accountability, many accrediting organizations as well as national and state funding agencies, such as the National Science Foundation, have taken leadership roles in defining new parameters for engineering education. The paradigm shift is clearly evident in the new criteria adopted by the Accreditation Board for Engineering and Technology (ABET) which promote the use of outcomes assessment as the measuring tool for institutional and program evaluation. The stated goals of the ABET accreditation include: (1) providing graduates of accredited programs who are adequately prepared to enter the engineering profession; (2) stimulating the improvement of engineering education; and (3) encouraging new and innovative approaches to engineering education [3].
To achieve these objectives, the ABET Engineering Criteria 2000 stipulates that individual programs must have published educational objectives consistent with the mission of their institution. Programs must evaluate the success of students in meeting program objectives using appropriate assessment methodologies. The ABET criteria also require engineering programs to include a continuous quality improvement process. In this model, the program evaluation process provides documentation of progress toward achievement of objectives established by the engineering program and uses this information to improve the program.
In addition, the criteria require that programs demonstrate student outcomes of such complex skills as the ability to design and conduct experiments, as well as to analyze and interpret data, the ability to design a system, component, or process to meet desired needs and an ability to communicate effectively. Types of evidence advocated by ABET to document these student outcomes can include portfolios, design projects, nationally normed subject content examinations, focus groups, and surveys of alumnae/alumni, students and/or employers.
College Assessment Infrastructure
As engineering classroom practices change, the evaluation of student development and program effectiveness must align with the new ABET emphases. Criterion 2 of the Criteria 2000 specifies that programs must have published educational objectives that are consistent with the mission of the institution. It also mandates a system that continually evaluates to determine if program objectives are met and if they meet the needs of the program’s constituencies. To this end, the University of South Carolina College of Engineering and Information Technology (COEIT) developed and implemented a college-wide infrastructure with supporting policies procedures, personnel and assessment tools to ensure the permanency and effective operation of the system.
The college-wide assessment system is linked with the continuous quality improvement processes initiated within each USC engineering program - Chemical, Civil, Computer, Electrical and Mechanical engineering. Together, the College-wide assessment processes and the program assessment processes comprise the USC COEIT Continuous Quality Improvement System. Both parts of this system are integrated within the College Strategic Plan. As seen in Figure 1, this plan connects the College to its institution through the statement of University of South Carolina’s vision, mission and goals.
Figure 1. Overview of COEIT Continuous Quality Improvement System
The purpose of the continuous quality assessment system is to continually assess the needs of the program’s various constituencies to ensure that the programs are achieving expectations as described by the objectives and to evaluate how effectively each program and the College have moved toward achieving stated mission and goals. Assessment processes show faculty, staff, administrators and others where improvements seem to be appropriate and guide the implementation of change within each program and college-wide service areas. Changes are monitored and re-evaluated to determine what improvement has been realized. Thus, the system is an ongoing evaluation of the effectiveness of the College and its programs.
The following sections will discuss both the College-wide system and the program systems. Examples and details will be given regarding the tools, policies, processes, and procedures that have been developed and implemented at USC COEIT to ensure the institutionalization of the CQI System.
Note. This monograph is a snapshot of the status at the end of the spring Semester, 2000. The CQI processes are relatively new and continue to change.
College-wide System
The College-wide infrastructure provides the coordination and collaboration efforts needed to facilitate: (1) continuous cycles of program improvement; (2) the attainment of college goals and objectives; and (3) the achievement of state-level and accreditation agency performance indicators. The structure supports the personnel and resources necessary to maintain the flow of data, information and evaluation results through the system. It also serves as the focus for the triangulation and synthesis of data from different constituencies and various reports.
The comprehensive character of the college-wide assessment structure is evident in the following diagram.
Figure 2. College-Wide Assessment Infrastructure
The diagram shows the integration of state and institutional parameters within the system. It also highlights the linking of college assessment processes to its departmental programs. A more comprehensive view of the departmental assessment processes within this continuous loop system is discussed in a later section.
The personnel and processes of the college-wide assessment infrastructure, however, are the focus of this diagram. The College-wide infrastructure consists of several formal, key components: College Executive Committee, Center for Engineering Education Excellence, the Center for Engineering Education Excellence Team, Assessment Director, Departmental Assessment/Education Committees and its various constituencies.
A brief overview will outline the responsibilities of each component and provide insight into how these personnel and committees interact to produce a continuous quality improvement process.
Executive Committee
The Executive Committee is composed of the Dean, Associate Deans, Departmental Chairpersons and the Center for Engineering Education Excellence Director. This committee meets at two-week intervals and provides oversight and decision-making duties for the College.
Center for Engineering Education Excellence
The Center for Engineering Education Excellence is an interdisciplinary organization of individuals who collaborate in the effort to promote self-study, innovation and reform within the College. The staff and support personnel involved in the Center include: the Director for the Center; a Program Coordinator, the Assessment Director, the Director of the Professional Communications Center and the Ethics Coordinator.
The mission of the Center includes all the major parts of engineering education: undergraduate, graduate, and research; and promotes meaningful integration of engineering education. The educational goal of the Center is to graduate students that understand the technology content of engineering as well as the social, political, ethical, environmental and economic context.
The objectives for the Center have both an internal and an external thrust. These objectives include:
- Development of students as engineering professionals with the motivation, capability and knowledge
base for career-long learning
- Emphasize effective teaching/learning strategies for all types of students
- Promote effective and (time) efficient student/faculty interaction
- Enhance the continuous quality improvement process (CQI) within the College
- Serve the engineering education community by encouraging innovation and reform
- Increase the visibility of USC to the engineering education community
- Provide a channel for learning about innovation in engineering education at other schools
Center for Engineering Education Excellence Management Team
The Center for Engineering Education Excellence Team provides the opportunity for collaboration among the programs, discussion of issues, planning activities, and making recommendations for college-wide initiatives. The committee consists of a Chairperson (Director of the Center), the Assessment Director, the Associate Dean for Academic Affairs, the Director for the Professional Communications Center, the Ethics Coordinator, and one faculty representative from the Chemical, Civil, Computer, Electrical and Mechanical programs. The biweekly committee meetings serve as one focal point for the distribution and discussion of report findings and information. Committee members then share this information with the appropriate committees within their individual departments.
The members of the Center for Engineering Education Excellence Team have been the primary personnel involved with the initial organization and maintenance of the assessment structure. Meeting on a weekly basis, the team addressed a range of issues relating to the implementation of a continuous improvement program. Substantive tasks accomplished by the Committee include: