Executive Assessment Summary, 1998-99
Executive Summary of Assessment of Student Learning and Development
Number 2
1998-99
“The key purposes of assessment are to ask important questions about student learning,
to get some meaningful information on these questions,
and to use the information for academic improvement” (Rossman & El-Khawas, 1987).
What Is Assessment?
When we think of “assessment,” we typically think of “tests,” “grades,” “evaluations,” and so on. In higher education today, the term “assessment” has come to mean much more. It goes beyond testing and grading of students by professors. It is more than evaluation. In the words of one researcher, assessment, as it is understood by most in higher education today, is “the measurement of the educational impact of an institution on its students” (Terenzini, 1989).
The goal of the assessment process is the improvement of all institutional practices that affect student learning and development, from the classroom to the athletic field. That is a monumental task indeed. It begins with setting clear objectives for student learning and development that are consonant with the University’s mission statement. It continues with the gathering of meaningful information to measure the accomplishment of institutional, departmental, and classroom objectives. The assessment “loop” is complete only when the information gained from this process informs planning and decision-making in each and every department and results in constructive change.
Why Do Assessment?
Effective assessment programs have been mandated for public institutions by more than two-thirds of the states and are required by all six of the higher education accrediting associations. The North Central Association, which accredits Concordia University, St. Paul, requires institutions it accredits to provide evidence of the effectiveness of their educational programs (North Central Association of Colleges and Schools Commission on Institutions of Higher Education, 1996).
But, accountability mandates aside, the essential goal of assessment is to improve the quality of student learning by improving the practices that impact learning. Improved institutional practices result in institutional excellence. And institutional excellence benefits all stakeholders: students and their families, faculty and staff, the church and the community (Wolff & Harris, 1994).
Relationship of Assessment Activities to the Evaluation of Institutional Effectiveness
The assessment of student learning and development, which is covered in this document, is only one aspect of the evaluation of institutional effectiveness (Lopez, March 1996). The evaluation of other aspects of institutional effectiveness is carried out by department chairs, deans, program directors, and vice-presidents as they complete comprehensive program reviews and other activities to evaluate the achievement of Concordia University’s strategic goals.
Assessment at Concordia University, St. Paul
The development of the assessment process at Concordia University, St. Paul was outlined in last year’s executive summary of assessment activities and is also found in the self-study document written for the 1998 North Central Association accreditation visit. The 1997-98 academic year was the inaugural year for the new semester curriculum and integrated assessment plan. Last year (1998-99) was the second year for this new curriculum and its associated assessment activities. The following sections summarize the ongoing departmental and institutional activities that assess student learning and development.
Learning and Development in the Academic Major
Academic departments documented their implementation of assessment and their use of assessment results for improvement by means of the annual assessment report for each major. These reports consist of four sections (see Appendix I). The first section outlines the expected learner outcomes, or goals and objectives, for that major. The second section indicates which assessment activities were conducted that year for students in that major. The third section reports the actual results of those assessment activities, that is, how students as a group actually performed when measured against the goals and objective for their major. Finally, the fourth section reports how those results were used by the department to improve or strengthen curriculum and instruction in the major.
Assessment reports for 1998-99 were completed for 29 undergraduate academic majors and three graduate programs. All departments in all colleges submitted assessment reports, which is an increase over last year. These reports will be reviewed by the University assessment team, which will provide feedback to academic departments using North Central Association guidelines (Lopez, March 1996) and principles of good assessment practice (American Association for Higher Education, 1992). This process is intended to lead to improvement in the assessment of student learning and of program curricula and procedures at the department level. These reports will be retained in the office of assessment and used to document the progress of assessment activities.
Overall, the quality of the annual assessment reports for the major improved. The reports in general were more focused, with irrelevant material eliminated. Faculty collaboration and their discussion of assessment activities and results were more clearly documented. The reports have made very evident that faculty are seriously engaged in reflecting on their practice and improving the curriculum and student performance. Many departments sought feedback from students about their academic program, in addition to standard course evaluations. At least two departments involved students by having them actually create and administer surveys or interviews, asking fellow students in the department about issues that were important to them. The delineation of clear goals and objectives for students to achieve in their majors continues to be a strength of the assessment process. The effective assessment of student achievement of those objectives is less consistently accomplished, but in most cases is being thoughtfully deliberated and improved upon. The reports suggest that departments with ambitious assessment plans find it difficult to implement them completely due to limited resources and time. In academic majors where further development of assessment is still needed, the assessment team will provide input and informational resources.
Brief summaries of 1998-99 assessment activities for majors, as reported in the annual assessment reports from academic departments, follow:
College of Arts of Sciences
Department of Art
Students with majors in the art department develop portfolios of their work, which culminate in an exit exhibition that is evaluated by peers and professors. Students were evaluated on the quality of their work and its professional presentation, and also on their level of understanding of it and its context. Faculty reported that their expectations regarding students’ exit portfolio exhibitions and expression of understanding were met or exceeded by students in 1998-99. According to these evaluations, all four graduates in the program demonstrated competencies in studio art production and professional presentation and had developed significantly in at least one studio discipline. In addition, students were informally interviewed about their experiences in the program, at which time 100% of them expressed satisfaction with the preparation they had received. Faculty noted that two of them had already been accepted into graduate programs. Faculty reported that their discussion of these outcomes and student feedback has led to significant revisions in the curriculum, making it more focused and concise. These results also indicated weak areas such as computer graphics, lack of studio space, and poor quality of some equipment and ventilation, which created safety concerns. Some concern was expressed by students regarding quality of instruction in two areas taught by adjunct professors, and the department has responded by requesting a tenure-track faculty replacement for those areas.
Department of Biology
The assessment plan for majors in biology and in environmental science takes place primarily through course projects and activities. In addition, the environmental science majors complete internships in their field, and take a national standardized test in biology. This year no assessment report was completed due to significant faculty turnover in the department. This same turnover has made implementation of assessment and use of results more difficult, but no doubt this will improve with the addition of faculty and their continued discussion of departmental vision and expectations.
Department of Business
The department of business offers three majors: accounting, marketing and management, and finance. The department had a preliminary accreditation visit by the Association of College Business Schools and Programs (ACBSP) and was granted candidacy status. In the accounting major, assessment activities included individual conferences with beginning students, on-going coursework and course exams, course evaluations, demonstration of competency in Excel, capstone course project in auditing, and a standardized exam in business (Major Field Test). The results of these assessments showed widely divergent performance in lower-level courses, with more consistently high performance in upper-level classes. Other results indicated student strengths in the auditing capstone case, and weaknesses in statistics, spreadsheets, and database skills. In response to these results, faculty have added MAT102: Statistics to the required business curriculum and have added a computer lab component to the introductory accounting classes.
In the finance major, assessment activities included course exams, income tax research problems, financial analysis of a major US corporation, a standardized exam in business (Major Field Test), and course evaluations. Results showed widely divergent performance on course exams, above average performance on the financial analysis project, and average performance on the standardized test. Students showed weaknesses in statistics, cost accounting, and spreadsheet skills. In response to these results, faculty made modifications to finance courses, added the MAT102: Statistics course, added a spreadsheet/computer lab, added the Cost Accounting course to the finance major and dropped Intermediate Accounting I, and developed additional assessment measures.
In the marketing and management major, assessment activities included on-going coursework and course exams, mid-term and final course evaluations, ongoing use of classroom assessment techniques, satisfaction surveys and interviews, and overview of student performance in statistics. Some surveys served a dual purpose of providing faculty with feedback from students and of providing a learning application for students in the major, who developed, administered, and analyzed the surveys. In addition, faculty surveyed course requirements for a marketing and management major at 30 other institutions as part of the process of reshaping the curriculum and preparing for accreditation by ACBSP.
Results from the student surveys and interviews indicated student satisfaction with professors, small class sizes, and teaching styles, but in general indicated dislike for group projects and limited flexibility of course offerings. They showed a preference for late morning classes and a desire for mid-term course evaluations. They thought a departmental mission statement would be a good idea and some showed interest in a business club. Course evaluations revealed students’ desire for greater clarification and examples for projects/cases, and a need for more time for some activities, including group projects. Survey results were shared with students via a new department newsletter.
Faculty reported that students expressed appreciation of faculty efforts to assess student needs. In response to the survey results, faculty clarified for students the reasons for limited course availability, namely limited resources. Student surveys will continue to be administered as part of coursework, and students may develop surveys for alumni and employers. A mid-term course evaluation developed in conjunction with a student team will be used by all department faculty beginning fall 1999. Plans for a business honor society are being pursued. The marketing and management curriculum was revamped. The MAT102: Statistics course was added. Tests and assignments were modified where appropriate to minimize ambiguity, allow sufficient time, and exclude invalid items.
Department of Communication Studies
Assessment activities are in place at entry to the communication major, during the course of study, and at the completion of the program. At the completion of the program, communication majors in the required capstone course submitted papers to the annual Undergraduate Communication Research Conference; four were accepted and presented in April 1999. In addition, four communication majors will present the paper they collaborated on at the National Communication Association Convention in November 1999. Senior students also compiled portfolios and those not presenting at a conference took an exit exam, but the aggregate results of those assessments and their use for program improvement were not reported. A survey of graduating seniors with a communication major was conducted regarding the communication curriculum and students’ experience in the department. Faculty discussed the implications of the results of the surveys and the results of the entry-level assessments. Based on those results, faculty have set goals to track beginning test scores in reading and writing (PPST) in order to recommend remediation; strengthen accountability in student internships in several ways; review the sequence and variety of courses offered; investigate a possible student organization; continue expansion of technology applications; revise the departmental mission statement; and (based on University assessment team recommendation) condense department goals and objectives.
Department of English and Modern Languages
Seniors majoring in English completed three outcomes assessment activities in spring 1999: the senior capstone course, “Framing the Literary Tradition”; a comprehensive essay exam; and an oral exam. On the written exam, faculty noted some improvements in results over 1998 in the area of genre, and noted strong responses in poetry explication and theme. Student results on the oral exam were similar to their performance on the written exam, with six of seven students achieving 70% on the written exam and five of seven scoring 70% on the oral exam. Areas found to need attention were preparation of readings by students, meaningful student-led discussions, congruence of faculty expectations for the capstone course, and compatibility of faculty for team-teaching. Plans include making the syllabus for the capstone even clearer about expectations. Of course, many outcomes for the major were assessed at the course level, but how those were assessed and what the results and implications of those assessments were was not included. The department plans to focus on course objectives and their link to the major during the next academic year.
In addition to assessment of student learning outcomes, assessment and improvement of instruction took place through classroom observations by the department chair, through the sharing of course activities and student responses, and through writing-instructor discussions of grading and course objectives.
Department of History
The practices that assess learner outcomes in the department of history continue to be developed. The former use of a standardized outcomes exam (Major Field Test) has been discontinued and will be replaced with a written examination designed by the faculty to better fit course content and learning goals. The previous plan for a capstone course has been put on hold due to low numbers of majors; instead, the majors do an independent study with similar goals and objectives. Because these plans for assessing outcomes in the major are still being developed, no results were available. The report indicated that many outcomes for the major have been assessed at the course level using essay exams, essay assignments, objective tests, group work, and presentations using primary and secondary sources. What the results and implications of those assessments were was not included. The faculty stated its intention to closely link course objectives with goals and objective of the major. It also will seek to maintain this link by meeting regularly to discuss grading standards and scales, by sharing classroom practices and student responses, and by observing classroom teaching of fellow faculty.
Department of Math and Computer Science
The focus of the assessment process this past year was the development of a meaningful outcome measure of competence in mathematics. A new, departmentally designed test was administered to three graduating seniors. Faculty were disappointed by the scores, but not surprised because no particular score was required, so students did not seem motivated to perform well. Assessment also included an analysis of student course performance (grades) and exit interviews with students. Faculty determined upon review of course performances that students were meeting course objectives, but were not retaining course material and applying it on the comprehensive exam. Faculty will discuss how to create incentive for students to prepare for the test. Faculty noted that an additional assessment activity is being developed for the future. It is the Science/Mathematics Seminar, a capstone course for students in the natural science major (comprised of two minors in students’ choice of two science/math disciplines). The small number of students completing this major thus far have made it difficult for faculty to draw reliable conclusions about curriculum and instruction in the math programs overall, and faculty indicated a need to increase student participation in math programs.