Assessment Analysis of Academic Units
This document is the compilation of the answers to the Criterion Three (Assessment) questions completed by the academic, and academic support, departments and units on campus. These units are organized in alphabetical order (using the newly formed departments established in Dr. Park’s March 20, 2006 announcement). The following units are listed:
Applied Sciences
Business and Economics
Communication Arts
Counseling, Psychology and Social Work
Education
Extended Campus
General Education
Health, Physical Education and Recreation
Honors
Justice Studies
King Library
Language and Literature
Library Media Program
Mathematical Sciences
Music
Physical and Life Sciences
Registrar
Social Sciences
Visual and Performing Arts
The questions that they were each asked are listed here (they aren’t listed every time to save space):
1. What are the student learning outcomes for each program in your unit?
2. What tools does your unit use to assess student learning? How is data collected, stored and accessed?
3. How are individual courses’ and program evaluation coordinated?
4. How have assessment results been used to improve student learning?
5. How have assessment results been used to improve teaching?
6. How has your unit contributed to effect learning environments, facilities and support services?
The majority of the units struggled with questions 3, 4 and 5. The issue which stumped units on question 3 was deciding whether to report the “chain of command” of how evaluation is done, or to explain how the unit insures that the various courses’ student outcomes combines to create the unit’s student outcomes. Most units went with the chain of command answer, and I think a view of the units’ assessment condition is better served by answering the second question. I wrote about this choice in the Applied Science area, and then referred back to it in other units where the same event occurred. (Another issue that I wondered about when I was analyzing these, but we didn’t explicitly ask about, was whether the unit insures that the different professors teaching the various sections of the same course [whether concurrently, or over different semesters] have the same learning objectives identified for the same course. I think this should be investigated.)
The issue with question 4 and 5 is that most units decided each question was asking the same thing. I think that is a mis-interpretation. I think that teaching is the activities that professors do to help students achieve the student outcomes, while learning is the ability of the student to demonstrate the student outcomes. I think the assessment issue here is how CSC uses assessment data to detect whether students achieve the outcomes (which is question 4) and how it uses assessment data to analyze why the teaching didn’t help the students achieve the outcome. Further, what does the assessment data indicated might help students better achieve the outcomes (these last two questions are asked for in question 5). Again, I wrote about this at some length in my reaction to Applied Sciences’ treatment of questions 4 and 5 being the same. I also referred to that answer when appropriate with other units.
After all the comments were inserted into each unit’s responses, I employed the Assessment Culture Matrix and the Patterns of Characteristics Analysis Worksheet (shown below) to analyze each department/program. (I didn’t use II Shared Responsibility, part b Administration and Board as it didn’t seem appropriate to a department/unit, as well as III. Institutional Support, part a: Resources as its characteristics are mainly concerned with institution-wide allocation of resources, and I assumed most department/unit heads don’t have that kind of latitude to change resource allocation that much.) I then averaged the numeric score of the other answers for the whole unit to create a “unit average” assessment culture score. These numbers are inserted at the end of each unit’s comments.
Lastly, here are the departments’/units’ “overall” average assessment culture score and the whole academic average assessment culture score:
All Departments and Units Departments Only
Average 4.41 (out of 9) Average 4.5 (out of 9)
Standard Deviation 1.948 1.98
The distribution is pretty bimodal with 5 of the 16 units only scoring 2 out of 9. Most of these were due to incomplete answers, not having a mission statement or being a brand new unit. These units are stuck in the beginning implementation of the culture of assessment. On the other hand, there are 6 of the 16 units above 5.5. In general, they are classified as making progress on the culture of assessment, or in the mature stages of implementing the culture of assessment. The values associated with each of these three classifications are listed below:
Classification Score
Beginning of Implementation 1 to 3
Making Progress in Implementation 4 to 6
Maturing Stages of Implementation 7 to 9.
(Note: Some department/units’ reports are not in this report—that will be remediated.)
See work sheet below:
Applied Sciences:
Agriculture/Range Management Student Learning Outcomes:
Students in this program will develop the knowledge, skills, competencies, and attitudes so they will be able to:
* Attain a career in an agriculture or range management related field.
* Recognize the highly competitive and global role of agriculture in the local, national, and world marketplaces.
* Achieve entrance into graduate programs in agriculture or range management related fields.
* Interpret and utilize current theory and research findings to enhance knowledge, skills, and abilities needed for agriculture or range management related careers.
FCS Student Learning Outcomes:
Students in this program will develop knowledge, skills, competencies, and attitudes so they will be able to:
* Analyze factors which contribute to the development of healthy individuals throughout the lifespan (conception through old age).
*Analyze nutrition and life course choices which influence lifespan wellness.
* Assess the relationship between managing resources (time, energy, money) and achieving personal or family goals.
* Apply critical and creative thinking skills in addressing individual and family problems and issues in diverse environments.
* Describe the physical, emotional, mental and social development of children.
* Illustrate the role food, clothing, and shelter play in individual and family consumerism and resource management.
* Summarize the history of the FCS profession as well as the multiple career paths available to FCS graduates.
* Demonstrate the ability to use knowledge, skills, competencies, and attitudes in a professional work experience.
ITE Student Learning Outcomes:
Students in this program will develop knowledge, skills, competencies and attitudes so they will be able to:
* Demonstrate ability in the application of basic management principles as related to an industrial or educational environment within the regional and global economies.
* Apply technical knowledge in manufacturing, energy, power and transportation, communications and construction in problem solving situations relevant to their study of manufacturing, construction or industrial education.
* Demonstrate behaviors and attitudes consistent to team building leadership strategies within the regional and global economies.
* Develop a skill set of industrial, construction and educational processes as they apply to the regional as well as global economies.
* Demonstrate strong interpersonal and communication skills that are required for successful functionality in a manufacturing, construction or educational environment.
There are three areas in applied sciences: Agriculture/Range Management, Industrial Technology (ITE) and Family Consumer Sciences (FCS). Each of the programs administers professional knowledge exams. The Agriculture/Range Management program tracks scores from the Undergraduate Range Management Exam administered by the Society for Range Management over time. Students are allowed to take it annually, and this allows the program to track student progress and compare it to other schools offering similar programs. Students are also encouraged to take the American Registry of Professional Animal Scientists exam. Again, appropriate corrections are made to animal science curriculum. The ITE program administers the Society of Manufacturing Engineers exam to its students. Detailed results are presented to faculty. Deficiencies in the exam are noted and corresponding adjustments are made to the curriculum and presentation methods. The FCS group administers the National Family & Consumer Sciences Certification Examination and makes appropriate adjustments to materials and programs.
Additionally each program has developed a set of rubrics to assess classes. Feedback is also garnered through fall semester class evaluations. Each faculty member then has to respond to these results and analyze how they can be utilized in adjusting course content. Feedback from recently employed students and their employers is also collected, analyzed and implemented.
The department tries to coordinate its assessment of each course, and for each of their three programs in the following manner. Each class is given an assessment in the fall semester. Rubrics are also constructed for by the programs for their classes. Each program has a faculty member that is assigned as the assessment coordinator. They prepare a full assessment report of each program based on these tools which then goes to the department chair. The chair then combines the reports for the department and submits it to the Dean.
The faculty of the Department of Applied Sciences uses these assessments as tools to adjust their course content, course offerings, and delivery methods. This is supplemented by direct student input from both current and past students. Attendance at state and international meetings leads to collaboration with faculty at other institutions and the sharing of educational content. These meetings also provide educators with emerging information.
Programs within the Applied Sciences department provide a mix of classes ranging from traditional lecture format, laboratories, field experiences, tours, on-line, and Interactive Television (ITV) to hybrid (combination of face-to-face and online) delivery formats. Most programs have migrated to a PowerPoint format for lecture materials. The CADD lab maintained by the ITE program is maintained to a level representative of industry standards. The Child Development Center provides a unique opportunity for students to gain experience in working with young children and observing them as they learn through various methodologies.
Applied Science’s Assessment Culture Matrix
Patterns / Score / Evidence/RationalI. Department Culture
a.) Collected/Shared Values / 7 / Assessment is valued by all, but the centrality of student learning is lacking.
b.) Missions / 6 / Assessment’s importance not indicated in mission statement.
II. Shared Responsibility
a.) Faculty / 6 / Good, but faculty may not know about assessment and also it’s unclear who besides the person in each area deals with assessment.
c.) Students / 2 / I think that students are involved with assessment committees, but there is no such evidence given.
III. Institutional Support
b.) Structures / 4 / A structure exists, but no time line, or order of things to do.
IV. Efficacy of Assessment / 6 / Good start; work on involving more students in assessment and being more formal about it.
Applied Science’s Assessment Culture Matrix average score is 5.67 (=31/6).
Business and Economics:
The objectives for the Department are:
A. The students will average at least in the second quartile of the major field achievement test in business produced by the educational testing service. The test will be taken during the student’s senior year.
B. Departmental Survey of Recent Graduates
a. 85% of graduating seniors, actively seeking employment, will be employed in an entry-level position within six months of graduation as indicated by the data gathered through a departmentally developed survey instrument.
b. 80% of the graduating seniors completing the attitudinal portion of a departmental survey will indicate they “agree” or “strongly agree” with the statement: “I had enough depth of study in an emphasis area to feel confident in learning and using the practical elements of that area.”
C. 75% of students who participate in a professional experience (student teaching and/or internship) will average at least 80% on their final grade. Grades will be assessed by the faculty coordinator based on attainment of initial goals set by the student, evaluation of paper work including the final paper, and recommendations given by the site supervisor(s) and internship & career services representatives.
The department has six separate tools for assessing student learning:
A. The ETS Major Field Test—this nationally normed test is used to assess the learning in the Department’s core classes.
B. The Departmentally Created Survey of Recent Graduates—this is used to assess student satisfaction with their learning.
C. The Attitudinal Portion of the above survey.
D. Evaluative instruments used by Site Supervisor(s)/Employers and Internship Representatives.
E. A Subjective Faculty Assessment of Meeting the Course’s Learning Objectives—This instrument is designed for the faculty of each class to assess how well they achieved the goals of each class they taught, what worked well, and what needed to be improved. This was started in Spring 2005 and was discontinued in Spring 2006 (because people were afraid that the frank assessment would be used against them/or they didn’t take it seriously and everything they did was superior).
F. Anecdotal Evidence of Employer Views of our Graduates, now being formalized into the Department’s Board of Advisors—In fall 2005 the Department Board of Advisors was formed with the express goal of providing us feedback about the performance of our graduates and to alert us about business trends.
The department requires that each graduating senior takes the ETS test. The goal is to have our department average in the second quartile of the test results. This seeming low level of achievement was chosen because the test is being used to assess our business core (accounting, business law, economics, finance, international business/economics, management and marketing), and so every student cannot be expected to know all of these things to the depth that the test expects (since they generally specialize in only one of those areas, as they choose an option in only one of these areas).
The department goal for the survey is that 85 percent of graduating seniors who actively seek employment will be employed in an entry level position within six months of graduation.
The department goal for the attitudinal portion of the department survey is that 80 percent of graduation seniors will indicate that they “Agree” or “Strongly Agree” with the statement” I had enough depth of study in an emphasis area to fell confident in learning and using the practical elements of that area.”
The department goal for the evaluative instrument by the site supervisor(s)/employer(s) and internship representative is that 75 percent of students who participate in a professional experience will average at least 80 percent on their final grade.