Assessment Report

July 1, 2010 – June 30, 2011

Program(s) Assessed: Psychology B.A., B.S., Minor

Assessment Coordinator: Martin P. Gooden, Ph.D.

  1. Assessment measures employed
  1. Learning objectives for both the Bachelor of Arts (B.A.) and Bachelor of Science (B.S.) degrees were assessed this academic year via an online survey service called SurveyMonkeyTM. As in previous years, psychology majors rated the degree to which the undergraduate curriculum provided opportunities to meet each learning objective. Links to the surveywere emailed to seniors taking their second capstone (Psychology 487) course. Seventy-one majors responded.
  1. Program outcomes were also assessed measuring enrollment in post-graduate education, employment status, and the application of knowledge of psychology to their life experiences. Links to the survey were mailed to psychology graduates following summer, fall, winter and spring graduation. Five graduates responded.
  1. The degree to which students in advanced courses demonstrated previously acquired knowledge, skills, and abilities was assessed via surveys administered to faculty instructing advanced special topic research methods courses for 54 B.S. students and the second capstone course for 137 seniors.
  1. The degree to which the curriculum provided Psychology minors opportunities to gain knowledge of psychology and to apply that knowledge was assessed via surveys mailed to minors following graduation. No minors responded.
  1. Assessment findings

A. Graduates earning either a B.A. or B.S. degree in psychology were asked to indicate how much (1 = not at all through 5 = very much) the curriculum provided opportunities to achieve each program learning objective (N=71).

Objective 1: Be familiar with current theory and research in diverse areas of psychology.

Response: Nearly 74% of the respondents answered 4 or 5 (M=4.06, SD= 0.79) indicating they agreed the curriculum provided opportunities to accomplish this outcome. Mean comparisonsfrom the previous year (2009-2010) indicate a statistically significant decrease (M=4.41, SD= 0.70, N=98), t(167)=-2.25, p.03 for this objective.

Outcome 2: Have fundamental research design and mathematical/statistical skills needed to understand psychological science.

Response:Approximately 70%of the respondents answered 4 or 5 (M=4.03, SD=0.78) indicating they agreed the curriculum provided opportunities to accomplish this outcome. Mean comparisons from 2009-2010 indicate no significant change (M=4.29, SD= 0.84), t(166)=-1.66, p>.09.

Outcome 3: Have skills in integrating and communicating about knowledge in self-selected areas of psychology.

Response:Approximately 72% of the respondents answered 4 or 5 (M=4.36, SD=0.79) indicating they agreed the curriculum provided opportunities to accomplish this outcome. Mean ratings from 2009-2010show no change(M=4.56, SD= 0.63, N=98), t(166)=-1.28, p>.20.

Outcome 4: Have skills in effective oral and written communication.

Response: About 77% of the respondents answered 4 or 5 (M=4.19, SD=0.84) indicating they agreed the curriculum provided opportunities to accomplish this outcome. Comparisons to mean ratings from 2009-2010 showed a statistically significant decrease (M=4.53, SD= 0.65, N=98), t(166)=-2.17, p<.03.

Outcome 5: (B.S. students only, N=25): Have advanced research design, mathematical/statistical, and computing skills needed to critically evaluate and conduct research in a self-selected area of interest.

Response: Only 68% of respondents answered 4 or 5 (M=3.59, SD=.76) indicating they agreed the curriculum provided opportunities to accomplish this outcome. Mean ratings from 2009-2010 reveal no significant change (M=3.78, SD= 1.10, N=40), t(63)=-.74, p>.46.

Suggestions for improvements:

Over this past year, performance in Learning Outcomes 1 through 4 has revealed declines particularly in Outcome #1 and #4. The declines were in contrast to the trends observed from the 2009 to 2010 academic year. During that period, performance across Outcomes #1 through #4 showed evidence of improvements (particularly in Outcome #3 and #4). Learning Outcome 5, however, shows evidence of a continual decline from 2007 to the present. As can be seen in Table 1 below, across the past 4 years, student evaluations of the development of skills in advanced research design, math, statistics, and computing skills have declined a full point (an average of 4.65 in 2007 to 3.59 in 2011). This trend suggests that either (1) BS students are failing to recognize the aims of the course, (2) faculty instruction requires greater clarity or coordination, or (3) some combination of the two. To address these 3 possibilities, faculty who teach the advanced method’s courses will be invited to review these findings and develop a strategy to curb this decline. One recommendation would be to better coordinate learning experiences in the basic method’s courses (PSY301, 302, and 303) with the advanced method’s courses. Another recommendation would be greater coordination among the advanced method’s instructors themselves. With such changes and a well-developed plan, it is hopeful that this decline can be stemmed by the next assessment period.

Table 1: Average Ratings of Curriculum Outcomes 2007-2011

Outcome #1 / Outcome #2 / Outcome #3 / Outcome #4 / Outcome #5
2007-2008 / 4.44 / 4.22 / 4.45 / 4.48 / 4.65
2008-2009 / 4.22 / 4.08 / 4.36 / 4.28 / 4.10
2009-2010 / 4.41 / 4.29 / 4.56 / 4.53 / 3.78
2010-2011 / 4.06 / 4.03 / 4.36 / 4.19 / 3.59

In addition to the declines in the courses that focus of research methodology (B.S. students, Outcome #5), students this past year also appeared less satisfied with the quality of instruction across core theory courses (e.g., Abnormal, Developmental, Cognition, etc.). The reasons for this decline are more uncertain. Speculating, student expectations regarding what they believe they need from these courses may not be consistent with what is being offered. A common refrain from students seeking academic advising suggests that confusion regarding their course offerings is common. Once enrolled in a course, students may question the relevance of the material, the approach in which the course is taught, the amount of work required, etc. Lower levels of satisfaction may simply reflect the changing expectation of students. Admittedly, to the extent that faculty have not responded to these changing expectations, the manner of instruction may also contribute to these findings. Since these assumptions are only speculative, faculty will be invited to review these findings and consider ways to curb the decline in student attitudes.

One of the most valuable skills that students learn while in college is how to effectively communicate. Psychology majors are provided with 4 opportunities to develop their writing and public speaking skills within the major. Four courses are designated officially as writing intensive, while others may involve writing assignments. Students are also actively encouraged to “speak up” in class, particularly those courses with instructors who insist on active participation. Thus, there is no shortage of opportunities for students to develop these communication skills. Speculating on the decline, next to math, the development of writing skills is one of the most challenging goals that are established for students. Performance is drivennot only by lots of practice by students, consistent, structured, and repeated supervision by faculty is also important.Comments from students have suggested that variation in the rigor required by one faculty member to the other in writing papers is frustrating. This frustration may explain the decline from 2009-2010 to the present. Upon reviewing these assessment findings, faculty who teach writing courses will be encouraged to consider steps that if taken enhance students’ perception of value, while preserving quality instruction.

Open-ended Feedback

Similar to previous years, student open-ended comments cover a wide array of topics. In general students are asking for more advance courses that involve topics related to mental health (e.g., clinical psychology). Students also want more courses that will help develop effective writing and public speaking skills.Practicum and research experiences are also desired so that student can develop tangible and marketable skills in areas more closely aligned with their interests. These requests are consistent with previous years. To meet part of this demand the department is introducing a greater variety of advanced capstone courses (N=42) in preparation for the semester conversion in Fall 2012. A more coordinated effort will also be directed toward encouraging participation in the practicum and research experiences. The department of psychology is also employing more personnel (graduate students/adjunct faculty) from Wright State’s School of Professional Psychology. Their expertise and background in the areas of clinical and counseling psychology will provide psychology majors with greater exposure to mental health professionals in an academic setting. Over time, this should help satisfy student interest by providing graduate student and professional role models for which undergraduates can both emulate as well as seek advice.

Shifting towards the BS

As noted in previous assessments, the trend towards the BS degree appears to be continuing. An examination of the number of students enrolled within the major, and the distribution across the 2 degree-types (B.A. vs. B.S.) revealsa continual increase in the number of students pursuing the B.S. degree (see Table 2). In just the past year, an 8% increase in the number of BS students was observed.

Table 2.

Qtr/YR / B.A. / B.A. Intent / B.A. Total / B.S. / B.S. Intent / B.S. Total / Total / Ratio B.S. to B.A. Majors / Ratio B.S. to B.A. Total
F06 / 410 / 49 / 459 / 61 / 118 / 179 / 638 / 12.95% / 28.06%
F07 / 329 / 30 / 359 / 157 / 146 / 303 / 662 / 32.30% / 45.77%
F08 / 279 / 49 / 328 / 234 / 183 / 417 / 745 / 45.61% / 55.97%
F09 / 259 / 44 / 303 / 299 / 160 / 459 / 762 / 53.58% / 60.24%
F10 / 309 / 75 / 384 / 327 / 152 / 479 / 863 / 51.42% / 55.50%
F11 / 273 / 37 / 310 / 355 / 189 / 544 / 854 / 56.52% / 63.70%

With the increasing shift from the BA to the BS degree, undergraduates who are interested in the B.S. degree have several concentrations for which they can pursue their interests in psychology. Students can elect Behavioral Neuroscience, Cognition and Perception, and Industrial and Organizational Psychology. The requirements for students eager to pursue these areas are rigorous, but each offers dynamic opportunities sure to make Wright State students competitive for graduate programs as well as the work force.

B. Findings regarding B.A. and B.S. program outcomes based on the post-graduation survey.

Graduate School Enrollment (N=69): 7.25% or 5 students reported they are currently enrolled in or have been admitted to a graduate program or a professional school. This figure is comparable tothose who reported being admitted to a graduate program in the 2009-2010 calendar year (6.25% or 6 students from a total N=97, z=.26, p>.7).

Employment (N=70): 7.14% or 5 students reported they had received and accepted an employment offer. Thisrepresents a slight but non-statistically significant decline fromthe 8 students (8.24%, N=97) who reported they had received and accepted an employment offerduring 2009-2010(z=.26, p>.7).

Application of Knowledge (N=70): 58.67% or 41 respondents answered 4 or 5 (M=3.81 SD=.98) indicating they agreed they have been able to apply their knowledge of psychology to the issues they have confronted in education, employment, and interpersonal relations. A comparison of the mean ratings from 2009-2010indicate that the increase is not statistically significant (M=3.97, SD= 1.03, N=95, t(163)=-1.02, p.31).

C. Findings regarding learning outcomes based on faculty rating the degree (1= not at all; 3= satisfactory; 5=excellent) to which each student demonstrated specific knowledge, skills, and abilities (KSA) acquired in previous courses.

Faculty instructing five advanced special topics methods courses rated B.S. students.

KSA 1: Be able to design a sound psychological study (N =54).

Response: Faculty rated 50% of the students as 4 or 5 indicating they had sufficiently met this objective (M=3.87, SD=.94). Comparisons to mean ratings from 2009-2010 showed no change (M=4.16, SD= .78, N=53), t(105)=-1.50, p.14.

KSA 2: Be able to analyze data (N = 54).

Response: Faculty rated 46.30%of the students as 4 or 5 indicating they had sufficiently met this objective (M=3.67, SD = .91). Comparisons to ratings from 2009-2010 showed no change (M=4.00, SD= 0.65, N=53), t(105)=-1.71, p.09.

KSA 3: Ability to communicate findings effectively (N = 54).

Response: Faculty rated 59.26%of the students as 4 or 5 indicating they had sufficiently met this objective (M=3.78, SD =.99). Comparisons to ratings from 2009-2010 showed no change (M=4.02, SD= .79, N=53), t(105)=-1.24, p.22.

****************

Faculty instructing 31 capstone courses across 2011 rated students taking their second capstone class (N= 137).

KSA 1: Have knowledge of current theory and research in diverse areas of psychology

Response: Faculty rated 78.39% of the students as 4 or 5 indicating they had sufficiently met this objective (M=4.07, SD= 0.73). Comparisons to ratings from 2009-2010 showed no significant change (M=4.15, SD= 0.60, N=126), t(261)=-.65, p>.51.

KSA 2: Fundamental skills in critically evaluating theoretical concepts, research design, and data analysis.

Response: Faculty rated 66.42%of the students as 4 or 5 indicating they had sufficiently met this objective(M=3.96, SD = 0.73). Comparisons of mean ratings from 2009-2010also indicate no significant change, (M=4.08, SD= 0.71, N=126), t(261)=-.97, p>.33.

KSA 3: Effective communication skills including skills for writing literature reviews and empirical research reports following the American Psychological Association Manual of Style, and skills for participating in seminars and making oral presentations following accepted standards for scientific conferences.

Response: Faculty rated 80.29%of the students as 4 or 5 indicating they had sufficiently met this objective (M=4.13, SD= 0.68). Comparisons to mean ratings from 2009-2010 indicate no significant change (M=4.08, SD= .74, N=126), t(261)=.40, p>.69.

D. Findings for the psychology minor. *

Objectives: Graduates earning a minor in psychology were asked to indicate how much (1 = not at allthrough 5 = very much) the curriculum provided opportunities to be familiar with current theory and research in self-selected areas of psychology. They were also asked to indicate how much (1 = not at all through 5 = very much) they had been able to apply their knowledge of psychology to issues in education, employment, and interpersonal relations. Unfortunately, none of the minor students responded. Given the difficulty in reaching psychology minors to complete assessment surveys in the past 2 years, a more targeted approach will be employed involving a return to the paper survey in addition to follow-up surveys with SurveyMonkeyTM.

  1. Program improvements

Now in its sixth year of existence, the Psychology Undergraduate Program (PUP) Office continues its collaboration with campus and community partners to support academic, research, and practicum opportunities available to undergraduate students. The PUP office also works with the undergraduate student organizations PsiChi (the National Honors Society for psychology undergraduates) and the Psychology Club. Student members of these organizations and PUP office staff continue their collaboration with recruiting, marketing and promotion of psychology, workshops, planning for professional conferences, and related programming.

IV.New assessment developments

SurveyMonkeyTMwas implemented this year to replace the paper and pencil surveys that were manually sent out to students. The new mode of administration simplified data entry, but likely contributed to a slightly lower response rate. Whereas nearly 100 students returned surveys during the 2009-2010 assessment period, only 71 students responded to the SurveyMonkeyTM solicitation this year. Despite the decrease, we’re committed to this mode of administering the assessment but concede that follow-up communications will more than likely be necessary.

****

END