1
2009-2010 Annual Program Assessment
MS Experimental Psychology
2009-2010
Annual Program Assessment for the
MS Experimental Psychology Graduate Program
Final Report
Submitted to Dr. Stephanie Stein, Chair
Department of Psychology
by
Dr. Wendy A. Williams
Program Director
June 14, 2009
1
2009-2010 Annual Program Assessment
MS Experimental Psychology
Table of Contents
I. Introduction and Overview ……………………………………………………….….3
II.Program Enrollment…………………………………………………………..……4
Admissions, Enrollment Changes, Enrollment Trends,
Recruitment, Degrees conferred, Attrition and Retention
III. Assessment of Student Learning Overview…………………………………….….9
Review of Record Keeping Procedures, Number of Enrolled Students
IV.Major Milestones……………………………………………………………….…12
General Academic Assessment Data, Course of Study Forms, Option
Approval Forms, IACUC/HSRC Approvals, Theses Accepted, Degrees
Conferred, Conference presentations
V.Time to Completion Data…………………………….……………………………15 Thesis Defense Meetings/Time to Completion
VI.Programmatic Course Offerings…………………………………….…………….17
VII.Quarterly Survey of Student Progress…………………………………………....18
Academic Performance Summary, Cumulative GPA,
Itemized Assessments:
• Content knowledge
• Skills mastery
• Quantitative analytical skills
• Logic-based analytical skills
• Comprehension
• Written and verbal expressive skills
•Appreciation of empirical evidence
• Ability to succeed in the experimental psychology program
• Disposition consistent with the field of experimental psychology
VII. Assessing the Quarterly Survey………………………………………………….27
Previous Limitations, Results of solutions Implemented, Limitations to the new 2009-2010 Assessment
IX. MS Experimental Psychology Program Assessment (Student Version)………….29
Overview, Breakdown by Program, Breakdown by Year, Open-ended Comments
X. Concerns raised in Previous Assessments (2007-2008; 2008-2009)………………33
XI.Conclusions………………………………………………………………………37
2009-2010 Annual Program Assessment for the
MS Experimental Psychology Graduate Program
Final Report
Introduction and Overview
The 2009-2010 academic year was spent continuing to improve the MS Experimental Psychology program assessment procedures, making important changes to the ABA specialization program, developing a new colloquium course for all MS Experimental Psychology students, moving the assessments to an on-line format, and recruiting for the 2009-2010 school year. A significant change in assessment procedures was issued on May 20, 2009. Program and departmental assessment data must be received by Dr. Tracy Pellet on or before June 15th. Every effort has been made to gather all data available for Spring 2010, however, some data was not available and was therefore not included.
Moreover, it makes more sense to talk about our program in terms of the academic year because it keeps cohorts of students together. Program changes and university changes make it difficult to lump students together across academic years. Therefore, the qualitative nature of this report will refer specifically to the 2009-2010 academic year. All evaluations are based on grades or formalized assessments submitted before June 14, 2010. Averages may not reflect responses from all faculty, due to the early deadline.
Program Enrollment
Admissions
Thirteen new students were admitted to the MS Experimental Psychology program this year. Three females and ten males. Eight were accepted into the General Experimental program; five were admitted to the ABA specialization. Their stats are outlined in Table 1.
Table 1. Average GPA and GRE scores (verbal and quantitative) for 2009-2010 incoming graduate class in the MS Experimental psychology program.
N / GPA / GRE-Verbal / GRE-Quant2008-2009 / 6 / 3.40 / 487 / 625
2009-2010 / 13 / 3.53 / 472 / 585
The range for undergraduate GPA (last 90 credits) for the thirteen applicants was 2.88-3.98; the range for GRE-verbal scores was 350-620; the range for GRE quantitative scores was 450-720.
Applicant Targets. The MS Experimenal psychology program’sminimum entrance requirements are:
1) BA/BS in psychology or a related field
2) GPA of 3.0
3) combined GRE score (verbal+quantitative) of 900.
This year, one student fell below the 3.0 GPA cutoff. This student was a CWU graduate; well known in the department; had earned a 3.89 in winter 2009; and was working to complete an honors thesis. Based on his reputation, his most recent work, and on his interest to continuing his work with his honors thesis mentor, he was admitted. Thus far he has proved to be highly successful in the program. The range for the combined GRE score was 860 – 1340. Two students’ GRE total scores fell below the 900 cutoff. Both student earned GPA scores well above the minimum. The overall average GRE combined score was 1058 – significantly above the minimum requirement. All applicants also need to have a research interest that aligns with at least one of our faculty members. This was true for all thirteen students admitted to the program.
Enrollment changes
During the course of the 2009-2010 school year, the status of several students’ enrollment changed. Three students announced they were dropping from the program. The program director met several times with each student to discuss their individual issues. None of the students left the program for reasons related to the curriculum, funding, staffing, or performance. All three students left for personal reasons. Two left after deciding that graduate school was not a good fit for them. The third has decided to remain active but to not enroll. She will be searching for a MS program in Organizational Development, a program that is no longer available at CWU.
Also during the 2009-2010 school year, two students changed their program affiliations. One student changed from the General Experimental program to the ABA program. The other student changed from the ABA program to the General Experimental program. Both students changed their specializations under the guidance of (and approval from) their academic advisors, the program coordinators and the program director.
Enrollment Trends for 2009-2010
Enrollment in the MS Experimental Psychology programs increased from 5 students in 2007-2008 to 6 students in 2008-2009 to 13 students in 2009-2010. This year (2009-2010) the enrollment in the MS Experimental Psychology program more than doubled from last year, and was greater that the two previous years combined.
This year (2009-2010) was the second year since the creation of the Primate Behavior and Ecology Program, which was previously expected to significantly reduce enrollments in the Psychology MS Experimental Psychology program. Only one student enrolled with an interest in primate behavior in fall 2009, but he changed his interest to social psychology after the first quarter. Hence, our reliance on primate behavior (CHCI) students has decline from more than half (3/5) in 2007-2008 to one third (2/6) in 2008-2009 to zero (0/13) in 2009-2010. This change is noteworthy in that primate behavior has historically been the primary draw to the MS Experimental Program for over 15 years. The addition of new faculty and the new ABA program have more than compensated for the elimination of CHCI as part of the psychology department.
Similarly, 2009-2010 was also the second year for the Applied Behavior Analysis specialization within the MS Experimental Program. Enrollment in the ABA specialization more than double from 2 students in 2008-2009 to 5 students in 2009-2010.
Furthermore, the Ms Experimental Psychology program appears to be growing as it reaches out to potential students with a more diverse faculty interest base (with research interests that include physiological psychology and neuroscience, human cognition and memory, applied behavior analysis, social psychology, and animal learning and behavior). This broadening of research interests and the development of curricula that meet national standards for licensure (in ABA and Animal Behavior) appears to be having a dramatic influence on enrollment. While we don’t expect every year to result in such a dramatic increase in enrollment, it does suggest that we are on the right track in terms of offering the kinds of programs that are desired by recent college graduates.
Enrollment Targets. There was no explicit program targets for enrollment in 2009 when the admissions process was underway. Since that time, it has become our goal to admit a minimum of 1-2 students per faculty member. In 2008-2009, we had 9 regular tenure-track faculty members on staff so that would place our new targets between 9-18 students. In 2008-2009 academic year, we admitted 6 students. In 2009-2010, our enrollment jumped to 13 – well within our target goal. It seems that our recent attempts to strengthen the MS Experimental Psychology program through new hires and the addition of the ABA specialization has been successful.
Next year we will see the loss of one tenure-track faculty member (Dr. Ryan Zayac, ABA program coordinator), although we hope to search to replace that position in Fall 2010. Also, Dr. Elizabeth Street will rejoin the MS Experimental Psychology Program in winter 2011 which may help to offset the loss of Dr. Zayac. Her background in applied behavior analysis will help us through the transition to finding a new program coordinator who is BCBA certified. Because the ABA program is in flux, we do not anticipate growing the program immediately. Rather, we anticipate a temporary stabilization in recruitment, enrollment and retention resulting in a total enrollment in the ABA specialization somewhere between 10-15 students over the next two years. However, changes to the ABA faculty and programming should result in another period of growth sometime around 2012-2013.
Recruitment for 2010-2011
This year, the MS Experimental Psychology program set target goals for recruitment and enrollment. Our goal was approximately 1-2 students per faculty member. With a reduction in staffing by 1.5 faculty members (Drs. Anthony Stahelski and Marte Fallshore), we were left with seven full-time tenure-track faculty members. Thus, our goal for admissions for 2010-2011 was for between 7-14 students.
In 2009-2010, the responsibility of recruitment and admissions was given to Dr. Kara Gabriel. She worked collaboratively with the MS Experimental psychology faculty to match students with individual faculty. We implemented an acceptance policy that requires at least one full-time faculty member with a shared research interested to accept responsibility as an applicant’s academic advisor. Also, at least one other faculty member must agree to take over in the event a change of advisor is needed.
Dr. Ryan Zayac, the Advisor for the Applied Behavior Analysis Specialization attended the Association of Behavior Analysis International Conference in San Antonio, Texas to promote the ABA specialization in MS experimental psychology at CWU. As a result f the combined efforts of Srs. Gabriel and Zayac, 8 new graduate students accepted positions in the MS Experimental Psychology program (3 in General Experimental and 5 in the ABA specialization). This is the first year that students were turned away in the ABA program. Enrollment in the ABA specialization is limited by the supervision requirements. Last year Dr. Zayac was our only fulltime, tenure-track Board Certified Behavior Analyst (BCAB) faculty member. Some assistance was provided by Season Almasen, M.A, and a BCBA from Children’s village (one our practicum sites). With only Dr. Zayac able to conduct supervision, a cap of 5 students was placed on enrollment. Now that Dr. Zayac is leaving, Ms. Almasen will be teaching courses in the ABA specialization as an ABD doctoral candidate and adjunct, and Dr. Elizabeth Street is beginning the process of submitting her paperwork for her BCBA. She has all the educational and supervised hours qualifications but she must sit for the Board exam. Dr. Wendy Williams has agreed to complete her ABA supervised hours under Dr. Street in 2011-2012. Additionally, it is anticipated that a search for Dr. Zayac’s replacement will be approved for Fall 2010. It is anticipated that the cap on enrollment in the ABA specialization will be raised for 2012-2013, after the staffing concerns have been resolved.
Incoming students.Of the students slated to arrive in September 2010, three are coming from other universities (2 from out-of-state, plus one international student). and the remaining 5 are CWU graduates. Five students are joining the ABA specialization and will be working with Drs. Street and Williams; one student is coming from Ghana to do social/health psychology research with Dr. Williams. One student will be working with Dr. Gabriel; another will be working with Dr. Polage. Drs. Greenwald, Lonborg, Fallshore, and Matheson will not be advising new students in the fall but the admission’s process for the MS Experimental Psychology (General Experimental specialization) remains open throughout the year. It is not unusual for us to accept students onan ongoing basis.
A comparison of enrollment trends between 2007-2008, 2008-2009 and 2009-2010 is noted in Table 2.
Table 2. Enrollments for 2007-2009 and offers made and accepted for the 2009-2010 academic year.
Academic Advisor / 2007-2008 / 2008-2009 / 2009-2010 / 2010-2011K. Gabriel / 1 / 0 / 2 / 1
R. Greenwald / n/a / 0 / 2 (-1) / 0
S. Lonborg / 0 / 0 / 2 / 0
M. Matheson / 0 / 1 / 1 (-1) / 0
D. Polage / n/a / 0 / 1 / 1
E. Street† / n/a / n/a / n/a / (5)
W. Williams† / 0 / 1 / 2 / 6
R. Zayac / n/a / 2 / 3 / n/a
M. Jensvold (CHCI) / 3 / 2 / 0 / 0
S. Schepman* / n/a / n/a / n/a / n/a
M. Fallshore (.5) / 0 / 0 / 0 / 0
A. Stahelski (.5) / 1 / 0 / 0 / retired
*Dr. Schepman works primarily with students in the School Counseling. School Psychology and Mental Health counseling programs.
† Dr. Williams will share thesis advisor duties with Dr. Street for all of the ABA specialization students from 2008-2009 to 2010-2011. Her additional students work in general experimental psychology.
Note: All MS Experimental faculty may also supervise thesis projects for student in the other Psychology graduate program. Hence, not all faculty members’ thesis advisees are noted in Table 2.
Degrees Conferred in 2009-2010
Last year the program director, Dr. Wendy Williams, made serious overtures to Nth year students (3rd year and beyond) who had lost momentum in their programs, or taken leaves of absence. Her goal was to clarify each student’s standing and intention, as well as to attempt to find ways to help them return and complete their programs in a timely fashion.
Ten students were contacted. Of those, five students have completed their degree programs. Two students completed their coursework and successfully defended their theses within the 2-year timeframe set as the goal to completion in both specializations. An additional three students are poised to defend their theses this summer.
Attrition and Retention
Three students (1st-Nth year) resigned from the program this year. Of the students who did not complete the program, three are poised to defend their theses this summer. The remaining students four appear to have stagnated and will approached again in the fall by the program director to see what we can do to facilitate completion of their programs. All remaining students continue to be actively working toward degree completion.
Assessment of Student Learning Overview
In 2007-2008, the MS Experimental (MS EXP) Psychology Graduate Program Committee met to revise the Student Learning Objectives (SLOs) and SLO Assessments. At the time, the Psychology Department had charged the MS Experimental Psychology Program Committee with reviewing and updating the SLOs and assessments in order to bring them more in line with the assessment procedures of the other graduate programs within the department. The work on the SLOs and Program goals was met with strong approval by the CWU Assessment Team.
In 2007-2008, a new tracking system was developed that included individual files, survey materials, and summary data. At the time, only a preliminary report based on a narrow application of those tools was available. Last year, the tools were implemented more comprehensively and thoroughly. This year, an electronic survey system was developed in an attempt to get greater cooperation from the faculty in terms of completing the forms. The assessment of student learning for the 2009-2010 academic year is based on these new, electronic assessment tools. direct comparisons between 2008-2009 and 2009-2010 will be made whenever possible. However, it is clear that the electronic form requires minor changes to ensure that the data are gathered in ways that allow such comparisons.
Review of Record Keeping Procedures
Supplemental Individual Files for ALL current students in the MS Experimental Psychology Program (going back to 2004-2005) are now created for every graduate student and maintained in the Director’s office. Much of the data in these files was gleaned from the departmental records and/or graduate school records, and was then summarized within each file. (See attached Sample Tracking Form). As each student progresses through the program, copies of the course of study forms, option approval forms, IACUC and/or HSRC approvals, updated annual transcripts and other relevant documents are placed in this file. Major milestones like proposal meetings, thesis defense meetings, theses submissions and approvals are also noted at the end of the year. Individual assessment reviews by course faculty, academic advisors and thesis chairs are maintained here as well. Initial fall assessments are averages across instructors; winter and spring are then added into that average across the year. By late June we have an overall assessment for each student that encompasses one entire academic year (quarterly averages are not currently being computed but could be if an individual case warranted such fine grain analyses) This year, electronic data collection was implemented in an attempt to streamline this system, increase user friendliness, and to provide printed summaries for the students’ files.
Strengths. The student files have centralized important information like names, ID numbers, course of study, major advisor, contact information. The use of the summary forms has allowed us to focus on the progress and needs of individual students. We are able to check off completed courses and note progress in a timely manner by cross-referencing quarterly transcripts with their official course of study forms. This has been easily implemented because the information is available on-line to the director. Furthermore, the new assessment data have allowed us to directly address many of the issues raise by the CWU Assessment committee last year, namely assessments based on measures other than grades and academic progress. These summary forms also make it easy to contact students and to maintain a record of conversational notes as a way to make sure that students are getting what they need in terms of resources, support and guidance.
Limitations. Tracking student milestones continues to be difficult. In the past, duplicate copies of the Course of Study forms were sent to Dr. Williams before being filed in the department office. This has become less than reliable due to budget cuts and reductions in staffing. Furthermore, proposal meetings, option approval forms, defense meetings, and other types of information do not have a regular track that funnels through the Director’s office. As such, tracking relies on self-report by advisors, or via manual review of departmental files. This system is too slow and requires repeated checks (almost on a weekly basis by the Director). We failed to develop an acceptable alternative tracking system with the graduate school that would pass through the Program Director’s office. Hence this year, the data were gathered by hand and only at the end of the year. Some faculty did report milestones electronically, others did not. The goal for next year is to make minor changes to the electronic assessment surveys that will make reporting easier (e.g. check boxes rather than narrative boxes). We anticipate even less departmental help as we have lost one office manager, and the secretary’s hours have been reduced.