2014-2015 Annual Program Assessment Report (Final Draft 12/15/15)

2014-2015 Annual Program Assessment Report (Final Draft 12/15/15)

Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to , director of assessment and program review, by September 30, 2015. You may, but are not required to, submit a separate report for each program, including graduate degree programs, which conducted assessment activities, or you may combine programs in a single report. Please identify your department/program in the file name for your report.

College: Humanities

Department: Chicana/o Studies

Program: B.A.

Assessment liaison: Rosa RiVeraFurumoto

Please check off whichever is applicable:

A. _____X___ Measured student work.

B. ______X__ Analyzed results of measurement.

C. ______Applied results of analysis to program review/curriculum/review/revision.

Overview of Annual Assessment Project(s). On a separate sheet,provide a brief overview of this year’s assessment activities, including:

an explanation for why your department chose the assessment activities (measurement, analysis, and/or application) that it enacted
if your department implemented assessment option A, identify which program SLOs were assessed (please identify the SLOs in full), in which classes and/or contexts, what assessment instruments were used and the methodology employed, the resulting scores, and the relation between this year’s measure of student work and that of past years: (include as an appendix any and all relevant materials that you wish to include)
if your department implemented assessment option B, identify what conclusions were drawn from the analysis of measured results, what changes to the program were planned in response, and the relation between this year’s analyses and past and future assessment activities
if your department implemented option C, identify the program modifications that were adopted, and the relation between program modifications and past and future assessment activities
in what way(s) your assessment activities may reflect the university’s commitment to diversity in all its dimensions but especially with respect to underrepresented groups
any other assessment-related information you wish to include, including SLO revision (especially to ensure continuing alignment between program course offerings and both program and university student learning outcomes),and/or the creation and modification of new assessment instruments

3. Preview of planned assessment activities for next year. Include a brief description and explanation of how next year’s assessment will contribute to a

continuous program of ongoing assessment.

2. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment activities:

RESPONSE: This year’s focus was on collecting, processing, analyzing and interpreting exit writing data from the capstone course CHS 497 Senior Seminar in Chicana/o Studies,taken by all Chicana/os Studies majors and double majors. The idea was to compare these results to last year’s analysis of CHS 497 to contrast and compare the findings and to note any patterns or trends that could inform our departmental teaching and learning processes.

Goal: Our goal was toconsider what the data were showing regarding the writing of CHS undergraduate majors and double majors at the exit point and to determine next steps for addressing the findings.

Program SLO’s: SLO 2: Demonstrate competency in oral, written, and research skills.

Context: The CHS 497 Senior Seminar in Chicana/o Studies is a course required of all Chicana/o Studies majors, double majors and Chicana/o Studies Social Science Subject Matter Program candidates. This integrative seminar serves as a capstone to the interdisciplinary major in Chicana/o Studies. One of the major assignments in this course is a 5-7 page research paper that must address an issue relevant to Chicana/os. The paper requires 15 scholarly sources and must use APA formatting style.

Assessment Instrument: The instrument used to assess students’ research papers was the “Chicana/o Studies Portfolio Evaluation Rubric” which was used to score research papers for the entry point CHS Stretch courses (CHS 113, CHS 114A, CHS 114B, and CHS 115). This same rubric was used in last year’s program assessment of CHS 497 (see CHS dept. 2013-14 Assessment Report). The instrument scores eleven aspects of the research paper according to a “High Pass; Pass; Low Pass; and No Pass.” The paper is then assigned an overall score of “4 for High Pass; 3 for Pass; 2 for Low Pass; and 1 for No Pass.”

Sample: One section of CHS 497 provided all the papers for the CHS majors and double majors. For this sample N=10.

Assessment Process: Three CHS faculty member served as raters of the CHS 497 final research papers. They met to review the rubric, discuss the categories, and calibrate themselves using sample research papers. All of the papers were read and scored by at least two readers and 40% of the papers were read and scored by all three raters. Inter-rater reliability was 9/10 or 90% with 10 opportunities to agree or disagree on the final overall score of High Pass; Pass; Low Pass; and No Pass.

Findings: N=10; 6/10 or 60% of the papers scored at a High Pass or 4.0 on a scale of 1-4 with 4 being the highest; 4/10 or 40% scored at a “Low Pass” level or 2. The average exit score was 3.2.

Analysis and Discussion:It is interesting to note that these results do not reflect a bell-shaped curve and instead indicate two extremes of either a High Pass or a Low Pass. Whereas the sample size is small, it suggests that we have a lot to do to bring more students into at least the Pass Level or 3.0. For comparison, 2013-2014 Assessment Program Results were as follows: N=14; 6/14 or 43% of the CHS 497 research papers were scored as High Pass or 4.0 on a scale of 1-4; 8/14 or 57% scored a Pass or 3.0. There were no Low Passes and the average exit score was 3.42 (between a Pass and a High Pass). Whereas, the two years of data do not necessarily indicate any trends, we are seeing similar patterns in at least two areas that deserve our attention.

Whereas a higher percentage of students 60% for this year, as compared to 43% for last year are scoring a High Pass, we are very concerned about the high number of Low Passes 40% for 2014-2015. Among the Low Passes we are still seeing a significant number of grammatical errors and errors consistent with Second Language (L2) learners. As in last year’s report, we definitely see the need for greater attention to supporting L2 learners’ writing via training for faculty on how to address L2 needs in writing, writing tutors, and support for the writing labs. There is a definite need for an institutional response.

In addition, we observed a general need for students to improve in their ability to critically analyze and interpret scholarly sources. This is also consistent with the 2013-2014 findings.

Conclusion: There is a need for a focused response to CHS Dept. majors’ and double majors’ academic writing. We need to figure out what type of classroom writing and critical analysis assignments and processes are being used in CHS classes and, among these, which may be more successful in supporting students’ academic writing and critical analysis skill development.

Relation to the University’s Commitment to Diversity: We see a definite connection between the department’s assessment activities and the University’s commitment to diversity. First, as a Hispanic Serving Institution (HSI) CSUN enrolls a large population of Latina/o students that also includes many Second Language (L2) learners. As a group many Latina/o students are low income and the first in their family to attend college. As historically underserved students they may need more support to stay in college and graduate successfully. In addition, research is consistent in showing that L2 learners often have predictable patterns of errors within their writing. Faculty and staff could use support to know about these patterns of errors and how to best address them and support the student’s progress in writing. There is a need for a focused well-planned and resourced institutional response to the needs of L2 learners. In addition, a significant number of Latina/o students including Native English speakers also come to CSUN lacking in preparation due to historical and institutionalized inequities such as poverty, poor schooling, etc. This means that faculty and staff need training and resources to best support specific student populations in their writing and critical analysis. Then we need ongoing support as we implement changes in the teaching and learning process in order to gauge whether interventions are working.

3. Preview of planned assessment activities for next year. Include a brief description and explanation of how next year’s assessment will contribute to acontinuous program of ongoing assessment.

Response: We are committed to do the following as part of next year’s assessment process:

Find out what scholarly sources have to say about supporting students’ writing in general and L2 learners’ writing in particular. For now these two questions can help to focus our efforts:

How do we best meet the writing needs of L2 learners?
How do we best support all learners to be able to critically interpret, analyze and discuss scholarly sources?

Commit to a half day faculty meeting (retreat) with a focus on three key courses (we are considering the selection of two or three of the General Education courses that also meet major requirements. We want to be mindful to select full time faculty for participation in this process. Our idea is to discuss the Assessment Report findings and rubric with them to figure out next steps on how to support our students’ writing development.
Using these 2-3 courses as a beginning point, we would examine course assignments and/or processes to have a beginning point for understanding how writing and critical analysis is addressed in these 2-3 courses. We would also like to see where there may be gaps and/or overlap in the skills and processes used in these courses.
Have the involved faculty come up with 1-2 changes to address student writing. Stay focused on 1 or 2 changes to see how this impacts student writing. Keep it simple and focused. Continue to collect relevant data on students’ writing.For example, one professor of CHS 497 has made the commitment to make some concrete changes in their approach with the literature review assignment for the class. The professor will be trying out the use of “sentence frames” to help students write the literature review. The idea is to provide them with a useful tool for writing up the empirical studies they have reviewed.