To: David Malik, Interim Executive VCAA
Charles Gallmeier, Chair, Faculty Organization
From: Mary Ann Fischer, Co-chair, General Education / Assessment Committee
Date:August 10, 2009
Re: IU Northwest General Education/Assessment Committee Report to VCAA and the Faculty Organization
The Faculty Organization adopted the Student Learning Outcomes for Principle 1 at the January 16, 2009 meeting. These outcomes are posted at our web site at
The General Education/Assessment Committee met three times during the spring 2009 semester. Two meetings were devoted to a discussion of the definition and learning outcomes for Principle 4, Diversity, because the College and Arts and Sciences Curriculum Committee had had some difficulties interpreting the definition in order to make course recommendations to the other schools and divisions. At the conclusion of the second meeting, the General Education / Assessment Committee agreed to let stand the original definition passed by the Faculty Organization on October 17, 2008. The definition can be seen at
On May 12, the committee conducted an assessment of writing skills using 46 papers written by students from W131 and advanced classes. A report summarizing the results of this assessment is attached. The results will serve as a benchmark against which future writing assessments will be judged and will also form the basis of a general discussion of writing skills and how to improve them among our students.
The committee chair forwarded to the Interim Executive VCAA a suggested schedule for the integration of revised general education requirements into school and division curriculums. The schedule calls for the integration plans to be submitted for approval to the Survey and Curriculum Committee of the Faculty Organization by November 1, 2009. A form for documenting the planned changes for each degree program was successfully pilot-tested by the School of Business. A copy of this form, “General Education Curriculum Integration, 2009” is attached.
An amount of $5000 was designated for stipends and materials needed fora year of Intensive Writing workshops. The first 3 workshops in the series, sponsored and advertised through CETL, were presented during the spring 2009 semester. Faculty writing specialists from the department of English led the workshops,entitled Designing Writing Assignments (Feb. 24, 25); Responding to Student Writing (Mar. 24, 25); and Assessing Student Writing (April 14, 15). Additional workshops are planned forfall 2009 and may include an outside speaker on the subject.
Our Committee thanks Interim Executive Vice Chancellor Malik for the help and support he has provided for the General Education initiative.
Assessment of Writing Skills among Beginning and Advanced Students
IU Northwest General Education/ Assessment Committee
May 12, 2009
Report Date: August 8, 2009
Participating Committee Members: Mary Ann Fischer (co-Chair), Doug Swartz, Barbara Peat, Latrice Booker, Sheila Marie Trzcinka, Linda Delunas, Karen Evans, Dee Dee Ige, Jim Thomas,
Eva Mendieta, Cynthia O'Dell, TaylorLake
Introduction
During May of 2009, the combined General Education/Assessment Committee of the IU Northwest Faculty Organization completed an assessment of writing skills revealed in a sample of forty six final papers selected from either W131, Elementary Composition, or from upper division courses in each of the six schools and colleges. This report details the procedures used the solicit the papers, to review and evaluate them, and the results as revealed in quantitative judgments and comments of the faculty evaluators who served on the committee. We hoped to establish a baseline of evidence for writing skills and to find evidence of more advanced skills among upper division papers than among W131 papers.
Selection of Writing Samples
The Associate Vice Chancellor of Academic Affairs requested that the Deans of the schools and colleges solicit sample papers from their faculty. The request was for at least two papers along with descriptions of the assignment, from a given instructor, one judged by the instructor as relatively skilled and the other as relatively weak. All identifiers except the course number were stripped from the papers before they were submitted to the AVCAA. The final sample included 23 papers drawn from W131 sections and 23 papers from upper division courses. All schools and colleges submitted at least 2 papers from upper division courses.
Assessment Procedures
The committee adapted a tool designed by Professor Anne Baylay, Department of English. It included eight categories, including definitions, of writing skills: research, persuasion, prose and syntax, counterargument, mechanics, complexity, reader pleasure, and organization, (See attachment). Each evaluator read three or four papers and judged them as excellent (3 points), acceptable (2 points), or unacceptable (1 point) in each category; additional open-ended comments were optional. Each paper was evaluated by only one rater; thus no inter-rater reliability can be reported.
Results
The mean rating for each skill category among W131 and upper division papers is shown in the table below. As expected, in every category upper division papers were given higher ratings than those from W131. Readers knew the course level of the paper, so expectations for better performance on the upper division papers may have influenced ratings. Future assessments should remove the course identifiers. Ratings for the W131 papers were low, that is, the means were below 2, defined as “Acceptable.” Mean ratings for the upper division papers were just above “Acceptable” in all but 2 categories.
Course LevelW131 (N=23) / Upper Division
(N=23) / Combined
sample
Mean / Mean / Mean
Research / 1.83 / 1.96 / 1.89
Persuasion / 1.74 / 2.17 / 1.96
Prose and syntax / 1.65 / 2.23 / 1.93
Counterargument / 1.61 / 1.68 / 1.64
Mechanics / 1.61 / 2.13 / 1.87
Complexity / 1.65 / 2.13 / 1.89
Reader pleasure / 1.73 / 2.09 / 1.91
Organization / 1.68 / 2.02 / 1.86
Total points / 13.70 / 16.24 / 14.97
The comments (see attachment 2) about the papers were much more negative than the mean ratings would have suggested in the case of the upper division papers. It may be that raters hesitated to classify the upper division papers us “Unacceptable” even though they were highly critical. If so, future assessment efforts should use a rating scale with a wider range.
In general, evaluators found the experience of reading student papers valuable and useful. The committee did not discuss the question of what steps should be taken to improve the quality of student writing. Comments and suggestions on the process included the following:
1) Our request to the units for sample papers should be clearer.
2) The rubric should be more universal to any assignment; suggested areas of
universal assessment include: focus, development, mechanics, philosophy
3) Sheila Marie T. brought a rubric that we might consider for next time
rather than the one we used this time.
5) Suggestion that everyone, including faculty, have a Handbook (perhaps purchased by the
university) that is used in most w130 or w131 classes to encourage students to
keep them and use them in all future classes.
Attachment 1 Writing Assessment Form
General Education/Assessment Committee Assessment Project
May 12, 2009
Reader ______Paper #______
This list of desired attributes has been adapted from one designed by Anne G. Balay (), although she strongly favors holistic assessment rather than evaluating attributes separately, as we will be doing here. She has argued that after a student has completed one year of a college-level writing course, a final essay should demonstrate most of these attributes. After reading each essay please use the definitions which have been provided to rate each sample paper for each attribute.
Excellent3 points / Acceptable
2 points / Not acceptable
1 point
use of research
effectiveness of argument or persuasion
style reflecting prose and syntax
use of counterargument without bias or ethos
attention to mechanics and punctuation
complexity of ideas
reader pleasure
structure reflecting organization and focus.
Total points in each category
Total points: ______
Additional comments:______
Definitions of the categories:
Use of research: Students should execute both library and Internet research in gathering sources for their projects. Sources appearing in student papers should be drawn from electronic databases, and contain a combination of academic, public, and primary sources. Students are expected to demonstrate an ability to summarize, analyze, and synthesize sources appropriate to academic writing.
Effectiveness of argument or persuasion: Students are expected to establish a research driven thesis supported by warranted claims that shaped their purpose in writing. Claims are supported by evidence gleaned from research, primarily, as well as from personal experience and common knowledge.
Style reflecting prose and syntax: At the surface level, the writing should be clear. Sentences should reflect a degree of sophistication appropriate to a college writer and should flow in prose that sustains the argument.
Use of counterargument: The essay should include a counterargument as reflecting a writer’s sense of ethics and fair-mindedness in working a thesis. Furthermore, a writer should guide the argument without undue bias, acknowledging opposing opinions and points of view.
Attention to mechanics and punctuation: Writing should be clear and free of errors. The writer should demonstrate mechanics that reflect the grammar of Standard American English, free of slang and jargon, and should attend to the impact of punctuation on the reader.
Complexity of ideas: Papers should reflect thoughtful interpretations and applications of research (considered critical thinking). As such, successful papers avoid pedestrian arguments on mundane subjects.
Reader pleasure: A good essay interests its writer as well as its reader, and is fresh in ideas and well written.
Structure reflecting organization and focus: Deep issues of organization and development define this category. A paper’s thesis and claims should guide the reading, and those claims should be developed through extended discussions and use of research as support.
Attachment 2 Comments on papers
Total points / Course level / Comments8 / 131 / this essay does not follow assignment, does not provide an adequate or clear presentation of the sources pursue a thesis, provide fresh ? Or cite sources. Furthermore, it contains an array of grammatical errors
8 / 131 / a lot of serious errors in sentence structure. I get no sense of her sources, arguments and writer relies too much on personal experience
9 / 131 / mostly writer opinion, lack of research support. Fails to cite sources in text references unacceptable, many writing errors (mechanics)
9 / 131 / works cited page has appropriate references but they are not really used in text as a basis for the "argument" this is simply an "opinion piece" with attempts to make it appear as if the research supports the writers opinion. Completely unsophisticated, no fresh ideas, very repetitive
10 / 131
10 / 131
10 / 131 / major claim was not clear enough and sub-claims weren't clear and didn't adequately support and develop major claim. Use of research indicates lack of critical reading skills.
11 / 131
11 / 131 / no central thesis other than title, just a flow of thoughts with personal example and a few quotes supporting only the authors point of view, problems with grammar
12 / 131
13 / 131 / Very mixed, when formal scores were low, in middle,l when personal, much better
14 / 131
14 / 131 / little depth to analysis
15 / 131
15 / 131 / flow more conversational than formal essay - no central thesis and so supporting TPs don't have connection other than conversational, but sent structure good and mechanics
16 / 131 / one of the sources is not cited. There are many generalizations about minorities and communities that would seem to require documentation
16 / 131
16 / 131 / met the criteria
17 / 131 / opening and closing ¶ reflect acceptable writing (whereas middle ¶'s seem more disorganized and subject to inappropriate sequence of ideas
18 / 131
19 / 131
21 / 131 / Basically good, arguments occasionally personal, bit long
23 / 131
8 / upper division / this was unacceptable for a 400 level class
8 / upper division / Student did not follow instruction in assignment and did not seem to even understand the point of the assignment
8 / upper division / no citations/references, can't tell what kind of assignment this is. Not required length for a take-home paper. No book identified (except 1984) so not a book review
10 / upper division / If student ? Article o a study to answer the questions in the instructions then, according o the ? Noted deficiencies in information the ? Should not have ? This article for review
12 / upper division / did not follow instructions of assignment; weak on syntax and grammar which hindered reader understanding - was persuasive in argument that good source for classroom instructor and did connect approp. To the standards
12 / upper division / show and tell paper
13 / upper division / Frankly, I didn't understand the assignment so evaluated the paper totally on its own. Was research required?
14 / upper division / ineffective use of transitions, spelling errors, and sentence structure errors detract from this broad look at schizophrenia and cautionary view of certain medications
15 / upper division / the paper is too much driven by summary: it's often not clear why we are being provided with description of people and events from the book being reviewed. There needs to be a better sense of purpose in the writing, a central evolution
15 / upper division / incorrect APA format, some misuse of words, assignment calls for figures to be placed at end of paper but writer includes them with text, conclusion is vague/cursory
16 / upper division
18 / upper division
18 / upper division
18 / upper division / good grammar syntax
18.5 / upper division
19 / upper division / Citations in text lack year and ?. Student uses footnotes instead -maybe okay? Problems with confusing outline structure. Intro poorly developed
20 / upper division
20 / upper division / well written paper
21 / upper division / good writer, little or no evidence of field research visits to the church
22 / upper division
22 / upper division / Good
23 / upper division / there weren't a sufficient number of sources, and I wasn't sure how the source provided historical background for the memoir. The citation system is also confusing
23 / upper division / well documented, effectively written paper!
General Education Curriculum Integration, 2009
To be completed and submitted to the Survey and Curriculum Committee by all Schools and Divisions for all degree programs by Nov. 1, 2009
Please use this form to indicate how your programs will meet the revised General Education Curriculum Requirements
Degree program(s) e.g. Psychology B.S.______School/Division______Date______
Principle 1 Foundations and Advanced Courses / How will the class requirement and learning goals be met?W131 / Required
S121 / Required
One course among the following: M100, T101, M118, M119, M125, M215
Natural science course with lab
At least twoIntensive Writing courses Guidelines
At least one Advanced Oral Communication course Guidelines
At least one Advanced Mathematical Reasoning course (e.g. K200, E270, K300, M216 and higher math courses, etc.) Guidelines
At least one Advanced Scientific Reasoning course (either natural or social sciences) Guidelines
At least one Information Literacy course Guidelines
Principle 2 Breadth of Learning* / How will the class requirement and learning goals be met?
Arts and Humanities-6 credits
Cultural and Historical Studies- 6 credits
Social and Behavioral Sciences-6 credits
Mathematical, Physical and Life sciences-3 credits
Principle 3Critical Thinking ** / How will the class requirement and learning goals be met?
Mandatory capstone course in the major or equivalent experience as determined by the unit.
Principle 4 Diversity** / How will the class requirement and learning goals be met?
Some of the Principle 2 courses and advanced/writing-intensive courses from Principle 1 should be designated as diversity courses.
Principle 5 Ethics and Citizenship**
Some of the Principle 2 courses and advanced/writing intensive courses from Principle 1 should be designated as ethics or citizenship courses.
*Credits used to satisfy Breadth of Learningmay also satisfy Advanced Coursework in the Principle 1. A student may not use the same class to satisfy more thanone domain in Breadth of Learning.
**Units must specify how their students will meet the outcomes for principles 3, 4, and 5. Each unit will provide evidence that students have met the appropriate outcomes. Units are responsible for creating or designating the necessary courses and/or enriching experiences that fulfill the requirement and for assessing the outcomes.
**Guidelines: Outcomes for principle 3, 4, and 5 will be met by a class or by an enriching educational experience such as service learning, an internship, a capstone project, student teaching, an independent study research/creative activity, study abroad, creation of an exhibit, or a performance.