EDAD 6335K-16 Accountability Policy Research

Spring, 2012

Thursdays, 5pm-7:50pm

Trimble 110

Instructor: Barbara Tobolowsky, PhD

Office: 103G Trimble Hall

E-mail –

Phone: 817-272-7269

Office Hours – By Appointment

Goals of the course:

This course explores student outcomes assessment with particular emphasis on the role of outcomes assessment in institutional accountability and accreditation. It also addresses the use of assessment at a local level, such as a classroom, as well as a national level.

Specific Course Objectives:

Students will learn assessment approaches that can be applied at the national, institutional, and programmatic levels.
Students learn to identify appropriate methods to address a variety of learning outcomes.
Students will designan assessment plan utilizing the appropriate methods and conduct a small-scale assessment.
Students will improve their writing and presentation skills.

Attendance Policy

An important aspect of this course is class participation and the learning, which takes place in the classroom. Students are expected to attend all class sessions. If circumstances prevent you from attending class, please contact me prior to the class meeting as a professional courtesy. You are expected to get material from classmates for any missed class sessions. Absences or repeatedly arriving late to class or leaving early will result in the reduction of your course grade.

Academic Integrity

The University of Texas at Arlington has clearly articulated its policies governing academic integrity and students are encouraged to carefully review the policy on Academic Responsibility. The area of greatest potential risk for inadvertent academic dishonesty is plagiarism. Students should also read closely the discussion of avoiding plagiarism that is included in the APA Manual. Any evidence of plagiarized material will result in an F on that assignment with no opportunity for revision.

Grading Policy:

A = 90-100

B = 80-89
C = 70-79

D = 60-69

Required Texts:

Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the

institution.Sterling, VA: Stylus.

Publication manual of the APA (6th edition) (2010). Washington, D.C. : APA.

Additional reading available on Blackboard.

Course Expectations

Over the years of teaching, I have learned that one of the main issues in a classroom is if there is a disconnect between course expectations and reality. For this reason, I list my expectations for excellence and will seek your expectations as well, which I will append to the syllabus. I think by openly discussing our assumptions, we end up clearing up any discrepancies early on and have a more positive classroom experience.

1. If you end up missing a class, I expect you to get any material missedfrom your colleagues.

2. I expect you to come to class prepared.

3. I expect class to start and end on time.

4. I expect everyone to treat each other in a professional way. I believe that means that we treat each other with respect and courtesy, but are open to questions. Questions make us all better, if they are presented in a thoughtful manner.

5. I expect all assignments to be turned in on time. All late assignments will lose a half letter grade for every day that they are late.

6. I expect all assignments to be well-written, which at a minimum means that the paper has no typos and no grammatical or punctuation errors. I understand that writing at a high level is challenging for all of us, so I hope that these other goals are ones you will work on during the course. These goals are (a) to clearly express your ideas with transitions, so the ideas flow from one to the other, and (b) to support all key ideas with evidence. In the Analysis/Discussion section, the evidence comes from your data.

7. Since I expect in-class engagement, students will not text or do email while in class. The use of smart phones and computers in class will only be for class purposes. If I find that you are using these devices for personal purposes during class time, it will affect your participation grade.

8. Follow the style guidelines contained in the Publication Manual of the American Psychological Association (6th ed., 2009) for papers. It is important that all work be cited based on the APA format. Because of the importance of the issue in academe and the field, any evidence of plagiarism results in an “F” on that assignment, if you do not revise it. If you choose to revise the assignment, you will have one week to resubmit the assignment once it has been returned. There will be an automatic reduction of one full letter grade on the revised paper. For example, if you earn an “A” on the revision, it would end up a “B” and so on).Plagiarism is the use of any one else’s ideas or words without providing appropriate attribution. (See page 15-16 in the APA manual.)

9. I expect to provide you with thoughtful, detailed, and prompt feedback one week after a paper is submitted (or the next class meeting). My intention is for the feedback to help you improve your work, so please let me know if is unclear or unhelpful.

10. I expect you to set a time to meet with me if anything is unclear in the class. I want to help you succeed, but I can only do that if you let me know about any problems you are having.

Your expectations (TBA):

Grading:

***All written assignments should:

Be on time and sent electronically. (No late assignments will be accepted)

Use a file name that puts your own name first. (e.g., Tobolowskydesign.doc or docx)

Be written and submitted in Microsoft Word.

Be in 12-point Times New Roman font

Be double-spaced with standard margins, including the references

Not have typos or misspellings

Follow APA (6th ed) style

Grading and Assignments:

Attendance and Participation (entire semester – includes thoughtful responses during class discussion, providing timely feedback for presentation, general class engagement, and at least five comments or questions about class sessions posted to Blackboard throughout the semester) – 20%

Assessmentplan (week 4)- 10%

Assessment protocol and/or survey (week 5)– 10%

Paper pt. 1 - Introduction, literature review/background, method (week 7)– 20%

Final Presentation -10%

Peer Review of final assessment report (week 12) 10%

Final assessment report (week 14 for peer review and week 15 to be turned in)– 20%

Participation- (20%)It is essential that you come to class prepared to discuss the readings and your own assessments. Therefore, this grade will be based on your active participation in weekly in-class discussions. In addition, I have set up a discussion section on each class session on Blackboard and ask that you go to that site and provide some feedback for at least five of the class sessions. I ask that one of those weeks is Week 7, because I conduct a brief formative assessment.

In the last class, I plan to bring in a hard copy evaluation to get your feedback then as well.

Assessment plan - Due week 4 (Feb. 9th) (10%) - Two-page assessment design includes an introduction to the program/class to be assessed, goals associated with the program/class, what a full assessment plan would be, and what you plan to address in this class, and what method(s) you will use. You do not have time to direct a full assessment, but I think it is valuable to be able to develop a complete plan. We will decide as a class the scope of your assessments for the class. It will be determined before this assignment is due. You will submit the assessment plan electronically by class time. (Submit Document file as: Last name Plan.doc or docx)

Assessment survey and/or protocol (10%) Due Week 5 (Feb. 16) This instrument/protocol is what you intend to use for your assessment. It is due by class time, so we can spend class time revising the protocol/survey, as needed. Submit the final version to me before the next class meeting for final approval.

Introduction, literature review/background, and assessment design or method sections – (20%) due week 7 (March 1). The introduction includes the reason a program or initiative was developed, information on the program, and the assessment question (Is X effective?). Often, even assessments require knowledge of similar work that has been conducted in the area. Therefore, some of you will want to include a brief literature review that includes the previous work that is relevant with this topic. In some cases, only a background on the topic is what is required. All papers will need a brief, but complete background section that includes specifics of the program (e.g., what it is, how it works, who participates, its goals). Not all papers will require a literature review. In the method section, include your assessment design (e.g., method you are using and why, sample numbers and rationale, and recruitment of sample). These sections will be the first sections of your final paper and should be approximately 4-5 pages. You will submit this electronically (File name: Last name Assignment.doc or docx) by class time.

Presentation – (10%)You will do a formal presentation of your assessment project in class. You will have 15 minutes for your presentation and 5 minutes to answer any questions. Your grade will be lowered if you go over your time limit. At conferences, you have a limited amount of time to present your work, so it is a good skill to learn.

Peer review of final paper – (10%) Due week 11 (April 5). Bring in a complete hard copy of your final paper for peer review.

Final assessment report (20%)– Due week 15 (May 3th). (Page lengths are approximate.)

Length - 12-15 pages - Paper will include 7 sections: an executive summary – Key issues and findings (no more than one page) an introductionthat will discuss why the issue and program/course/initiative is important (in other words, the rationale of your program/class) (approx. 1 page), a brief literature review/ Background (is this an area that others have studied, what does literature tell you about the issue/course/program/student population/goal? Basic information on the program/initiative) (no more than 2 pages), assessment design and goal of the assessment, introduction to population, if appropriate (no more than 2 pages), findings (4-6 pages), plans for improvement or list of recommendations (1-2 pages), Conclusion (no more than 1 page).

The final paper is submitted electronically by 5pm on May 3th. (Submit as follows: Your last name Final report.doc or docx)

We will determine the scope of the assessments as a class.

Class sessions:

Week 1 - Jan 19 Introduction to course. What is accountability? What is assessment? What is research? Are they the same?

Assignment:

I’ve asked you to go the Blackboard discussion website to provide your thoughts after five of the classes (one of those weeks is week 7 for the formative assessment). I invite you to start this week.

Week 2 –Jan 26 K-16 Accountability and Policy

Reading:

Amrein, A.L. & Berliner, D.C. (2002, March 28). High-stakes testing, uncertainty, and

student learning. Education Policy Analysis Archives, 10 (18). Retrieved January 9, 2012 from (in Blackboard)

This article is just over 50 pages long. I would recommend reading the first 10 pages (to the Education, Learning, and Training section) and skimming the rest, if you have limited time. It is interesting take on the policy implications of high-stakes testing.

Hursh, D. (2007). Exacerbating inequality: The failed promise of the No Child Left

Behind Act. Race Ethnicity and Education, 10 (3), 295-308. (In Blackboard)

Shavelson, R.J. (2010). State higher-education accountability and learning assessment

(pp. 133-160). Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press. (still need to put in Blackboard)

Brittingham, B. (2008, Sept./Oct). An uneasy partnership: Accreditation and the federal

government. Change, 40, 5, p. 32-39. (In Blackboard)

This article focuses on the tensions between uniform measures and the need to consider the uniqueness of institutions in measuring quality. The federal government’s goals are in conflict with institutions’ positions regarding assessment.

Assignment:

Remember to check in to the Blackboard website after class to provide your thoughts about the class.

Week 3- Feb 2 Introduction to Programmatic and Classroom Assessment Approaches. Developing learning objectives. Intro to making an assessment plan.

Introduction to qualitative and quantitative assessments.

Assignment:

Remember to check in to the Blackboard website after class to provide your thoughts about the class.

Reading:

Assessing for Learning – chapter 3-4

Shavelson, R. J. (2010). Exemplary campus learning assessment programs (pp. 70-101).

Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press. (Still need to put in Blackboard)

Week 4 – Feb. 9 Workshop – assessment design and creating surveys/protocols

Assignment:

Remember to check in to the Blackboard website after class to provide your thoughts about the class.

Reading: Assessing for Learning – pages 4-6

Assessment plan due:Two-page assessment design includes an introduction to the program/class to be assessed, goals associated with the program/class, and the method you intend to use to assess the effectiveness of the program or class.

Week 5 - Feb. 16–Workshop on the development of protocols/surveys

Assignment: Email Protocol or survey

Question: The class will decide about meeting next week regarding the writing assignment due week 7 or if there are some issues you are confronting in your assessments. Let me know. You can voice your thoughts in class or in the Discussion section (week five or week six) in Blackboard.

Week 6 - Feb 23 – Workshop on the development of protocols/surveys (class meeting TBD) If needed, this class session can be focused on writing an assessment.

Week 7–March 1 – Analyzing data – Part 1

Assignment:Introduction, literature review, and method section of paper due. (4-5 pages)

Remember to check in to the Blackboard website after class to provide your thoughts about the class.

Week 8– March 8TBD

March 15 No meeting spring break

Week 9 - March 22 – Analyzing data – Part 2

– Update on your assessment projects

Week 10–March 29 – Workshop (class meeting TBD)

Week 11 - April 5- Feedback loop - Using data for improvement

Assignment: Peer review papers

Week 12 - April 12- No class meeting - AERA

Week 13 - April 19 – Presentations on assessment project

Week 14 - April 26 - Presentations on assessment project

Week 15 - May 3

(Last day of class) Presentations – Papers due by class time

Class evaluation.