Defense Against the Dark Arts 2014-2017Assessment Plans
Submitted to the
Academic Programs Assessment Committee
By
Elladora Cresswell, N.E.W.T.,
Chair, Defense Against the Dark Arts Department
October 31, 2014
Defense Against the Dark Arts Assessment Plan 2014-2017
Defense Against the Dark Arts (DADA)will not make any changes to thefive student learning objectives (SLOs) developed in the first cycle of data. Therefore, our five learning objectives are:
SLO1:Students will be able to explain the fundamental differences between the light arts and the dark arts (i.e. Intention, Outcome, and Context). (Assessed in DADA 100: Intro to the Defense Against the Dark Arts)
SLO2:Students will be able to perform the basic defensive spell set (i.e. Expelliarmus, Impedimenta, and Stupefy). (Assessed in DADA 106: Basic Defense Practicum).
SLO3:Students will demonstrate understanding of Stevenson’s Laws for magical defense. (Assessed in DADA 601: N.E.W.T. Preparatory Course I.).
SLO4:Students will demonstrate the ability to identify threats in a target-rich environment and minimize collateral damage. (Assessed in DADA 206: Intermediate Defense Practicum, and DADA 306: Advanced Defense Practicum).
SLO5:Students will be able to perform the advanced defensive spell set (i.e. Protego, Horribilis, and Expecto Patronum). (Assessed in DADA 361: O.W.L. Preparatory Course II).
Two student learning objectives are assessed annually across a three year cycle. Thus, our assessment schedule is:
Assessment Year / Student Learning Objectives Assessed2015 / SLO1:Students will be able to explain the fundamental differences between the light arts and the dark arts (i.e. Intention, Outcome, and Context).
SLO 3:Students will demonstrate understanding of Stevenson’s Laws for magical defense.
2016 / SLO2:Students will be able to perform the basic defensive spell set (i.e. Expelliarmus, Impedimenta, and Stupefy)
SLO4:Students will demonstrate the ability to identify threats in a target-rich environment and minimize collateral damage.
2017 / SLO5:Students will be able to perform the advanced defensive spell set (i.e. Protego, Horribilis, and Expecto Patronum)
Specific SLO Assessment Strategies
Year 1: SLO1 & SLO3 Assessment
SLO 1: Explainand Apply Fundamental Differences
SLO 1 will be assessed in our introductory course, DADA 100: Introduction to the Defense Against the Dark Arts. DADA 100 is taught every Fall semester and is the first class incoming first years take in the DADA sequence. Because these are fundamental concepts, we expect students to be able to achieve mastery by the time they finish this course. Since there are multiple sections of this class, we will make sure to collect data on more than one section (to avoid inter-house differences); however, assessment will remain the same.
The introductory class contains both academic and practical sections, andthis SLO will be assessed in both. For the written portion, the faculty will work together to create short answer questions that will assess this knowledge and will insert them into the exams. There are two exams – midterm and final – items will be inserted into both.
For the practical portion, the student is involved in training scenarios throughout the semester that demonstrate the ability to apply the three fundamentals. The four full-time faculty members in DADA developed an evaluation rubric during the last round of assessment for this section of this assignment through consensus agreement. Care was taken to establish consistency in evaluation across the faculty members and inter-rater reliability was high. The rubric was as follows:
For the academic portion:
Not MeetingApproachingMeetingExceeding
ExpectationsExpectationsExpectationsExpectations
(1-2 points)(3-4 points)(5-6 points)(7-8 points)
Cannot explain Can explain concept butExplains conceptExplanation
concept, nor apply application is not correct correctly,is exceptional,
it to the given application isapplication is
problemacceptablecorrect
For the practical portion:
Not MeetingApproachingMeetingExceeding
ExpectationsExpectationsExpectationsExpectations
(1-2 points)(3-4 points)(5-6 points)(7-8 points)
Consistently fails Often fails to apply concept,Often applies Consistently applies
to apply conceptbut understands when concept, when theyconcept in practice.
in practiceconcept appliesfail, they understand
why.
We will continue to use this rubric for this round of assessment as it worked well last time. Once the students written and practical work have been evaluated, the criterion for achieving SLO 1 will be that 80% of the students meet or exceed expectations.
SLO 3: Demonstrate Understanding of Stevenson’s Laws
SLO 3 will be assessed in DADA 401: N.E.W.T. Preparatory Course I. DADA 401 is taught in the Fall semester, and students that pass move on to DADA 402: N.E.W.T. Preparatory Course II which is taught solely in the Spring Semester. A requirement for passing DADA 401 is a term paper unpacking the ramifications of Stevenson’s Laws in relationship to the need to keep the Wizarding World a secret from the Muggle World.
Stevenson’s Laws are as follows:
1. A wizard may not injure a muggle; or, through inaction, allow one to come to harm.
2. A wizard may protect their own existence as long as such protection does not conflict with the First Law.
3. A wizard must obey the laws of the Ministry of Magic, except where such laws would conflict with the First or Second Laws.
While not all papers follow the same argument or address the laws in the same way, all of the papers require a section explaining the laws themselves. First, as Stevenson intended them; then as they were reinterpreted during the Defense Summit ten years later.
The four full-time faculty members in DADA will develop the evaluation rubric for this section of this assignment through consensus agreement. Care will be taken to establish consistency in evaluation across the four faculty members. Inter-rater reliability will be established. Once all papers have been evaluated, the criterion for achieving SLO 3 will be if 80% or more of the students meet or exceed expectations.
Year 2: SLO2 & SLO4 Assessment
SLO 2: Perform Basic Defensive Spells
SLO2 is assessed in DADA 106: Basic Defense Practicum. This class is taught in the Fall semester and is the first (of 4) of our required practical defense series. This series is designed to give witches and wizards a chance to practice all defensive spells they might need in a safe environment. The theory of the basic defensive skill set (i.e. Expelliarmus, Impedimenta, and Stupefy) is taught in the semester prior to DADA 106 (in DADA 102: Basic Defense Theory) and so practice on this set begins on the first day of class.
After the Second Wizarding War, the Defensive Decree required that all wizarding schools (as a condition of accreditation) require a practicum on defensive skills and that all students be able to complete the basic defensive skill set before they are able to graduate. As such, our expectations for this particular SLO are higher than the others and mastery is required to pass the course.
Therefore 100% of the students need to be at or above the expectations by the end of the semester. If they are not, they must re-take the course. The tests in this class directly assess these exact concerns so the scores related to the three spells in question can be used as a stand in measure for the SLO.
SLO 4: Identify Threats
SLO 4 is assessed in two courses, DADA 206: Intermediate Defense Practicum, and DADA 306: Advanced Defense Practicum.
DADA 206 is offered in the Spring semester and is the second (of 4) courses in the required practical defense series. This course focuses largely on the theory of defensive magic application and so contains an entire section on the ethics of bystander management and the responsibility every witch and wizard has to maintain the International Statute of Wizarding Secrecy. Every student must identify an event in wizarding history where these ethics were involved and present it as a case study to the class. In the presentation, they must address the bystander component specifically. Two faculty members will rate each presentation on its treatment of this issue. The four full-time faculty members in DADA will develop an evaluation rubric for these presentations through consensus agreement. Care will be taken to establish consistency in evaluation across the four faculty members. Inter-rater reliability will be established.
DADA 306 is offered in the Fall semester and is the third (of 4) courses in the required practical defense series. In addition to learning new spells, this course focuses on the application of defensive magic in real-world scenarios. Students run various drills in controlled environments that are meant to mimic possible defensive situations. These simulations pit them against three basic scenarios (with variations week to week): a scenario in which multiple magical attackers are present, and a small number (one or two) of muggles are present (the goal of this exercise is to learn to differentiate between muggle and attacker); a scenario in which one magical attacker is present, and there is a large crowd of muggles (typically this presents as a chase); and a scenario in which a large group of magical attackers is pitted against another large group of attackers (this plays out in two versions – one intended to practice tactical ability, the other intended to give students a sense of battle from “eye-level”).
SLO 4 is more directly assessed in the first two of the three scenarios presented above. The four full-time faculty members in DADA will develop the evaluation rubric for these two scenario types through consensus agreement. Care will be taken to establish consistency in evaluation across the four faculty members. Inter-rater reliability will be established.
Once all students have been evaluated in both 206 and 306, the criterion for achieving SLO 4 will be if 80% or more of the students meet or exceed expectations.
Year 3: SLO5 Assessment
SLO5: Perform Advanced Skill Set
SLO 5 is assessed in DADA 361: O.W.L. Preparatory Course II. This course is offered during the spring semester each year. There are three prerequisites for this course: an A (Acceptable) or higher in DADA 351: O.W.L. Preparatory Course I; an A or higher in DADA 306: Advanced Defense Practicum; and an A or higher in DADA 298: Advanced Defensive Theory. As such, the standards for judging the outcomes are higher, but we still expect a similar proportion of students to meet or exceed those expectations.
In DADA 361, students are required to do several drills of a broad range of defensive spells. At three points during the semester, they are given the opportunity to “test out” of the class. That is, if they are able to perform all of the defensive spells required for the course, they are moved to an advanced training program where they learn more complex spells. The expectation is that the vast majority of the students will “test out” by the third test. If they are unable to do so, we recommend additional training before sitting for their O.W.L.s.
The advanced defensive spell set (i.e. Protego Horribilis, Everte Statum, and Expecto Patronum) are not the only spells expected to be performed during the test, so scores on the test are not a good stand-in for this SLO. However, during the test, instructors take records on individual spell completion so we will be able to use this data to assess the SLO. Once all students have been evaluated, the criterion for achieving SLO 5 will be if 80% or more of the students are able to perform the three identified spells by the end of the semester(i.e. 80% of the students will meet or exceed expectations by the last test).
A centralgoal of program assessment is to gain knowledge about the successes and shortcomings of a program's curriculum and resources in terms of achieving its student learning objectives. When SLOs were not meet successfully, changes in curriculum or pedagogywere discussed as well as which resources might better help us meet these standards. While some of our difficulties meeting our SLOs in the last round of assessment were do to external circumstances (the Second Wizarding War, Dark Professors, the school being a battleground, etc.) some were due to curricular and pedagogical concerns. For example, we now separate the training of defensive theory and defensive practice into different courses as we found that (particularly the younger) students were distracted by the practice and did not absorb as much of the theory. Separating them has increased performance drastically.In addition, in the past year we have hired two new full time faculty members – bringing our departmental total to four – which will allow more sections to be taught of each course, thereby lowering the class size.
COURSE / SLOs1 / 2 / 3 / 4 / 5
DADA 100 / E, D
DADA 106 / D
DADA 401 / PA
DADA 206 / PR
DADA 306 / D
DADA 361 / D
PA - PaperE-ExamD-DemonstrationPR - Presentation
1