Harrison, MacClennan, and Cunningham
Daily Quizzes: Methods and Results
Revised by: Mark Harrison 2 29 08
Abstract
This paper describes a new methodology for administering daily quizzes. The quizzes are constructed so as to be easy enough that any student making a reasonable effort to prepare for class can get a perfect score. They are then presented to the students as a “reward” for preparing for class. Data from a freshman Principles class show that this methodology produces a high rate of student acceptance and a modest increase in attendance and learning outcomes.
Introduction
Socratic dialog is effective in the classroom, but it requires that students prepare for class by reading in advance. Students who fail to read an assigned case or article cannot participate in the classroom discussion, miss the opportunity for active learning, and presumably suffer reducedlearning outcomes. Yet, as every experienced instructor knows, many undergraduate students fail to read in advance, and, worse, some fail to attend class altogether. Practical methods for motivating student preparation and attendance therefore attract our attention.
Thompson (2002) describes an intriguing approach to the problem of motivating student preparation and attendance. He gave a brief, mandatory quiz over the assigned reading at the beginning of every class meeting, a “daily quiz”. Thompson deliberately uses “easy”questions in the daily quiz, so that any student who has read the assigned material can earn a perfect score, and thus receive a reward. In other words, the daily quiz, although it isa course requirement, serves as an incentive and positive reinforcement, not as an instrument of assessment or a threat.
Though Thompson gives anecdotal evidence of student acceptance of this system, he reports no data on attendance or learning outcomes. Other authors have studied quizzes as incentives to read articles (Carkenord, 1994; Solomon, 1979; Padilla-Walker, 2006) or to attend class (Hovell, Williams, and Semb, 1979; Wilder, Flood, and Stromsnes, 2001). All of these authors, however, study extra credit quizzes rather than mandatory quizzes, and only one, Padilla-Walker (2006), attempts to determine the impact of quizzes upon learning outcomes (using multivariate regression, she found a significant increase in exam scores). In sum, we find no published data on the effects of mandatory daily quizzes, and no published point estimates of the change in learning outcomes that might result from daily quizzes of any kind, including extra credit quizzes.
This paper describes our version of a system of mandatory daily quizzes and reports the impact of the daily quizzes upon student attendance frequency and average student exam scores.
Daily Quizzes at Daniel Webster College
Harrison (2004)describes a system of mandatory daily quizzes for a freshman course in Principles of Business Management. The standard format for the quizzes (see Appendix I) contains exactly four multiple choice questions – the maximum that can be displayed on one projector screen. The instructor presents the quiz at the beginning of each class meeting, students complete standardized answer forms, and, after a few minutes, the answer forms are collected. The instructor reviews the correct answers, and asks the class how many earned full credit (so thatthe students’ show of hands publicly demonstrates that the “reward” was given). This process takes 5 minutes or so. Reviewing the daily quiz brings the class to order, engages the class’s attention, and sets up a nice segue into discussion of the day’s topic.
After the class meeting, the instructor grades the quizzes on a simple scale: zero points for a no-show; one point for “signing your name”; and two points for getting at least three out of four questions correct. Quizzes from a section of thirty students can be graded and recorded in about ten minutes. The answer forms are never returned to the students, as returning them would consume too much class time, but from time to time students are invited to review their scores with the instructor. The quiz scores constitute 20% of the student’s course grade.
Other systems are certainly feasible, but our experience has been that the daily quizzes work best ifthey are simple to grade, consistent in look and feel, and forgiving of minor student errors (hence the system of awarding full credit for three correct answers). Our daily quizzes are delivered with much verbal “topspin”, emphasizing that they are “easy points”, designed as a reward for preparation, etc. Many students are initially uneasy at the prospect of so many quizzes, but after a few weeks of individual and collective success they gain confidence. By the end of the course students report positive attitudes to the daily quizzes.
Several years of experience suggested that the daily quiz system produces better student preparation and attendance, and, presumably, better learning outcomes. We formed two hypotheses:
- Hypothesis 1: Daily quizzes produce an increased frequency of attendance.
- Hypothesis 2: Daily quizzes produce improved learning outcomes, as measured by exam scores.
We conducted an experiment to test these hypotheses.
Experimental Methods
Two sections of a course in Principles of Business Management were used, comprising 75 students in all. The students were “day” students, of traditional age, at a small college in New England. A typicalfreshman here has a high school GPA of 3.0 and an SAT of 1065. Students in the course came from several academic majors, and a substantial number of them were upperclassmen taking the course as an elective. The course was a survey course; a typical week’s reading assignment included a short chapter in a textbook by Ferrell and Hirt, 2007, an article from the Harvard Business Review, and preparation for an activity (e.g., reading a 4-page case from the Harvard Business Review). This is considered a heavy reading load for a freshman course at our school.
The semester was divided into three time segments, as shown below. In the first two time segments, quizzes were scheduled in one section but not the other, which thus served as a control. The instructor and the lesson plan were exactly the same for both sections. An exam was scheduled at the end of each time segment. The exams given the two sections contained the same questions (although presented in different order).
Table 1: Schedule of Quizzes
Time Segment 1(10 meetings) / Time Segment 2
(12 meetings) / Time Segment 3
(13 meetings)
Section A / No quizzes / Quizzes / Quizzes
Section B / Quizzes / No quizzes / Quizzes
This schedule of quizzes and exams measures student performance in each section under both the quiz condition and the no-quiz condition. Thus it controls for variation in student motivation and exam difficulty between the two periods of interest. In the final time segment of the semester, quizzes were scheduled for both sections.
Results
Scores on the daily quizzes and exams were recorded, and, at the end of the course, a confidential survey of student attitudes and behaviors was administered. Students that dropped the course or failed to complete all the exams were eliminated from the data set.
The daily quizzes produced a 4.05 point (6.5%) increase in exam scores, shown in Table 2. The column labeled “Deviation” shows the difference between the exam scores with quizzes and the exam scores with no quizzes. A paired-sample test on the deviation of the exam scores, with n = 66 and a standard deviation of 14.61, reveals that the improvement in exam scores is significant (p < .01).
Table 2: Mean Exam Scores
Exam 1Mean Score / Exam 2
Mean Score / Exam 3
Mean Score / Deviation
(Quiz – No quiz)
Section A
(n = 29) / 60.93
(No quiz) / 66.00
(Quiz) / 63.75
(Quiz) / 5.07
Section B
(n = 37) / 63.24
(Quiz) / 59.97
(No quiz) / 57.97
(Quiz) / 3.27
Aggregate
(n = 66) / 62.23 / 62.62 / 60.51 / 4.05
The daily quizzes also produced increased frequencies of attendance,shown in Table 3. The data, however, show a trend of decreasing attendance over time, much as was reported by Van Blerkom (1992). This complicates the statistical analysis. To de-trend the data, we constructed a statistic called the Attendance Index. The Attention Index for a student is simply the number of meetings he or she attended in a period divided by the overall average number of meetings attended in that period.
Table 3: Mean Frequencies of Attendance
Time Segment 1Attendance / Time Segment 2
Attendance / Time Segment 3
Attendance / Semester
Attendance
Section A
(n = 29) / 91%
(No quizzes) / 89%
(Quizzes) / 86%
(Quizzes) / 88%
Section B
(n = 37) / 96%
(Quizzes) / 86%
(No quizzes) / 85%
(Quizzes) / 88%
Aggregate
(n = 66) / 94% / 87% / 86% / 88%
The Attendance Indexes are shown in Table 4. The quizzes produced a .034 point improvement in the Attendance Index, which can be interpreted as a 3.4% increase in the number of class meetings attended. The column labeled “Deviation” shows the difference between the Attendance Indexes with quizzes and the Attendance Indexes with no quizzes. A paired-sample test on the deviation of the Attendance Index, with n = 66 and a standard deviation of 0.15, reveals that the improvement in the Attendance Index is significant (p < .03).
Table 4: Mean Attendance Indexes
Time Segment 1Attendance Index / Time Segment 2
Attendance Index / Time Segment 3
Attendance Index / Deviation
(Quiz – No quiz)
Section A
(n = 29) / 0.97
(No quizzes) / 1.01
(Quizzes) / 1.01
(Quizzes) / 0.039
Section B
(n = 37) / 1.02
(Quizzes) / 0.99
(No quizzes) / 0.99
(Quizzes) / 0.030
Aggregate
(n = 66) / 1.000 / 1.000 / 1.000 / 0.034
Discussion
We estimate that the daily quizzes produced a 3.4% increase in the frequency of attendance and a 6.5% increase in exam scores. Our data corroborate the instructor’s classroom impression that mandatory daily quizzes, used as an incentive for student preparation and attendance, produce richer discussions and enhanced learning outcomes. Theseresults are consistent with Padilla-Walker (2006) and with our own unpublished data (see Appendix II) from an earlier series of experiments.
Further, both survey data and post-hoc student interviews indicate that the daily quizzes are well-accepted by the students. Given a standard Likert survey, 76% of the students responded “Strongly Agree” or “Agree” to the statement, “The Daily Quizzes, on balance, were worth the time and trouble”. When given the statement “In choosing future classes, the inclusion of Daily Quizzes would make me more likely to sign up for the course”, only 16% disagreed, although it must be stated that the modal responseto this statement (37% of respondents) was “neither agree nor disagree”.
Further research is needed, however, to pinpoint the mechanism by which the daily quizzes produce higher exam scores. To conclude that the daily quiz system improves learning outcomes by 6.5% would require several implicit assumptions about causality: that manipulating the classroom environment (i.e., requiring daily quizzes) produces a change in student behavior (e.g., reading in advance), which produces an increase in learning outcomes (unobserved), which, finally, results in increased exam scores (which measure the learning outcomes). Some of these assumptions may be wrong. The experimental results do not prove that the students actually read more because they were quizzed. Other learning mechanisms may be at work. On the other hand, the assumption that the change in learning outcomes is accurately measured by examinations is open to question. It may be the case that some of the classroom learning,which emphasized applicationof concepts (e.g., case analysis) rather than comprehension, was simply unmeasured by the exams. If this is true, the benefits of the daily quiz system are understated above. These questions regarding mechanisms and causality deserve further investigation.
References
Carkenord, D. (1994). Motivating students to read journal articles.Teaching of Psychology, 21, 162 –164.
Ferrell, O. C., Hirt, D. A., & Ferrell, L. (2007). Business: A Changing World.
Harrison, M. (February 2004). Daily quizzes: What works and what doesn’t. Paper presented to the faculty of Daniel Webster College.
Hovell, M., Williams, R., & Semb, G. (1979). Analysis of undergraduates's attendance at class meetings with and without grade-related contingencies: A contrast effect. Journal of Educational Research, 73, 50 – 53.
Padilla-Walker, L. (2006). The impact of daily extra credit quizzes on exam performance. Teaching of Psychology, 33, 236-239.
Solomon, P. (1979). The two-point system: A method for encouraging students to read assigned material before class. Teaching of Psychology, 6, 77 – 80.
Thompson, B. (2002, June 21). If I quiz them, they will come. Chronicle of Higher Education, 48, pp. B5.
Van Blerkom, M. (1992).Class attendance in undergraduate courses.Journal of Psychology, 126, 487 – 494.
Wilder, D., Flood, W., & Stromsnes, W. (2001). The use of random extra credit quizzes to increase student attendance. Journal of Instructional Psychology, 28, 117 – 120.
Appendix I
A typical daily quiz in Principles of Business Management.
Quiz on Mintzberg
1)Which of the following is true:
a)Mintzberg says, “Most managers are reflective, systematic planners.”
b)Mintzberg says, “Managers have few regular duties because most regular jobs can be delegated to a subordinate.”
c)Mintzberg says, “Managers often use MIS systems to help make accurate, timely decisions.”
d)Mintzberg says, “Managers use judgment and intuition rather than science and logic.”
2)Minztberg would say that a manager’s scarcest resource is
a)Time (calendar time)
b)People
c)Money
d)Time (their own time)
3)Mintzberg says that managerial roles include
a)monitoring
b)approving payroll checks for payment
c)entertaining clients at lavish lunches
4)Mintzberg believes that
a)management is more art than science.
b)management is more luck than skill
c)management is more hype than help
d)the Bosox will beat the Yankees this year
Appendix II
An earlier experimental design.
We report here on an earlier experimental design intended to test the same hypotheses discussed above. The earlier design randomly assigned students within each section to a condition in which the quiz counted towards their course grade or to a condition in which the quiz did not count towards their course grade. This design failed, however, because many students whose quiz did not count towards their course grade nevertheless chose to act as if it did. Some students did not know if their quizzes counted or not, and some students reported that they prepared and studied for the quiz as if it were a “real” one, even when they knew it had no impact upon their grade. This is interesting, because it indicates that daily quizzes perhaps need not rely upon the grade incentive to be effective.
Experimental Methods
Two sections of a course in Principles of Business Management were used, with a total enrollment of 63 students.
The semester was divided into two time segments, with an exam scheduled at the end of each segment. The instructor and the lesson plan were exactly the same for both sections. An exam was scheduled at the end of each time segment.
The test subjects within each section were randomly assigned to two groups, Group 1 or Group 2. For Group 1, the daily quiz scores prior to the first exam did not count towards the course grade; while for Group 2 the quiz scores prior to Exam 1did count. During the time segment after Exam 1 but prior to Exam 2, the counting of the quiz scores for the two groups was reversed: the quiz scores of Group 1 counted while those of Group 2 did not. It is important to note, however, that all the students took all the daily quizzes. The only difference between the two groups lay in the way their quiz scores impacted the students’ course grades.
A double-blind experimental protocol was used: an assistant assigned the students to groups and graded the daily quizzes. The instructor, who,as it happened, also graded the exams, did not know the group assignment of any student. For consistency across sections, the instructor and the lesson plan were exactly the same, and the exams contained the same questions (although presented in different order).
Results
Forty-three subjects agreed to participate and thirty-nine of them completed the experiment.Scores on the daily quizzes and exams were recorded, and, at the end of the course, a confidential survey of student attitudes and behaviors was administered.
The Exam 1 scores are shown in Table 1. Consistent with the hypothesis, mean scores on Exam 1were higher in the “quizzescount” condition by 3.2 points, or 4.5%. This difference, however, is not statistically significant.
Table 1: Scores on Exam 1
n / Mean Score / Standard DeviationGroup 1
(quizzes do not count)) / 19 / 68.9 / 12.7
Group 2
(quizzes count) / 20 / 72.1 / 13.5
The mean frequencies of attendance prior to Exam 1are shown in Table 2. Consistent with the hypothesis, attendance was higher by 4.6 percentage points. Again, however, this difference is not statistically significant.
The lack of statistical significance may be the result of inadequate sample size, but other informationindicateadditional problems with this experimental protocol. First, Group 2 continued to outperform Group 1 on later exams, even when their daily quizzes did not count towards their course grade. This could be an endowment effect that resulted from Group 2’s early incentives to complete daily quizzes. Second, a survey revealed that only 76% of the students correctly understood whether or not their quizzes counted towards their course grade. And finally, oral debriefing at the end of the course revealed that a number of students did not care whether or not their quizzes counted towards their course grade. They viewed the daily quizzes as a pleasant challenge or as an opportunity to exercise their learning. One student stated that he decided to treat all the quizzes as if they counted in order to better prepare himself for the exams.