Dr. Olson’s Comments on Students’ Responses to Forum 9
Forum 9: Test PrepAfter reading Mehrens & Kaminski, Popham, and the two articles on the Internet (Study and Test Prep Strategies, and No Pain, High Gain), consider the test preparation strategies you have seen employed in the schools. What is your reaction? Do you think the procedures used in the schools are ethical, or educationally defensible? As a school leader, what would you suggest your teachers do to prepare students for the EoG or EoC tests? In framing your response, refer to material in all four articles.
Keeley, you wrote, “I know the state’s DPI website has released sample test items that my teachers utilize to help students practice the format and test-taking strategies in the weeks leading up to the EoGs/EoCs. Although I do understand the argument of possible outsiders’ thoughts of using such releases could be unethical, teachers are not stealing or utilizing the actual assessment used for the final test.”Actually, I think Popham would say that the use of those items in the way you describe is not exactly unethical; rather, not educationally defensible. My own take is that it is ok to use the items to a limited extent. Popham’s argument is that education should be aimed at developing knowledge and skills beyond what is needed to answer multiple-choice test items. Surely we are not educating our students to perform well on Jeopardy, or to be “smarter than a 5th grader.The test prep strategies you describe in your second paragraph are the more appropriate strategies…and, they do not take all that much time. Also, I agree with you that using formative assessment, in which test-taking strategies can be taught at the same time, is a good way to go.
Pam,thanks for the Gipps (1994) quote, “It is not that teachers want to narrow their teaching, nor to limit unduly students’ educational experience, but if the test scores have significant effects on people’s lives, then teachers see it as part of their professional duty to make sure that their pupils have the best possible chance they can to pass the test.” As I hope you, and everyone else is aware, this course is aimed at the principles and ideas embodied in Popham’s and Mehrens and Kaminski’s test prep approach. In the Craft an Assessment Exercise, you first had to identify well-articulated learning target, then you had to construct tests to assess students’ performance relative to those targets. The next obvious step is to teach for mastery of those targets. Your child’s teacher apparently has a good grasp on how to do Test Prep. AndPam, in your reply to yourself, you support and extend what your child’s teacher provided. Good list. I hope everyone else reads it. Also, Amy, in your reply to Pam, I like your suggestion about taking the Test Prep strategies slowly. However, I think one per week would suffice.
Tracee, I agree that “getting copies of previous EOC’s and letting students know questions which are addressed on the EOG or EOC after you have seen the initial evaluation,” definitely is on the boarder of being unethical; it is clearly not educationally defensible, for two reasons. First by focusing on what was on previous tests, instruction can easily miss important knowledge and skills that might appear in items on a new (or alternate form of the) test. Second, if students performances improve, is it valid to infer that they have gained the knowledge and skills in the SCOS?That teachers acquire knowledge of what appears on the state tests. either through exposure to the tests or by conversing with other teachers who have seen the tests, is well known. It is one of the factors contributing to improvements in test scores over years. As Amy implies, in her replay, when they have specific information about what will be tested, teachers cannot help but prepare their students for it. To me this seems to be human nature. The problem lies with test security.
Olivia, I would hope that all teachers in NC have read the Testing Code of Ethics. Failure to do so could lead to infractions that are taken quite seriously.You provided some excellent examples of how certain test-prep strategies might lead to increased test scores without concomitant increases in content mastery. In agreement with Stefanie’s reply to your post, and Elizabeth’s post following yours, I would say that, If teachers can teach students, in the few days before the tests, to perform better on the tests, why can’t they do this throughout the year? In regards to your comments about released items and test format, see my earlier comments to Tracee’spost, earlier.
Elizabeth, You are correct, when the instructional intention is to “teach English, … students [should] be reading literature and not just excerpts from a test prep book.” This, obviously is where the importance of clearly articulated learning target come into play. However, teaching vocabulary is an “enabling,” but not a critical, learning target. Vocabulary acquisition is probably better assesses in the process of Assessment for Learning. Also, I agree with you that when your students, “learn to write and formulate their own arguments” you are not violating ethical standards. However, you could generate some original prompts. As for those teachers who copy tests, that, it seems to me, is clearly an infraction of the Testing Code of Ethics. As you stated, “it’s important that the leadership of the school keep the teachers from going to unethical testing practices.”
In replying to your post,Elizabeth,Kendra wondered about your school’s use of previous tests to help students understand what students have learned. She wondered if this was unethical. It my opinion, it depends. If the tests that are held arehigh-stakes tests that will be used again, then, yes, it would be considered unethical. On the other hand, if those tests are locally-constructed tests, and not used for grading students, but rather for identifying students’’ strengths and weaknesses, than I would say OK. That would be a formative use of the tests.
I can agree with you,Misty, about the use of ClassScape. It can be very useful in formative assessment. I believe, as you do, that the “drill and practice” should not be over done.I believe, also, that students should be exposed to test items in formats other than multiple choice.
Brian, You are not alone in wanting high test scores for your students. But how you get them there is important. Surely you want to be able say that your students performed well on the tests because they did a good job at mastering the content in the SCOS. As you said yourself, when educationally indefensible test-prep practices are employed, “… data reflected by their scores may not be an accurate account of the content domain tested.” It seems to me that teachers, if they really think about their mission, would try very hard to be consistent with Popham’s defensibility criteria. After all, isn’t the primary purpose of teaching to imbue our students with credible knowledge and skills? And, isn’t it the administrator’s job to see to it that teachers are doing this?And, Amy, Olivia, Stephanie, Jessica, and Kendra, thank you for your great (and honest) replies to Brian’s post.
Stephanie, Your post was wonderful. I hope everyone read it. You had so many good things to say that I cannot possibly recount them here. You give some really good examples of the insidiousness of questionable, if not indefensible, test-prep practices find their way into the schools. But, Amy, in your reply, while mnemonics are helpful for memorizing facts, just remember, that your is to teach more than fact…it is to teach students to think and be able to use facts for higher-level accomplishments.
In reply to several of the replies to your post,Stephanie,I have two comments to make. First, teaching the test-prep strategies suggested earlier by Pam and some of the others really do not need to consume all that much classroom time. Second, the rest of the classroom time is better spent teaching the content of the SCOS. Again, as school leaders, one of your responsibilities is to see that teachers spend most of their time teaching.Kevin, in his postfollowing yours, apparently agrees.
I am somewhat appalled,Kendra,that teachers at your schools“are currently preparing booklets [from previous year testing materials…and…from websites that provide examples of test taking items] for students to prepare them for the end of the year testing.” Surely the teachers’ time could be better spent helping students master their learning targets. Furthermore, your schools’ use of boot camps to, “for four weeks instruction [only] in the areas that are being tested,” is, in my opinion, and with which you appear to agree, also reprehensible.
Amy, a long post, but a good one!You’ve given your classmates a lot to ponder. I liked your suggestions such as the one for giving students a classroom test, “let’s stop a moment and see what we have learned and understand in our simple addition unit of study.”Also, maybe the two teachers whose students performed better on the tests were spending more time covering the SCOS learning targets. You are on target, though, in regards to easing students’ anxiety. The literature, I believe, gives many suggestions for doing so. Just Google “test anxiety.”
Also, Amy, with respect to schools’ and teachers’ use of benchmark tests, we’ve discussed them before, in Forum 2.Locally-constructed benchmark tests often possess the same degree reliability and validity as high-stakes state tests or commercially-available tests. This could be problematical. For this and other reasons, it is also questionable whether benchmark tests should be used for summative proposes such as giving report-card grades. In my opinion, benchmark tests are best used for formative assessment.
Katherine, Another long post with a lot of good information and material.Thanks for pointing out that letter grades often do not provide enough information to be useful. Perhaps when we discuss student-led conferencing that will help. Your paragraph, where you discuss Whitehead, is worth a read by everyone.I especiall liked your final statement, “Using assessment methods throughout the school year that focuses on a small number to learning targets, fosters effective communication regarding students’ learning needs, and improves teachers’ instructions will serve as a successful framework for EOG or EOC tests.”
Nikkie, I’d like to comment on your statement, “I know that the article says these methods are not defensible and unethical. However, I disagree. I feel students need practice with how test questions are stated to allow them to be comfortable with the format. The format of the question does not need to keep students from being able to show their mastery of the content.” The authors of the articles assigned as readings are quite clear in pointing out that “learning” the content goes well beyond being able to answer, correctly, multiple-choice questions. In earlier readings, it was pointed out that such questions are limited in terms of what they are capable of assessing. Regardless of what you might think, your job as a teacher is not to teach kids to take (or even do well) on an end of course or end or year test; it is to TEACH students the content in the SCOS. If using those strategies that you seem to agree with, but others call indefensible or unethical, results in higher test scores, than what do you infer when students do achieve higher test scores? Will you take credit for having taught them the content?
Regina, I think that what you suggest (or what you would do) in your post is great. Your post is well worth reading by everyone. It provides good guidance for school administrators. Your Abbot and Costello reference was great.
Finally, thank you Jessica, for sharing your experiences at the Asheville conferences, and for the links. Everyone in this course need to read your post and visit the links. To borrow a line from Dillon, “The times, they are a changing.” It is going to be interesting to see how all this unfolds.