Carly Hill /
ED TECH 505-4172 /
12/9/2011 /
Contents
Learning Reflection
Executive Summary
Purpose of the Evaluation
Background Information
Description of Evaluation Design
Results
Discussion of Results
Conclusions and Recommendations
Appendix 1: EPD
Appendix 2: Timeline
Appendix 3: GradeCam Survey
Learning Reflection
Before taking this class, I put evaluation and research projects into the same category thinking that the two were one in the same. After completing my own evaluation and this course, I have learned that they are two very different things and that each can be useful to a teacher in the classroom. By completing this evaluation project, I have learned more about a tool I use in my own classroom as well as how I can evaluate other programs that I use in my own classroom which links with AECT Standard 3.3. I also know that I can use this process to help my peers by being able to give them pros and cons of using a program within their own classroom. I feel confident that I could do this process now on my own and deliver the results to the administrators within my building and other colleagues on various programs and this links with Standard 5 of the AECT Standards.
I can also take the principles of the evaluation and apply them to my own teaching. One thing that I know I will immediately implement in my classroom is evaluation process of long-range planning. I already do this to some extent, but learning the evaluation process and the steps have shown me an ordered way of going about analyzing my teaching and how I can make it better. I think that this piece of the evaluation is just like the reflection that many teachers already do, but looking at it as part of the ordered process of a goal-oriented evaluation, I can improve my teaching by looking at my students’ results on the end of course tests and compare that with how I taught the material and look to see where I can make improvements. Completing this process relates to the AECT Standard 1 as I can now analyze more effectively how my assessments compare to the objectives of my course as well as determining if I am using the correct assessment in my class to decide on how I want my students to show me what they have learned.
I also learned about how to create my own survey which I have never done before. I found that SurveyMonkey was such a great and easy to use tool that I can implement in my own classroom to have students do an anonymous mid-course evaluations of my class and of my teaching style. I like this rather than having them complete it in class because they might feel like they cannot say what they really feel if I am in the room with them. This way they can respond honestly to what they think about the class and how they would want to fix it or what they like. I can also use it for other purposes within my government class where students can use it to create surveys on different topics we discuss in class. This new tool that I learned how to use connects to AECT Standard 2.
Executive Summary
This evaluation was conducted on the GradeCam software program and how it is used by teachers at Timberline High School. This particular program was chosen to be evaluated because it can be used easily by many teachers as a way to quickly check student understanding on concepts that appear on quizzes or tests a teacher might give. It is also a less expensive option for the school than the Scantron machine as it does not require the purchase of one big, immobile machine and specialized answer sheets. GradeCam has the capability to be moved easily from room to room and also comes with templates of answers sheets that teachers can easily copy as many times necessary. The software is easily installed on most computers and can be done without much hassle. The evaluator was curious to see how many teachers used this program, what content area was it most utilized in, and if more advanced features were used to get more complex data about the objective assessment they gave to their students. A survey was created by the evaluator using the SurveyMonkey website and a link was distributed via email to all teachers at the high school. In total, 37 teachers responded and only 10 teachers use GradeCam currently in their classrooms with the majority of use being in the Science, Social Studies, and the English departments. Eight teachers reported using the feature to allow students to grade their own work and nine teachers responded that they use the Item Analysis function to get more insight into which concepts their students are struggling with. The survey also asked for teachers personal reasons behind why or why not they allowed their students to grade their own work and how the item analysis function impacted their teaching. Many teachers responded that they did not really allow students to grade their own work using the “Student Window” function, but many did respond that they used the item analysis to directly inform their teaching.
Purpose of the Evaluation
What was the purpose of doing this particular evaluation?
The purpose of completing this evaluation was to see how teachers utilized the GradeCam program in their classrooms. There are fifteen document cameras and licenses that Timberline High School has access to and the evaluator wished to see how many teachers used this instead of the Scantron machine as it is less expensive due to the fact that GradeCam does not require the purchase of specialized answer sheets. GradeCam can also quickly check student understanding of the material and they can use that feedback to inform their teaching
What were the central questions to be answered through the evaluation?
In order to look at how teachers are using GradeCam in their classrooms, the evaluation looked at three specific questions:
- Are teachers using Grade Cam as a formative assessment tool to guide instruction?
- Is Grade Cam used more often by teachers in different content areas?
- Do teachers allow students to grade their own work through the "student window" or do teachers primarily scan the work themselves?
The first question was chosen to look at how teachers used the item analysis function that GradeCam has built into the software and how teachers used that information to guide their instruction. Quizzes or tests can be graded quickly and the item analysis can be used to get instant feedback while students are still in class immediately after they turn in the work or possibly addressed the next day. For the second question, the evaluator wanted to see if different content areas were more likely to use GradeCam over others. For example, teachers in the content areas such as Math or Science might be more likely to give their students objective assessments that can be graded using GradeCam software whereas English teachers might not find it as useful as they are more likely to used subjective assessments. Lastly, the GradeCam has the ability to allow students to grade their own work and teachers were asked if they allowed students to grade their own work or if they primarily did this themselves. This question was asked to see if teachers who used GradeCam were more willing to let students take charge of this or if they continued to want to be the ones scanning the work.
Who would be most impacted by the results of the evaluation?
Results from this evaluation will affect those teachers who already use GradeCam as well as those who do not utilize the program. For those who already use it, they can look at the results of this evaluation (particularly the teachers’ personal responses) and see how individual teachers are using it their classrooms and they might be persuaded to change how they are using it in their classroom. Teachers who do not use GradeCam, but might use Scantron could see the results of this evaluation and see the pros of using it to complete quick, objective grading in their own classroom rather than going down to the copy room to use the only Scantron machine in the high school. They will also be made aware of different functions that GradeCam offers like item analysis or other functions like being able to compare students of the same class that are in different sections and being able to see how students are performing on questions based on state standards.
Background Information
Rationale and Goals
GradeCam is a software program that was purchased by the Boise School District and distributed to teachers at Timberline High School to allow teachers to quickly grade any multiple choice quiz or test in order to provide feedback to students in a timely manner. By using a document camera that is connected to a teacher’s computer with a USB cable, students or teachers can scan the answer sheet and the assignment is graded right on the spot. Data that is collected by the GradeCam software can be used to do an item analysis of questions on the test so that teachers can see what areas students are struggling with so that those areas can be revisited quickly in class. GradeCam comes equipped with a feature where questions given by the teacher on a multiple choice exam can be linked to state standards so teachers can measure how well their students are doing on those items. These results can be used by a teacher to re-teach content that could appear on a state standardized test that students might not be understanding. Teachers in the classroom can also share their results on their tests with other colleagues so that those who teach the same class can aggregate their scores together to see how their students are doing as an entire group. GradeCam also has a feature that allows students to grade their own work one at a time so that they can get instant feedback and it frees up the teacher to answer questions that students might have about what they missed. Once all of the students work is graded, GradeCam can then import results that are collected in the software’s grade book to any computer-based grade book system that a school may use. This allows the results of the assignment to be quickly added into a student’s score so that they can be updated on how they are doing in class. Here is a picture of what is included in the basic kit:
Past Products
Before the purchase of the GradeCam, teachers primarily used the Scantron hardware and answer sheets to perform the same tasks that they could complete with the GradeCam. The Scantron machine at Timberline High School is located in the main copy room and teachers must use the special answer sheets that can only be purchased through the company that produces the hardware. These forms cost $59.95 for a package of 500 forms. A Scantron machine that is in use at the high school costs $1465.00. The machine does many of the same things that GradeCam does, but there is only one in the school and cannot be moved out of the copy room so teachers must bring their tests to the machine for them to be grade. Here is a picture of the Scantron and a copy of an answer sheet being scanned:
Product Development
GradeCam was developed by the Porter family primarily to address the need of cutting down on teacher grading. Tami Porter, who at the time was a middle school teacher, was inspired by what she saw at the grocery store where she witnessed a store clerk scanning items for purchase. With this idea, she went to her teenage son Robert who at the time was working as a software developer and he came up with the image recognition software that would become GradeCam. The small company located in Livermore, California employs Tami, Robert, and Tami’s husband Rick. Their mission is provide teachers and schools with low cost equipment that still performs at a high quality level. The company is continuing to improve their product and since the first GradeCam was released, the company has tried to update their product’s software in a timely and efficient manner. There is a customer support feature on their website that has forums where people post common issues to get help as well as a link to contact customer support with any issues a user might have.
Characteristics of the GradeCam
As mentioned above, the GradeCam can be attached to any computer using a USB cord and is easy to install on any computer. By following simple steps, teachers can quickly put this software on their computer and use it with ease once it is installed. The document camera that comes with the GradeCam is portable and is very light to carry from room to room if teachers have to share it. Rather than having to go down to a central copy room, these cameras can be passed around if necessary and used as long as the software is installed. Another feature of the GradeCam is that schools do not have to own the GradeCam camera that comes with the based kit in the picture above because the GradeCam software is compatible with many document cameras that a school might have already purchased. If a school needs to purchase a camera, the GradeCam software and document camera used at Timberline High School costs $349.20. This includes a user license and one document camera. Another feature of GradeCam is that it comes with answer sheets that are built into the program. Rather than having to pay for Scantron sheets, GradeCam has customizable answer sheets that can be printed off and copied depending on how many the teacher needs for each class.
Description of Evaluation Design
The evaluation of the GradeCam software that I conducted was based upon the goal-based model. This model requires the use of stated objectives or goals that an evaluator is looking for that rely on questions stated before the process begins. Using those questions, objectives are also formed that direct the evaluation. With this model in mind, I established three questions and objectives that I measured through a qualitative survey that I made through the website Survey Monkey. The questions that I looked at in my evaluation were:
- Are teachers using Grade Cam as a formative assessment tool to guide instruction?
- Is Grade Cam used more often by teachers in different content areas?
- Do teachers allow students to grade their own work through the "student window" or do teachers primarily scan the work themselves?
With these questions in mind, I used them to direct the following objectives that I used to craft my survey and what my evaluation would look like:
- Grade Cam has a reports section that allows teachers do item analysis on any formative assessment that they are giving in class. This can provide them instant feedback on what they need to reteach.
- Grade Cam can only be used for summative assessment grading. It can be utilized in many content areas though for a quick check of understanding basic terms.
- Through the student window, students can scan their own assessments and get instant feedback on how they did and what they missed so that they can address their weak areas.
After formulating the questions and objectives, I used these to create a survey that I emailed to the 67 teachers that I work with at the high school. I made sure to include questions that addressed each of the objectives that I wanted to look at. In order to find out if teachers used GradeCam as formative assessment tool, I asked teachers “Do you use the immediate feedback you receive from the Item Analysis function in Grade Cam to inform your teaching?” I also added a follow up question, “If yes, how does it affect the topic that you are teacher?” With the first question, I wanted to see if teachers use the item-analysis feature for instant feedback. As soon as tests/quizzes have been scanned, an item analysis can be done on the assignment instantly so teachers can get that feedback and do with it what they feel best for their students. My main motivation for the follow up question was so that teachers could go more in-depth than just a simple ”yes” answer and so I could see the ways that teachers used those results in their class.
For my second objective, I wanted to see two things; how many teachers use GradeCam and what content areas the program was most used in. First, I asked teachers if they used GradeCam in their classroom and if they did not, I did not have them continue with the survey as they would not know how to answer the questions that followed. Next, I asked teachers what content area they taught in so I could see where the most concentration of the use of GradeCam exists. I was curious see if teachers in one content area or another were more likely to use GradeCam in their classroom.