ACIE 2010_Tabas
Innovations in Education Awards Program 2010
Provost’s Advisory Council on Instructional Excellence
A Novel Approach toTeaching Clinical Decision Making
Using Virtual Patient Technology
Gary Tabas, MD
Final Report
Gary Tabas, MD, FACP
University of PittsburghSchool of Medicine
Department of Medicine
UPMC Presbyterian Shadyside
308 School of Nursing
5230 Centre Ave
Pittsburgh, PA15232
phone: 412-623-2395
fax: 412-623-6621
beeper: 412-263-9245
Teaching clinical decision making to medical students and pharmacists is one of the most crucial aspects of medical education. It is often taught during activities that include direct patient care such as bedside rounds, but each student has the opportunity to interact with only a limited number of patients. To address these limitations, we developed a web-based program that uses virtual patient simulation software (vpSim) developed by the Laboratory for Educational Technology at the University of Pittsburgh, to teach effective clinical decision making about an important complication of diabetes, diabetic ketoacidosis. We administered the program to about 130students in two schools (Schools of Medicine and Pharmacy), and evaluated the program’s effectiveness. The rationale for the project is that it represents a first step in addressing unmet learning needs.
The teaching program we createdis a novel and innovative tool designed to simulate real-life clinical situations in which the learner acts as a health care professional involved in making diagnostic and therapeutic decisions based on the patient’s history and physical examination and the ordering and interpretation of laboratory tests. The software allows creation of a “branching” system (see Figure)whereby students can immediately see the outcomes of their decisions and can make new choices based on those outcomes. Through this process, students learn and practice clinical decision making skills and receive learner-specific feedback that depends on their choices.
Wealsodevelopedinstruments to score and track students as they engage in the program,constructed a test to assess the students’ decision making ability, developed a post simulationsurvey instrument to assess the students’ perception of the effectiveness of their learning experience associated with the branched simulation technology and generated reports on the number of steps taken, the time spent and scores students achieveas they complete the simulation program.
Results:
The mean path score in the virtual patient program among students was 84 out of a possible score of 100when all optimal paths were chosen. Mean time spent completing the case was 29.6 minutes (range 12 to 59 minute). In the survey, learnersindicated increased confidence in managing DKA and felt that the ability to see consequences of their decisions resulted in more effective in learning than with lecture or small group. Students felt that the program was of high educational value.Fourteen medical students completed the pretest. The mean score was 33% correct. The posttest scores among medical students, pharmacy students and endocrine fellows were 68%, 69% and 79% respectively. Post test scores of all groups were significantly higher than the pretest score (p < 0.001). The mean posttest score for endocrine fellows was significantly higher than post test scores of both the medical students and pharmacy students (p=0.026). Analysis of selected posttest questions revealed that scores more than doubled. The virtual patient path score was 84 for medical students, 89 for pharmacy students, and 92 for endocrine fellows (p< .001 for endocrine fellows versus medical students). The program also tracked VP time-on-task which was 30.2 minutes for medical students, 46 minutes for pharmacy students, and 28 minutes for endocrine fellows (p=0.004 for pharmacy students versus other groups). VP time-on-task correlated with path score (Spearman’s Rho = .21, p=0.013 and path score correlated with post test score (Spearman’s Rho = 0.013, p=0.0002).
Discussion
Using authoring software that facilitates the creation of multiple pathways through a simulated case, it is feasible to construct an interactive web-based virtual patient to teach clinical decision making in a simulated patient with DKA. The case-based single best answer multiple choice quiz was constructed to test DKA-specific clinical decision making and the pre- to post test score improvement among students shows a significant improvement in DKA-specific decision making skills. The fact that post test scores were still well below 100% may relate to the level of the learner but may also indicate that optimal teaching of this complex topic requires multiple teaching methods including supplemental readings before completing the VP. The VP path score was significantly higher among endocrine fellows suggesting that the path score is a valid measure of the higher level of DKA-specific decision making skills that would be expected among learners with more training and experience. Most of the endocrine fellows indicated having managed 5-10 DKA patients or greater than 10 DKA patients whereas only 9 of the 135 other students indicated having managed any DKA patients (0-4 patients for each of the 9 students). Time-on-task in the VP was significantly higher for the pharmacy students likely related to higher stakes placed on the exercise because of its role in the clinical pharmacology course. The significant correlation of time-on-task with path score suggests that more deliberate and thoughtful approach to the exercise led to better decision making (and thus better path score), reflecting real-world practice. Finally, the correlation between path score and post test score suggests that the vignette-based post test was a valid measure of DKA-specific decision making.
Students felt that the virtual patient was effective in teaching management of DKA and preferred that instructional method over other teaching methods. By using the virtual patient simulation, we are able to expose students to patients they may not encounter during clinical rotations. Furthermore, this educational technique incorporates adult learning theory in that students are actively engaged in making decision-making, are able to use visualization by experiencing the case as it unfolds and they are able to practice management skills. Learning was adaptive in that there was individualized instruction depending on which path was taken. Although learners did not review their learning path upon completing the VP, such a review is possible and would allow self-reflection as learners reviewed steps taken. Instructor feedback could be both formative and summative based on review of the path taken. Since only a few of the students indicated having managed a patient with DKA on the survey, the DKA VP simulationrepresents oneway to help fill an educational gap.
Despite this VP’s effectiveness, several limitations should be taken into account. Constructing a branched-narrative VP is time consuming requiring up to 20 hours for authors with expertise in the subject of the VP and up to 40 hours if consultation with an expert is required. The pretest and post test used identical questions and the pretest was administered to only 15 medical students. We made the decision to administer the pre-test to only a subset of students because we wanted students’ time and effort to be concentrated on completing the VP and we tried to avoid cuing. Furthermore, we felt, because of the difficulty of the questions, that pretest scores would be low, and little would be gained by requiring all subjects to complete a pretest. We feel that the pretest scores are representative for the group of medical students because pretest scores were nearly identical for students completing the pretest at either the beginning or the end of the academic year. Because only medical students completed the pretest, we can make no conclusions about improvement in scores among PharmD students and endocrine fellows. Finally, this study was a pilot study using one case in two schools, but within one institution, and there were no patient outcomes.
Conclusion
Using the methods demonstrated in this project, educators are able to develop and study future VP’s that teach decision making in other complex medical illnesses. Future research should examine which features within virtual patients are most useful. We plan to compare a branched-narrative VP to a linear-narrative VP with the same content. Other VP features for study should include effectiveness of free text entry, interactive graphics, video, avatars and games.
1