SECTION ONE
A. Project Goals:
The innovative use of new devices like Tablet PC requires special software such as Adaptive Book that takes advantage of digital ink and other integrated collaborative design and development tools. The purpose of our pilot project was to study the effectiveness of these tools in an introductory programming course. Adaptive book(AB) [ provides the most flexible way of supporting this activity on Tablet PC. Students were able to generate, share, critique and archive all interactions on digital content. We believed that this level of student collaboration significantly increased the learning effectiveness. Our goal was to demonstrate the use of ink-based annotation tools in Tablet PC, and innovative new content management software like Adaptive Book will increase classroom interaction, ability to markup and share all content, and opportunity for greater collaboration and archival of the best of all interactions. We also hoped that this grant would help us assess the cognitive impact of new hardware and software in learning. We hoped that the tablet PC project would give us the chance to establish what is educationally important about Tablet PC’s - how they can be used most effectively in education.
B. Project Outcomes:
Do students interact more and therefore learn more, if they have access to all of the content including the digital textbooks in a tablet PC? Would a collaborative learning environment, where professors and students can annotate, highlight and link content (digital books, notes etc.) and share those markups with others, significantly increase the learning effectiveness? If all markups created by students and professors were labeled and archived in a searchable repository for common access, would student find them useful? We measured the impact of tablet PC and Adaptive Book(AB) using frequent online surveys, markup data collection and analysis. One of the major goals of the pilot project was to get students to interact with the digital book in a closer way so we can collect and analyze their interactions. The markup tools (highlighters, sticky notes, file and web links) in AB allowed students to markup important content from the book that was used to complete the assignment. All of the survey data indicated that students were more likely to markup the digital book, because of the tablet PC. As expected we saw students taking more electronic notes as well as carrying the PC with them to every class they took. We believed that the pilot project provided valuable data on the best practices of the tablet PC use. One of the goals of the original project was to get students to use the tablet PC and electronic book more by creating beneficial activities that would allow students to create and/or find appropriate markups for their work. For the most part we succeeded in doing this. We were able to collect and analyze markups and draw conclusions on learning as a result. However, we believe that the existence of an online markup repository will significantly improve the student interactions and collaborations.
C. Student Learning:
We believe that the student learning was positively impacted by this project in two ways. First, the form factor of Tablet PC and accessibility to all content (including textbook) in digital form made a tremendous difference in their user experience. Based on online surveys [ students clearly liked the idea of having the digital book available on Tablet PC. Over 80% of the students indicated that they are more likely to read and interact with a digital book, because of Tablet PC. Another impact of the pilot project is that we were able to get students to markup the text and share those markups with others. More specifically, given a specific assignment, we asked students to markup sections from the book that were helpful in completing the assignments. We then compared student markups with instructor markups(which we call reference markups), using a simple key word location criteria to understand “closeness” of student markups to the reference markup. The results then were compared to assignment grades to understand if there is a correlation between “close” markups and good grades. We noticed that in general 60% of the markups were significantly close to the reference markup and among the close markups, over 80% of the students had earned higher grade for the assignments. Therefore the preliminary results are very encouraging. However, we believe that more data is needed to validate the hypothesis that better markups lead to better learning experiences or grades. Overall, students liked the tablet PC experience, its form factor and the ability to annotate notes etc.
D. Key Findings:
The target student group for the project was a group of introductory programming students. In the fall semester, we provided 20 students with tablet PC’s and in Spring semester another 20 students with the tablet PC’s. In the spring semester same instructor (Don Slater) taught the same course to two groups (one using tablet PCs and others in a traditional classroom). In spring 2005, we provided the same standardized Advanced Placement Test in Computer Science to two groups (one with tablets and one w/o) towards the end of the semester. Interestingly, Tablet PC group average for the test was 7.2 (out of 10) with a standard deviation of 2.4. The traditional group had a 6.8 average and a standard deviation of 3.4. We hypothesize that the smaller standard deviation in the tablet PC group is due to the fact that pilot students did lot more collaborative work as a result of having the same computing environment and the access to online textbook. In addition, our markup analysis tools displayed some significant correlation between student performance and closeness of markups. All data and graphs from the project are available at [
E. Project Challenges:
We encountered some hardware, software and behavioral challenges during the pilot project. More specifically, hardware problems were related to smaller keyboards and screen real estate, shorter battery life, misbehaving pen, and bad display units etc. Smaller screens were particularly challenging to students taking programming courses, as they needed to keep multiple windows open. In general students found programming on tablet PC bit uncomfortable. Also if a student owns their own laptop, it was significantly difficult for them to make the tablet PC their primary machine. We constantly encouraged them to do all their work on the tablet PC as they can easily maintain copies of their notes. We also worked hard to make students realize the value of marking up text. In the beginning, students considered marking up text (after completing the assignment) as unnecessary. However, we created a system, where students can refer to other student’s markups, only if they have contributed to the markup pool themselves. As students realize the value of markups, they began to create their own markups and compare with others using our automated comparison tool. In some rare cases, we encountered students who thought tablet PC was the important part of the course and therefore lost the actual objective of the course. We had to work hard to emphasize that tablet PC activities were designed to enhance their learning experience and table PC is a tool to increase course interaction.
F. Number of Students:
Overall, we had around 100 students who were exposed to the tablet PC and Adaptive Book. We tested over 40 students who had the complete access to tablets PC’s during the year. This is small subset of the total of 400+ students taking this course every semester.
G. Faculty and Courses:
Jim Roberts, teaching Professor of Computer Science at CMU, was the first to utilize the tablet PC in his freshman computer science course. He provided students with annotated slides, and reference markups. Students generated windows journal based worksheets and samples are available at [ Jim continued to use Tablet PC in classroom instructions. In spring 2005, Don Slater, Lecturer in Computer Science used Tablet PC’s with his introductory programming course. He also taught another section of the same class using traditional teaching methods. Many other courses have already utilized or are under consideration to utilize the techniques from this pilot. Among the courses are, Advanced Programming techniques, Introduction to Human Computer Interaction for High School Students, Theory of Computing, and a computational geometry course.
H. Project Visibility:
There were number of presentations given on campus based on this project. PI was the invited speaker at the Computer Science Pedagogy Colloquium at CMU in Fall 04. In November 2004, PI conducted a tablet PC workshop (utilizing the HP tablets) at the Three Rivers Technology Symposium ( where HP tablet and Adaptive Book technology was introduced to over 40 school administrators from Pittsburgh area. In November 2004, PI conducted a workshop on Adaptive Book on tablet PC at ICTCM ( In January 2005, PI presented tablet PC project results at the annual Mathematical Society of America meeting in Atlanta. In March 2005, PI presented the Tablet PC/ Adaptive Book project at IEEE/ITCC meeting in Las Vagas and was published in the proceedings. In May 2005, Pittsburgh Tribune Review, a major newspaper in Pittsburgh published a significant article related to the project.
SECTION TWO
- Technology vision
a. Carnegie Mellon has long been a leader in mobile computing. It had what were probably the first 802.11 network coverage in all buildings on campus and the first large-scale all campus coverage with 802.11b. The latter was achieved in the year 2000. Since then, the wireless network has, for many, become the primary network on campus. Many experiments have been conducted with our wireless infrastructure since it was first deployed. Carnegie Mellon has a long history of using wireless computing for educational enhancement. With the inception of Wireless Andrew, the university began exploring; in particular the educational value of unwired computing using both laptops and handheld devices. Hewlett Packard has played a substantial role in supporting many of these experiments with equipment and funding.
[ Prior to Carnegie Mellon’s WLAN, a faculty member would often struggle to use a laptop for presentation in class. The combination of the WLAN and equipping every classroom with digital projection has dramatically changed this picture. Getting the technology out of the way is one of our fundamental design principles in technology-enhanced education. Today, Carnegie Mellon is taking educational enhancement through wireless computing to a whole new level, primarily in the areas of online learning tools, the textbook of the future [ Open Learning Initiative (OLI) [ and Cognitive Tutor Project [ Therefore our institutional commitment to wireless/handheld/tablet computing is very strong. We know mobile devices are both the immediate and long-range future of computing.
b. CMU is clearly moving towards a complete mobile technology model. Our next substantial steps are a) evaluate the potential uses and impacts of tablet computing, b) upgrade the WLAN to higher bandwidth - 802.11a/g, and c) identify more research projects for the test beds we create. We believe we are at scale 4.
2. Synergy
The main purpose of the project is to evaluate the impact of tablet PC in education. We understand, in order to make an impact with this technology, we need to create software infrastructures and educational methodologies that take advantage of mobility and sketching capabilities of the tablet. We believe the mobility of the Tablet PC is well established. However, the justification to use a tablet PC, instead of laptop has not been established. Our survey data showed that students are more likely to use an online book and create interactive markups, if they have access to a tablet PC. In addition to highlighting text and adding links, students can also add ink sketches to better describe their markups. We have created a flexible interface [ and the technology infrastructure to facilitate the project. The Tablet PC/Adaptive Book project is already a highly visible project on campus and we believe as a result of more students using the tablet PC’s will drive institutional technology vision towards more mobile technology.
Academic and Institutional Leadership
3. Academic leadership
Many universities have adopted our teaching methods. Prof. Malu Roldan from San Jose State University’s (SJSU) School of Business has been using Adaptive Book in Tablet PC for the last two semesters[ Prof. Victor Adamchik from CMU’s School of Computer Science has used the Adaptive Book technology for the past 3 semesters for his Data Structures course. Professor Jorge Sarmiento from County College of Morris has used the Adaptive Book for Linear Algebra in Fall 03 and has created a complete library of sharable markups using the Adaptive Book tool set. Adaptive book is also available from Brooks Cole Thomson Publishers [
Adaptive Book and HP tablets are also being used by Ellis School ( for 8th grade Geometry Class. Mt Lebanon high school is planning to use Adaptive Book and HP tablets in Fall 05. The PI initiated all projects. PI is also an invited focus group member at the Microsoft Faculty Research Summit, and has given invited tablet PC presentations at Grove City College and Merrimack College.
4. Instructional leadership
Instructional leadership is demonstrated by significant course changes during the pilot. Students were provided with instructor reference markups to prepare for lectures and tests. All annotated classroom lecture slides were saved for student access. As a result of this pilot students were given the opportunity to create and share markups with others. Clearly, increased interaction has resulted in positive attitude towards the course and some measurable performance differences. Based on online surveys we have conducted [ students clearly enjoyed the access to tablet PC and interactive markups. Tablet PC student group has showed an increased average in standardized testing and lower standard deviations. We felt that students were closely interacting as a result of tablet PC and markup activities.
Project Details
5. Extension project executive summary:
Research shows that marking up content is critical to student learning. Labeled markup management system unique to Adaptive Book provides a flexible way to archive, share and collaborate on personalized markups. Tablet PC with mobile computing capability provide the ability to connect users anywhere, anytime to any resource. In the proposed extension project, we plan to deploy a better Adaptive Book user interface redesigned based on user feedback and a complete online markup repository system for managing private, public and group markups. The markups, consisting of highlighted sections, sticky notes and pen annotations will be labeled and archived in a searchable repository. Students will be able to search and find relevant markups for their learning needs. All markups will be assigned a dynamic ranking that allows the adaptive book system to display markups based on their relevance. Markups will be analyzed and compared to reference markup to understand learning differences.
6. Teaching and learning issues
Learning in the classroom, such as from lectures, places significant information processing demands on students. Previous studies have shown that students’ notes taken during class are rarely generative, that is, they lack organization or synthesis and are usually incomplete, and verbatim. Furthermore, studies have shown that when note-taking structures that facilitate the recording of more generative and complete notes, subsequent performance improves (Kiewra et al, 1991). The use of advanced technologies, such as the Tablet PC and Adaptive Book, may provide a mechanism to reduce the cognitive load imposed by listening, organizing, synthesizing and recording information, and facilitate the recording of more generative and valuable notes. To investigate how these new technologies could facilitate student performance, a series of studies will be designed to compare student performance under various user scenarios of the technology. Despite the availability of all educational content in digital form, there is no easy way to capture, label, share and analyze student interactions. Existing systems (such as course management systems) can only track the time spent on a particular page or category. Although this data is somewhat useful, we do not have access to actual student interactions (exactly what text is marked up etc.) and therefore we are unable to specifically predict if a student has mastered a specific topic or not. This project is an attempt to provide technology and an alternative educational pedagogy that allows students to markup content they find useful in studying or completing assignments. These markups then can be compared with a “reference markup” created by the instructor (or subject expert) and we can recognize the learning gaps. Mobility and inking capability of the tablet PC becomes critical to positive user experiences necessary to collect good data.
7. Goals, objectives and outcomes
The major goal of the project is to create an environment where students can create, archive and share markups generated from course content, and where instructors can analyze and predict learning gaps related to specific topics. Our preliminary analysis of the pilot study data showed that, when students were asked to create a markup to explain how they came up with the answer to a particular question, students’ markups were rarely generative. However, when asked to create a group markup, students were lot more organized, engaging and productive. Furthermore, when asked to analyze a markup created by a colleague, students were lot more critical and creative in improving the markup. We have observed that students can greatly benefit from a collaborative markup-sharing environment and our goal in the extended pilot is to create this flexible environment. We have already seen significant learning differences between those who had the tablet PC and those who did not. For example, we gave a standardized AP test to students in Tablet PC and traditional groups. Tablet PC group average for the test was 7.2 (out of 10) with a standard deviation of 2.4. The traditional group had a 6.8 average and a standard deviation of 3.4. We hypothesize that the smaller standard deviation in the tablet PC group is due to the fact that pilot students did lot more collaborative work as a result of having the same computing environment and the access to online textbook, assignments where they collaboratively created supporting markups and access to markups describing variety of topics created by instructor and other students. We would like to measure the educational impact with a larger group of students with access to tablet PC’s.