Medical Applications Involving Virtual Environments
New Jersey Institute Of Technology
CIS 732
Eric Antonelli
December 17, 2001
Table of Contents
- Abstract
- Introduction
Historical perspective1
Current Information Systems2
- Virtual Environments
Media-spaces systems3
Video-conferencing systems4
Collaborative virtual environments5
Avatars5
System examples6
Telepresence systems7
Collaborative augmented environments8
Transportation and artificiality9
Spatial representations9
- Medical Applications and Virtual Environments
Computer Integrated dexterous work10
Medical Simulations11
- Current Surgical Aspects
Image-guided surgery12
Preoperative planning13
Telemedicine13
- Medical Education
Considerations14
Anatomical simulations16
Surgical Simulations17
- Collaborative Medical Environments18
- Conclusion19
- Figures
- References
Abstract
As medicine enters the twenty-first century it is apparent that computer technology will change the delivery of healthcare. In particular, the use of virtual environments represent a future where virtual worlds mix with actual patient data to form an augmented reality system where complicated medical procedures are simplified leading to improved patient care and reduced cost. There are five basic types of virtual reality categories these are media-spaces, spatial video-conferencing, collaborative virtual environments, telepresence systems and collaborative augmented environments. Each possesses unique features and functions that allow for degrees of artificiality, spatiality and user transportation. Medical education has made use of virtual environment simulations to train healthcare professionals using computer generated anatomical models thus allowing for medical procedure training and the improvement of user hand eye coordination. Virtual environments have also made advances possible in preoperative planning, image-guided surgery and in computer mediated communication as a means to enhance collaboration and to change and extend the medical knowledge base.
Introduction
The way in which healthcare is delivered is changing, that is there are forces within medicine and external to the field that are bringing about change. For instance, managed care, a means of limiting the cost of medical treatment, initiated by the insurance industry has pressured the healthcare industry to rethink doctor-patient interactions and medical procedures.
There have been major strides in the treatment disease, however, there are aspects of medicine that have changed little since the time of the Egyptians or Greeks. Since the time of Hippocratus, doctor-patient interaction has changed little. There are still interpersonal interactions allowing the physician to unravel a patient's aliment. On the other hand, technological advances can be seen in all disciplines of medicine. These include the development of antibiotics and vaccinations enabling many scourges of man such as syphilis or smallpox to be controlled or eradicated. One of the major advances to medicine as been the ability to non invasively peer inside the human body through the use of imaging techniques including X rays of dense tissues and magnetic resonance imaging (MRI) or to understand physiological processes using positron emission tomography (PET). Therefore, most advancement to medicine has been technological, and health care professionals as well as society as a whole have become dependent on technology. Simply stated, the ability of technology and computing to further modernize and change healthcare and its delivery is limitless.
At present the bulk of healthcare applications deal with information systems that consist mostly of administrative applications such as strategic decision-support, enterprise wide networks or with clinical support applications. With clinical support applications, the patient's medical record that contains temporal physiological information and treatment modalities is the basis for the system. The incorporation and retention of patient information using information technology such as distributed computer-based patient record databases and clinical support systems enhances the overall quality of care as well as constraining medical costs.
"Most healthcare organizations and integrated delivery systems operate separate clinical services information systems, particularly in areas such as pharmacy, clinical laboratory and radiology" (1). The laboratory information system is the most common type of application system. Broadly speaking, these system are concerned with automation of routine laboratory practices and fall into two categories, those involved in actually automating the testing process itself and those involved in the processing of laboratory data. Another category is the pharmacy information system. "This type of application has been used to check prescriptions, monitor medications administered to patients as well as monitoring drug therapies for possible adverse reactions" (2). One of the more advanced systems concerns the viewing and storage of radiological data. These systems are used by radiologists to view and interpret actual patient data in the form of images. These images have been acquired through radiological procedures such as X ray or magnetic resonance imaging. It is this category of clinical application that is leading the way in the use of virtual environments because it deals extensively with three-dimensional volumes and the rendering of surfaces.
Virtual Environments
The dependence of our society on computer technology has lead to the blending of reality with the silicon world. The use of virtual reality, a computer-generated illusion of three-dimensional space, can be found in many aspects of business and science. The technology itself are varied, what is specific about shared-space technology is that user exploits spatial properties such as movement and containment in order to carry out a particular task or for communication. For instance, virtual reality systems can be used as a mode of simulation, where that simulation represents an aircraft control system, a theater where poetry is read, or a three-dimensional sectioned human body. Most of these systems are networked enabling multiple users to participate, manipulate objects and share experiences within a proxy of reality. "More precisely, virtual environments involve the merging of the numerically modeled space in the computer with the user's experiential three-dimensional space" (3).
There are five basic types of virtual reality categories these include media-spaces, spatial video-conferencing, collaborative virtual environments, telepresence systems and collaborative augmented environments (4). Media-spaces are often used in offices and are used to enhance the existing workspace. This type of system makes use of integrated audio and visual technology. The basis for this category is the placement of various cameras in a room so as to give the users different views of the activities of others. Media-spaces can be best described as social systems where individuals located at distant sites may collaborate on long term projects. The drawbacks to these systems include difficulty in sharing text documents, the manipulation of objects and only modest peripheral awareness.
Spatial video conferences can be considered an advancement to the media-spaces category. This type of conferencing also makes use of integrated audio and visual technology however it improves on peripheral awareness and instead relies on gaze- direction. "Gaze-direction has been identified as a major element of conversation management and helps in understanding the viewpoints of others when engaged in collaborative work" (4). This principle of gaze-direction is what separates this technology from that of media-spaces. Many spatial video conferencing systems have the ability to manipulate documents and make use of shared drawing surfaces. There are limitations to this system as far as the number of individuals that may participate must remain low and participates can not dynamically enter and leave the space.
Up to this point, both systems use mixed audio and visual technology as a proxy for face- to-face communication. The intention of virtual reality is to bring individuals together over great distances and could be best described as a means of reducing time involved in travel. The above systems do not attempt to augment or replace reality with actual or modified representations of the real world. However, collaborative virtual environments (CVE) attempt to do exactly that, the replacement of reality. Collaborative virtual environments make use of networked virtual reality systems to support group work and activities. The main concept behind the use of a virtual environment is the users are replaced with alternate representations, or avatars. The users in the space can change their locations and view perspectives, interact with others in the space as well as manipulate data represented as objects unlike other systems where data and communications are located in separate windows. Additionally, collaborative virtual environments aim to provide an integrated, explicit, and persistent context for cooperation that combines both the participants and their information into a common display space (4, 5). Because of these reasons the category of virtual reality lends itself to certain medical situations, especially surgical applications.
What remains similar in many virtual reality systems is the creation and utilization of avatars. Avatars, derived from the Indian word for one regarded as an incarnation, serve as a physical proxy for the user. The avatars fall into several categories based on appearance. Some of the more common avatar classes include animal representations, cartoon characters and abstract avatars which may have shocking or unusual form. The users of these types of avatars usually wish to remain anonymous another intent is that they may attempt to adopt some or all of the character's qualities. "Therefore, in many cases there is a psychological relationship between the user and their avatar" (6). Some users who are not concerned with anonymity prefer to utilize actual user images and are known as real-face avatars.
In virtual environments people may communicate and setup social spaces through the utilization of avatar proxies. "As with humans, avatars have defined areas of perception the closest being the manipulation range where objects can be moved or inspected, next is the audio perception range and finally the visual perception range" (7). Communication in this three-dimensional environment can take place using text messaging or voice additionally, this communication must take place synchronously (8). Voice is the preferred method of communication because it is the most natural form associated with human behavior (9). When voice communication is used facial animation can be added to the avatar. So the character can visually project emotion with a smile or a frown and thus adding to the realism of the experience.
Two examples of sophisticated collaborative virtual environments include MASSIVE (Model, Architecture and System for Spatial Interaction in Virtual Environment). Other systems include DIVE (Distributed Interactive Virtual Environment) and the large-scale military system known as NPSNET. "MASSIVE allows for the complete implementation of spatial model of interactions and includes network-supported communication of up to twenty users" (10). It also possesses an advanced interaction model that allows users to generate and manipulate objects within the virtual world. The most current version of the system strives for data consistency and the ability to build or structure worlds.
World structuring in as system such as MASSIVE is based on the concept of locales. This is a technique, which allow for the formation of appropriately sized "chucks" of information. Locales provide a means of structuring and composing a virtual world according to spatial characteristics. The locale is the fundamental unit of the virtual world, it could represent a distinct region such as a room or surgical suite (7, 10, 11). The locale may also contain virtual objects as varied as a table, a wall or a surgeon's scalpel. With the locale representation of space there is no global coordinate system instead each locale contains its own independent coordinate system. The complete virtual world and associated objects are linked together using inter-locale links.
Many examples of low cost virtual reality systems can be found on the World Wide Web. Two of most familiar are Activeworld and Onlive Traveler; however, both differ in their metaphors. Activeworld is intended as an actual replacement of civilization with towns, streets and buildings, with the system operating in a modified browser. Whereas, Onlive Traveler is a communication based system that requires a downloadable client (Figure 1).
Still other systems are based on open technologies such as virtual reality modeling language (VRML) and open GL. This is the case with the DeepMatrix system, which is a web based three-dimensional multi-user system. The significance of this system is that it focuses on e-commerce and it is the first system that utilizes Java on all levels to achieve full platform independence on client as well as the server side. Therefore, as a system, it is highly applicable in areas that rely on high user accessibility as is required in B2C e-commerce.
Another category of virtual environments is telepresence systems. These systems allow users to experience and manipulate objects in remote physical space through computer and communication technology (4, 11). Telepresence applications typically involve the creation of a physical proxy of the remote person in the form of a robot which has cameras attached to it and which can move through the physical environment (4). This technology also provides the same immersion as found in the collaborative virtual environment.
The final category of virtual environments is what is termed the collaborative augmented environment. This type of system attempts to mix virtual environments and reality therefore they are called shared-space systems. "The advantage of this virtual environment is the ability to overlay graphical objects onto a real world scene with some degree of dynamic registration between the two" (4). As an example, in shared-space systems, multiple users may manipulate a virtual object from across a physical space (4). An alternate approach to the use of mixed spaces includes the ability to augment real objects with specific digital information to enhance the user interaction with the physical object. Peripheral information such as sound and light further add to the enhancement. The long-term goal of collaborative augmented environments is to provide the natural integration of digital and physical information.
When dealing with systems utilizing telepresence, collaborative virtual environments or shared-spaced environments all must include dimensions of artificiality transportation, and spatiality. All the above must work together at various levels to provide some degree of user immersion. Transportation is the dimension that deals with the extent to which a group of users and objects leave behind their local space and enter a new remote or virtual space in order to communicate with others or to perform set of tasks. The degree of transportation from the real world can be little to as much as totally immersion as is the case with a CAVE, a room system whose surfaces project multiple synchronized images to completely simulate abstract reality. At an intermediate levels of transportation participants find split levels of involvement, where users attend to aspects of both their immediate physical environment and that of virtual reality (4).
Artificiality is the dimension concerned with the degree to which a virtual space is either synthetic or based on the physical world. With synthetic artificiality, deals with spaces that are totally independent or devoid of human external reality and synthetic information may include electronic synthesized sounds and three-dimensional geometric representations. Actual physical information may be represented as an image of a face or body as well as electronic documents.
The last dimension that concerns shared-space virtual environments is the concept of spatiality. The fundamental attributes of this dimension include containment, topology, distance, orientation and movement (4, 10). Of the above attributes movement of participants is considered necessary and lends itself to the development of distance and orientation. Movement also allows for the exploration of digital spaces and also plays a role in dynamic group formation (5).
Medical Applications and Virtual Environments
Medicine is in a state of transition whereby virtual environments and scientific visualization has furthered patient care and medical education. Virtual reality is being used to enhance medicine in four main areas: education and training, medical disaster planning and casualty care, virtual prototyping and rehabilitation and psychiatric therapy (12). The use of virtual environments is being applied to a wide range of medical disciplines, including remote and local surgery, surgical planning as well as treatment of phobias and other causes of psychological distress. It is also used for the visualization of data intensive medical record set.
Today, surgery remains mostly a visual and manual discipline. What this amount to is that many aspects of medicine require dexterous work, or the ability match hand-eye coordination with the specific task at hand. Therefore, the use of virtual environments, especial those involved in actual surgical procedures or training in virtual simulators must mimic physical interactions with instruments, eye coordination and medical planning.
The concept that medicine is a dexterous task is central to the development of virtual medical environments. These systems must allow for delicate work of actual surgical representations that can be performed for hours on end without strain. Such systems already exist for brain and heart surgery, certain minimally invasive techniques and craniofacial repair. "Therefore, the approach of medical simulations is to bring the computer-modeled work object such as a three-dimensional medical image, a scalpel, cutting tool or laser into the user's natural work volume" (13). Additionally, as a surgeon performs dexterous work he is usually within one foot or so of his work, this affords good depth perception and reduces arm strain. So when one attempts to develop system to replace reality the above concerns must be taken into account.
The key to dexterity is hand eye coordination. "In the abstract, a mouse cursor seems far better than a finger, pointing more precisely at a point in the monitor screen" (13). However, in three-dimensional systems stereo vision as well as the position of the user's body is of importance. Therefore, it is both the physical state of the user's body, the position of the head and the arms as well as vision that must come together in the accomplish a given task.
When merging the user's workspace with that of the computers it is of importance that the user must be able to perceive an objects relative location and to have some sort of tactile sense for the object. The sense of touching can be accomplished through the use of a generalized tool handle this is what the user actually manipulates. The virtual tool becomes the "working end" and it may portray various surgical instruments such as a scalpel blade or a laser. "It is generally understood that if the user can see the tool and feel the tool then the perception matches" (13).