Christopher Yaple
553
Term Project
The Digital Library User: A Review and Evaluation of Public Library's Digital Collections from the User's Perspective
1. Introduction
Digital libraries (DL) are often evaluated by their designers, librarians, or other information professionals with a focus set on aspects such as the library's system performance and infrastructure, the quality and completeness of its collection, or the economic factors surrounding it, such as the funding and other resources needed to maintain their digital collection. While these evaluations are of importance and a necessity for the ever growing field of DLs, there is another group with their own interests and criteria to consider – the users. DLs are not simply storage units for information or archives that benefit only a select number of information professionals. So why is it that the users, the individuals driving the creation of these DLs, are often disregarded in the research surrounding digital libraries (Bollen & Luce, 2002; Fuhr et al., 2007; Monopoli, Nicholas, Georgiou & Korfiati, 2002; Xie, 2008; Yan, Zha, Zhang & Hou, 2013)? Another trend in DL research, as well as in the research of their physical counterparts, is to give focus and attention to academic and other large organizational collections, while public libraries, institutions that serve and benefit users of all ages, economical backgrounds, cultures, situations, etc., are ignored.
Because of these trends this paper plans to evaluate a number of public libraries' digital collections from a user's perspective, focusing on the usability and user satisfaction associated with each. It is this author's hope, by exploring these criteria and the users' perspective in evaluation and then applying it to actual collections, that future designers, librarians, and other information professional will better understand what users want, expect, and need from their DLs. To do this we will first exploring the criteria, looking at other library and information science literature in hopes to construct suitable definitions and guidelines that can then be used in our evaluation of The Mercer County Public Library, The Hamilton Free Public Library, The Trenton Free Public Library, and The Princeton Public Library.
2. The Criteria: Literature Review & Definitions
2.1 Usability
There is no doubt that the term usability is broad and can conjure an assortment of definitions or ideas. Numerous LIS publications (Chen, Germain, & Rorissa, 2011; Frias-Martinez, Chen, & Liu, 2008; Fry & Rich, 2011; Fuhr et al., 2007; Xie, 2008; Yan et al., 2013) have discussed usability as criteria and a means of evaluating DLs, all of which understand that usability is “essential for providing high quality services to a broad and diverse population of users” (Bertot, Snead, Jaeger & McClure, 2006, p. 18). Chen et al. (2011) states, “a clearly articulated definition of usability is crucial for stakeholders of a system to gain a consistent understanding of its construct” and that “this, in turn, could serve as a building block for establishing good usability policies, standards, and guidelines” (p. 599).
A DL's usability is judged using a number of aspects, one of which is often the first encountered by users – the interface. Nadjla Hariri and Yaghoub Norouzi's (2011) Determining Evaluation Criteria for Digital Libraries' User Interface states that “a DL is only as good as the interface it provides to its users” (p. 699). Fuhr et al. (2007) also express the importance of the interface as a gateway, describing the interface “as a bridge between the user and the system-environment so that the user can interact with the DL system” (p. 27). While the interface is the entryway to and means of use for the DL, it also affects the user before they “step into” the digital realm. Hariri and Norouzi (2011) state that the “interface characteristics...have a strong impact on perceived ease of use and its subsequent effect on the intention to use an online library” (p. 703). If the user interface is such an important component of the DL and a critical aspect of usability as well, what makes a good interface?
In their list of the eleven attributes of usability, authors Chen, Germain, and Rorissa (2011) describe the interface and its design as “the technical and visual design concerns of the system or website interface, including its design elements...consistency, navigation, information architecture, and task flow” (p. 606). Successful interfaces offer attractive design elements, while at the same time providing simple and easy to understand menus, tools, and information. In the case study Tweaks We've Made to the Digital Branch the author explains how their branch's digital library received complaints about having “too much content on the main page” and how “all the content seemed to run together” (2012, p. 8). A successful interface avoids clutter and confusion. Fuhr et al. (2007) also stresses the importance of integration when considering the different components and functionalities of the DL interface. Finally, Xie (2008) reminds us that a successful interface “delivers results in a readable format” (p. 1358). Font, color, and layout choice can either facilitate or hinder usability.
The view and output options are important for a DL's usability. When discussing the view and output of a DL, phrases such as “visually clear” (Fuhr et al., 2007, p. 27) were used to describe users' preferences, along with the desire “for high resolution items” (Xie, 2008, p.1364) Other common options include zoom features or multiple layouts for when looking at multiple documents at once. One major problem found in the view and output of DLs was small fonts used when displaying text (Xie, 2008, p. 1364).
A third aspect brought up in the discussion of usability in DLs is that of control and flexibility, a characteristic that fits into the idea of a DL's design elements as well. A DL needs to provide its users with flexibility and control over the provided tools and options during their use (Bertot et al., 2006; Chen, et al., 2011; Fuhr et al., 2007; Hariri & Norouzi, 2011). Control and flexibility can be seen in a DL's ability to allow users to "manipulate, adapt, customize, personalize, and access, using various devices and means," as well as being "compatible with varying applications" (Chen et al., 2011, p.606). Flexibility is provided through customization tools such the ability to group result, limiting the results through controls such as library or type of document, search in either a basic of advance setting, as well as by subject, author, title, keyword, etc., and refining a search after the initial results have been received (Bertot et al., 2006, p. 21).
Flexibility can also be seen as the way in which a DL adapts to a user's preference and skill (Fuhr et al., 2007). Authors Frias-Martinez, Che, and Liu (2008) explain that because of the wide range of possible users DLs can reach, there is "a greater variability, in terms of their background, knowledge and skills" among them (p. 48) and usually a preference for either field dependence or fiend independence when performing cognitive tasks (p. 49).
The previously mentioned aspects of usability have focused on the physical components, the design and layout of the DL, but there is another side of usability to consider. A DL's learnability and memorability are a crucial component of its usability (Bertot et al., 2006; Chen et al., 2011; Fuhr et al., 2007; Hariri & Norouzi, 2011; Yan et al., 2014). When discussing learnability and memorability Chen et al. (2011) state that “the system should be efficient to use so that once the user has learned the system, a high level of productivity is possible” and “the system should be easy to remember so that the casual user is able to return to the system...without having to learn everything all over again” (p. 606). These definitions are echoed almost verbatim by Hariri & Norouzi (2011) and expanded by Bertot et al. (2006) who state that a system should allow “users to intuitively use a digital library's various features” (p. 19). In their study of digital libraries and virtual communities Yan et al. (2013) discuss system quality, an aspect of DLs, and its usability, which relies on easy to learn software, features, and functions.
Although a DL's learnability and memorability are important to the overall usability of it, they also rely on the inclusion of features that can help users throughout their time and use (Fry & Rich, 2011; Hariri & Norouzi, 2011; Monopoli et al., 2002; Xie, 2008; Xie & Cool, 2006). Simply put, help features “provide instruction on how to use a service” (Monopoli et al., 2002, p.111). According to Xie & Cool (2006) users look to help features when they find themselves in “help-seeking situations,” situations when they are unable to move forward in their search process or use of information retrieval system. Help features can appear in a number of forms, such as frequently asked questions (FAQs) or chat options, and can also be divided into general or specific help (Xie, 2007). A DL's help features are especially important to its overall usability because it has a direct relation to a user's view of the entire system. Xie & Cool's (2006) study shows “that users' evaluation of help mechanisms of an IR system is related to the evaluation of the IR system” as a whole. Much like how a DL's interface can effect a users presumed interest or use, the DL's available (or lack of) help features will drive a user's preference in a positive or negative manner.
Interestingly, a majority of publications that express the importance of help features in DL also share that many users do not use these functions (Fry & Rich, 2011; Monopoli et al., 2002; Xie, 2007; Xie & Cool, 2006). In their study, Monopoli et al. (2002) published that less than a quarter of their respondents used the provide help features, while the users in Fry & Rich's (2011) study “expect[ed] online systems to be easy to use and self-explanatory,” but simultaneously did not use the help tutorials provided (p. 393). This disinterest and disuse of help features may be due to the lack of knowledge about where to start looking for help, the need for directions, help features that are not specific or personal enough, and help that is difficult to understand (Xie & Cool, 2006) or the help features may simply not be helpful to users or provide contradictory information that cannot be implemented in the actual DL (Xie, 2007). Despite the contradictions present in the discussion of help features, the fact that it is so prevalent leads the author to believe that it should be included when discussing usability in DL.
After collecting and examining these previously explored publications this paper has defined usability as follows: Usability focuses on the DL's ability to allow users to achieve their goals. This ability includes a number of components that can be separated into two groups. The first group focuses on the design element of the DL interface, which also includes the control and flexibility of the system for its users, as well as the inclusion of view and output options. The second group of usability criteria includes the system's learnability and memorability, as well as the inclusion of help features. This definition of usability will be used from here on out in this paper.
2.2 User Satisfaction
In practically all systems a satisfied user is one of the preferred outcomes. This is no different for DLs. One challenge DLs face when working to leave their users satisfied is the fact that they are working with a full spectrum of users, each with different skills, levels of knowledge, and other abilities or disabilities. Frias-Martinez et al. (2008) explains that “users of DLs may have a greater variability, in terms of their background, knowledge, and skills” (p. 48). Because of this “DLs must reach out to users from all walks of life, serving information needs at all levels” (Hariri & Norouzi, 2011, p. 708). Another variability to consider among DL users is the possibility of disabilities at all levels. DLs must provide their users with “appropriate functionalities” (Chen et al., 2011, p. 602) no matter the disability, be it “visual, auditory, mobility, cognitive, learning [or] others” (Bertot et al., 2006, p. 24).
Chen et al. (2011) sum this up in their discussion on user satisfaction in which they state that “the system addresses users' cognition, information, processing, mental model, level of knowledge/skill, and demographic characteristics” (p. 606). All of this is important because, as Fuhr et al. (2007) put it, “the final aim of a DL system should be that of enabling people to access human knowledge any time and anywhere, in a friendly multi-modal way” (p. 26).
Another way DLs allow for user satisfaction is through the provision of communication and feed back channels for their users (Tweaks We've Made,2012; Xie, 2008; Xie & Cool, 2006). Xie (2008) states “interaction between users and digital libraries is...an important component for evaluation” (p. 1350) and being that user feed back can drive user satisfaction, “a communication channel is needed between developers of digital libraries and their targeted audience” (p. 1360). In the case study Tweaks We've Made to the Digital Branch the author shares improvements that were made after receiving feedback from users and the subsequent increase in user satisfaction, stating that DL should “listen to complains when they come in” (2012, p. 9). But for comments or complaints to come in, avenues must be laid out in the DL. Communication and feed back channels can consist of standardized forms for users to fill out or listed email addresses, but it is note by Xie & Cool (2006) that “chatting with a librarian is a preferred...feature for them.” Unlike the communication discussed when looking at help features in a DL, the communication and feed back here revolves around users being able to share their feelings about a DL, be they positive or negative.
While it may seem like a given, the final component of user satisfaction in DL is a satisfied user. DLs should strive to have their users leave satisfied and with their original goals met (Fuhr et al., 2007; Hariri & Norouzi, 2011; Xie, 2008). Hariri & Norouzi (2011) stress the importance of users' satisfaction, explaining that “DLs as a whole will thrive or wither only as they serve or fail to serve their user communities” (p. 708). This is echoed by Xie (2008) who simply states “user satisfaction is essential for the success of a digital library” (p. 1367). Satisfaction is a hard aspect to pin down, especially for the user who cannot see the DL's system performance or know what other aspects are going on behind the scenes, but is often associated with “a pleasant and rewarding experience” (Hariri & Norouzi, 2011, p. 705) that is also “efficient and enjoyable” (Fuhr et al., 2007, p. 26.)
Through the process of examining these included publications this paper has defined user satisfaction as follows: User satisfaction evaluates the subject's ability to provide for users of different levels of knowledge, skill, and ability, as well as providing for communication and feedback with its users, and overall leaving its users satisfied with their experience. This definition of usability will be used from here on out in this paper.
3. The User: Literature Review
When evaluating any DL it is important to consider its usability and the user satisfaction it provide, but there is still another aspect that must be explored – the user. DL users view and experience DLs differently than those who design and build them, as well as information professionals such as librarians who also use them. Xie (2008) explains that “while users focus more on the usefulness of digital libraries from their own perspective instead of from researchers' and professionals' perspectives, they care less about cost, treatment, preservation, social impact of digital libraries, and so on” (p. 1371). This difference in views is directly affected by what users can and cannot see or experience. Fuhr et al. (2007) note that a DL's performance, often a standard point of evaluation, “is an aspect of the system that the users cannot see or evaluate directly” (p. 26) and that the same goes for other “practical issues” such as cost, effectiveness, time, personnel, and infrastructure (p. 31).
Despite this, information professionals have not given much attention to the users and their needs in their publications (Bollen & Luce, 2002; Chen et al, 2011; Fuhr et al., 2007; Xie, 2008). Bollen & Luce (2002) state “the study of user needs and preferences has largely eluded quantitative analysis,” which has resulted in a lack of focus on the “actual effectiveness” of DL. Chen et al. (2011) share their findings that library practitioners “seemed less attentive to users' effective concerns” (p. 617) when using DL, and did not consider the users' environment, be it “cultural, social, economic, political, or organizational” (p. 620). Xie (2008) described users as “passive subjects” in most DL research and evaluation (p. 1351). Additionally, Fuhr et al. (2007) explain that “relevance, as measured by information retrieval evaluation cycles, does not take into account user satisfaction or pleasure” (p. 23).
Some authors have attempted to provide reasoning for this though. Fuhr et al. (2007) simultaneously express the user's importance and complexity, stating that “users is the first component of any interaction process and his characteristics are complex and constantly evolving,” which could be a reason for the lack of published research on DL users (p. 25). Bollen & Luce (2002) express a similar sentiment in their article, explaining that “user search focus can shift from one scientific domain to another between, or even within, retrieval sessions” and because of this researchers do not always have “stable characteristics” to focus on. Xie (2008) shares that users' “evaluation of DLs is affected by their preferences, experiences and knowledge structure,” and that “different users have different preferences” (p. 1370). It is not that researches are not interested in the DL users though. Fuhr et al. (2007) show this awareness when they explain that “digital libraries are destined to serve user” and “if unused these systems fall into oblivion and terminate their operations” (p. 31). Xie (2008) agrees, stating that “only the targeted users of a digital library can determine the usefulness of a collection” (p.1359).
Nevertheless, research has provided us with some important aspects of DL users to consider. Hariri and Norouzi (2011) found that “regardless of users' information technology backgrounds, their expectations of DLs functionality are the same,” and that “users' previous experiences with DLs” affect their current DL usage (p. 710). Xie (2008) echoes this, stating that “users' past experience of other IR systems influences their use of digital libraries” (p. 1370). Xie's use of the term IR system and its interchangeability with the term DL touches on another user-focused findings. Early on Xie (2008) explains that “to end users, digital libraries are similar to the world wide web with improvements in performance, organization, functionality, and usability” (p. 1346) and that they “consider digital libraries more as IR systems [and] less as libraries” (p. 1359).