20
SERVQUAL: A Client-based Approach to Developing Performance Indicators
Colleen Cook, Vicki Coleman, and Fred Heath
3rd Northumbria International Conference on Performance Measurement in Libraries and Information Services, 27-31 August 1999
Abstract
The Sterling C. Evans Libraries at Texas A&M University administered the SERVQUAL survey to university users in 1995, 1997 and 1999. SERVQUAL is a gap model for assessing service quality. Reliability, or internal consistency, of scores for all three years was evaluated by computing Cronbach’s alpha coefficients, and construct validity was evaluated with a factor analysis. Specific issues of strategic interest for local library administrators are considered at the individual question level. Additionally, a specific analytical model, Six Sigma, is evaluated for its applicability for quantifying the gap. Future applications for the use of SERVQUAL in identifying best practices among research libraries are explored.
SERVQUAL: A Client-based Approach to Developing Performance Indicators
Colleen Cook, Vicki Coleman, and Fred Heath
3rd Northumbria International Conference on Performance Measurement in Libraries and Information Services, 27-31 August 1999
Background
As the Academy of Marketing Science celebrated the 10th anniversary of SERVQUAL in 1998, Terry Grapentine, the symposium’s organizer, opened the meeting quoting Shakespeare’s Julius Caesar: “There is a tide in the affairs of men, which, taken at the flood, leads onto fortune." He explained:
In the early 1980s, concerns about customer satisfaction and
product quality became emerging tides in the affairs of industry
and academia. And in the early 1980s, three academicians boarded
their service quality boat and rode the tide. And what a ride they had.
Their work not only spawned numerous articles, books, conference
presentations, and consulting engagements, but also significantly
affected how many organizations went about measuring service quality.
(Grapentine, p. 4)
In their landmark papers, "A Conceptual Model of Service Quality and Its Implications for Future Research," (1985), and “SERVQUAL: A Multiple-item Scale for Measuring Customer Perceptions of Service Quality” (1988), A. Parasuraman, Leonard Berry, and Valarie Zeithaml proposed a Gap Model for assessing service quality. According to the authors, the key to optimizing service quality is to maximize the difference between two customer-derived measures, perceptions and expectations. The associated Gap equation, (Q= P – E), became an e=mc2 analog for service marketing enthusiasts.
As service providers, the Gap Model was immediately and intuitively appealing to research librarians in North America. Librarians had struggled for some time to augment the tried and true production oriented statistics represented in ARL Statistics with service assessments (Andaleeb and Simmonds, 1998; Coleman et al, 1997; Nitecki, 1996a, 1996b, 1998; Stein, 1998). Over the past several years the Association of Research Libraries has confronted the potential divergence between statistical measurement of expenditures for ranking purposes and the need for additional measures to assess outcomes. From an ARL retreat held in Tucson in January, 1999 emerged a dedicated effort to develop the data and measurement tools that could accurately describe today’s research libraries. (ARL New Measures: 2). A recent call by the Association of Research Libraries’ (ARL) Board for new measures for determining library performance included user satisfaction as one of eight areas of focus for study. Although the use of SERVQUAL to assess service quality in library settings has been the subject of several studies, to date there has not been a report of comparisons of SERVQUAL results over multiple years in academic libraries. The General Libraries of Texas A&M University in College Station, Texas was a SERVQUAL pioneer in administering the survey in 1995 and has surveyed users subsequently each two years in 1997 and in 1999.
Our analysis of SERVQUAL considers both issues generalizable to academic libraries as a whole, and local, strategic issues of use in managerial decisions at the institution level, in our case, the General Libraries at Texas A&M University. General issues include:
· Are SERVQUAL scores valid and reliable?
· (Are there statistically significant differences between years and among user groups by year?
Specific issues of strategic interest for local library administrators are considered at the individual question level, particularly :
· What are the six most/least important questions for users?
· Are most/least important minimum, desired and perceived expectations the same from year to year?
· What obvious gaps in service quality, i.e., differences between minimum and perceived expectations exist?
The SERVQUAL instrument used at Texas A&M consisted of 22 questions and a set of 5 questions that were used to assign weights, or relative importance, to the larger question set. As shown in Table 1 respondents answered each question on three scales: minimum, desired and perceived ratings on a Likert-type scale of 1 (lowest) to 9 (highest). An identical instrument was used for each of the three assessments in 1995, 1997 and 1999. Samples of staff, faculty, undergraduate and graduate students were derived on a random basis from university student registration and payroll databases. The survey was mailed to respondents with standard follow up procedures. A total sample of nearly 700 responses for the three years was analyzed.
Q.1 / Prompt service to customersQ.2 / Employees who are consistently courteous
Q.3 / Employees who deal with customers in a caring fashion
Q.4 / Providing service at the promised time
Q.5 / Employees who understand the needs of their customers
Q.6 / Visually appealing materials associated with the service (e.g., clear and concise forms)
Q.7 / Having the customer’s best interest at heart
Q.8 / Willingness to help customers
Q.9 / Maintaining error-free customer and catalog records
Q.10 / Keeping customers informed about when services will be performed
Q.11 / Providing services as promised
Q.12 / Employees who instill confidence in customers
Q.13 / Knowledge who have the knowledge to answer customers’ questions
Q.14 / Readiness to respond to customers’ questions
Q.15 / Dependability in handling customers’ service problems
Q.16 / Performing services right the first time
Q.17 / Visually appealing facilities
Q.18 / Giving customers individual attention
Q.19 / Employees who have a neat, professional appearance
Q.20 / Convenient business hours
Q.21 / Modern equipment
Q.22 / Assure customers of the accuracy and confidentiality of their transactions
Table 1
General Issues
Are SERVQUAL scores valid and reliable? Reliability, or internal consistency, of SERVQUAL scores for all three years was evaluated by computing Cronbach’s alpha coefficients, which were uniformly high across various scales, years and user groups. Construct validity was evaluated with a factor analysis of SERVQUAL scores to determine whether we were accurately testing the five dimensions intended by the originators of the instrument (i.e., tangibles, reliability, responsiveness, assurance and empathy). In similar results to those reported by Nitecki (1996) and Andaleeb and Simmonds (1998), our factor analysis identified three rather than five factors:
· Affect of Service Experience, which is primarily a confluence of the responsiveness, assurance and empathy dimensions;
· Service Reliability, fairly analogous to the reliability dimension; and
· Tangibles (Cook and Thompson, 1999).
Were there noteworthy differences between results by year or by role group? Analysis of variance (ANOVA) tests for minimum, desired and perceived expectations at the dimension and item level by year yielded little statistically significant data and when p<.05 results were obtained, eta2 effect sizes were very small. Even these may have been more the result of fairly large sample sizes rather than any particularly noteworthy change in user expectations over the five years under study. Analyses by user group--faculty, university staff, graduate or undergraduate student--also produced similar results.
Institution-level Strategic Issues
Since a recent factor analysis (Cook and Thompson, 1999) failed to recover the conventional five SERVQUAL dimensions, and therefore, the validity of analysis by five dimensions appears questionable, we examined responses to item or individual questions for trends which would be useful for library managers.
What questions were the most and least important to respondents in terms of highest and lowest means? Were these consistently rated over the three survey administrations? The same six questions received the highest mean responses of the 22 questions for minimum expectations each year. Question 11, “Providing services as promised,” received the highest mean response for minimum expectations for all three years. The next five questions with highest means were the same for the three years under study, but varied in order. Those included, Question 20, “Convenient business hours,” Question 13, “Employees have knowledge to answer customers’ questions,” Question 8, “Willingness to help customers,” Question 9, “Maintaining error-free customer and catalog records,” and Question 4, “Providing service at the promised time.”
Figure 1
The six questions receiving the lowest minimum means scores fell into the tangibles factor and were also consistent over the three survey administrations: Question 17, “Visually appealing facilities,” Question 6, “Visually appealing materials associated with the service,” and Question 19, “Employees who have a neat, professional appearance.” It is noteworthy that overall means for minimum expectations have been gradually rising from 1995 to 1999. Users expect more of us over time.
Regarding desired service expectations, Question 13, “Employees having the knowledge to answer customer questions” received the highest or second highest mean score all three years and Question 11, “Providing services as promised came in a close second. Question 8, “Willingness to help customers” and Question 20, “Convenient business hours” were also consistently ranked in the highest six mean scores across years in a fashion parallel to minimum expectations.
Figure 2
Three questions were included inconsistently in the top six desired across years: Question 15, “Readiness to respond to customers’ questions,” Question 21, “Modern equipment,” and Question 4, “Providing service at the promised time.” In that these questions were not included in users highest minimum mean rankings, these may constitute the discriminant factors in respondents’ views of minimum vs. desired expectations. Consistent with minimum responses were the three questions related to tangibles with the lowest desired mean scores.
Figure 3 shows those questions receiving the highest mean scores for perceived quality service by year.
Figure 3
Parasuraman, Berry and Zeithaml (1985, 1988, 1990, 1991, 1994) measure service adequacy as the difference between customers’ minimum and perceived expectations, and service superiority as the difference between desired expectations and perceived expectations. Service quality is assessed by discerning where perception falls within a zone of tolerance, i.e., the range between minimum and desired scores.
Management’s goal is to achieve perceived rankings as close to desired scores as possible on the service quality issues most important to users (questions with highest minimum means), while altogether avoiding perceived scores falling outside the zone of tolerance. As an example, Figure 4 indicates the zone of tolerance in the 1999 survey.
Figure 4
The question with the highest perceived mean score for all three years was Question 20, “Convenient business hours.” This question was also consistently ranked among the top six minimum and desired expectations --- good news for library managers at Texas A&M University. Question 4, “Providing services at the promised time”, Question 8, “Willingness to help customers,” and Question 11, “Providing services as promised” were also consistently included in the top six mean scores for minimum, desired and perceived expectations—more good news. In contrast, however, Question 9,“Maintaining error-free customer and catalog records” was listed in the top minimum means, but among the three lowest mean perception rankings over the three years—not good news for library management. In a similar manner there is an apparent gap between minimum, desired and perceived expectations for Question 13, “Employees have knowledge to answer customers questions.” While this question ranked highest or second highest in mean scores of the desired responses; perceived rankings have precipitously declined from 3rd highest in 1995 to 13th in 1999.
Further insight into the nature of the two problem areas identified in Question 9, “Maintaining error-free customer and catalog records,” and Question 13, “Employees have knowledge to answer customers’ questions” is discernible by examining responses by user group to these questions. Figure 5 shows that faculty and graduate students are most concerned by the lack of reliability in record keeping.
Figure 5
While improvements have been made in the minds of undergraduate students in this area, earlier marginal improvements in 1997 have been lost in the view of faculty and graduate students in 1999. Figure 6 shows dramatically how perceptions of Question 13, “Employees have knowledge to answer customers’ questions” have fallen precipitously in all but the staff user group in 1999.
Figure 6
Quantifying the Gap Model: Six Sigma
Typically, manufacturing firms take the lead in implementing quality programs to enhance productivity and to improve customer satisfaction. Now, service organizations, such as libraries, are beginning to understand what their manufacturing counterparts learned--that quality does not improve unless you measure it. A successful program employed in manufacturing, six sigma, could also have application in the service sector using SERVQUAL data.
The Greek letter sigma is used as a symbol by statisticians to denote the standard deviation of a set of data. Six sigma is a disciplined, quantitative approach to analyzing the root causes of problems and solving them. It involves measuring, analyzing, improving and controlling a process such that tolerance limits are six standard deviations away from the process mean. A process rated at six sigma will produce fewer than 3.4 defects per million operations.
Phillip Crosby, in Quality is Free (1979), introduced the concept of zero defects. During the 1980's, Motorola Corporation took the concept of zero defects a step further and institutionalized six sigma methodologies to improve production of pagers, cellular phones, and other products. Their success with six sigma techniques popularized it as a tool for product improvement in every facet of business. The purpose for practicing six sigma techniques is to create a process for tracking annual improvements in customer satisfaction and to provide a common basis for benchmarking against best-in-class libraries. Also, "this measurement standard allows comparisons of similar and dissimilar processes and companies of various sizes and in various industries" (Fontenot, p. 73).
The radar graph derived from the SERVQUAL responses highlights the library's strengths and weaknesses with regards to customer satisfaction. In particular, Figure 4 shows that the average perceived score for Question 9, Maintain error free customer and catalog records, falls out of the zone of tolerance. When the perceived score is subtracted from the minimum score for each of the 231 survey respondents, the distribution is represented by the bar graph in Figure 7. The grey bars (negative scores) represent scores where perceived is less than the minimum level. The black bars represent perceived scores that are either equal to or greater than the minimum level.