Information Technology Survey 2008
Angelo State University
ABSTRACT
The Information Technology (IT) survey has been used to obtain information about the technology services provided by Angelo State University for several years. Since services are implemented for students’ use, the feedback from the survey helps the IT department identify trends and needs in the technology area from the students’ perspective. The results are then used to determine how to allocate funds between improving current services and developing new services that students request.
During the 2008 spring semester, surveys were distributed from March 4th to April 10th. There was a hard copy version of the survey that was distributed to classes throughout campus. Although classes were randomly selected, an attempt was made to include students from all the colleges. There was also an electronic version of the survey that was available online. Two different methods were used to promote participation using the online version of the survey. As in previous years, a link to the survey was placed on all computers on campus inviting students to participate and give their opinion. Also, the assistance of all professors that taught a class in a computer lab was solicited. Those professors received an email that urged them to direct their students to the survey link. The number of surveys completed was 534, with 261 paper surveys and 273 online surveys being returned.
Using the findings from previous SERVQUAL analysis, three dimensions of service were found to be relevant when measuring students’ satisfaction with the IT department at ASU. The three dimensions, along with the characteristics that make them up, are described below.
Staff (Factor 1) display the following characteristics:
· Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence
· Empathy – Providing care and individualized attention
· Responsiveness - Willingness to help customers and provide prompt service
Services (Factor 2) display the following characteristic:
· Tangibles – Appearance of physical facilities, equipment, and personnel
Professionalism (Factor 3) display the following characteristic:
· Reliability - Ability to perform the promised service dependably and accurately
Factor analysis was used to determine what variables made up each dimension of service. Once the variables were grouped into the appropriate factor, a SERVQUAL score was calculated for each factor using the model below:
Q = PERCEIVED – DESIRED
The SERVQUAL score measure the difference between a student’s perceived satisfaction level and a student’s desired satisfaction level. A mean score was then calculated. The graph below shows the SERVQUAL mean score for each factor.
The results show the biggest difference between a student’s perceived level of satisfaction and their desired level is in Factor 2 – Services. For services, the average difference between perceived and desired satisfaction level is -0.558. Below are several recommendations that we feel will decrease the margin for Services:
Computer labs need to provide and make available more software and hardware for students to complete their assignments.
Staff needs to provide routine test and maintenance to ensure software and hardware operates smoothly.
Computer labs need to provide a wider variety of software, both educational and entertainment.
Personal disk space provided on the server (P: drive) needs to be increased.
After analyzing the feedback provided by students in Part III of the survey, we suggest the following:
Although the IT department does a good job of providing multiple services to the students, many of the software programs are not available in all the computer labs. Students don’t find this convenient because the lab that contains the program they need may not be open 24 hours. Students want all the software programs to be available at every computer lab.
Students are no longer satisfied with having lab assistants that have basic computer knowledge. They want the lab assistants to have more expertise and be able to support the software programs that IT offers.
Students know that the IT department provides many services to the students. Many students mentioned that they would like to be more informed about what those services are. Many times they don’t know about the availability of a service until it has been around for a semester or two. The IT department needs to promote their services more in order for students to take full advantage of them.
Last, most students requested for a faster log on time.
This report goes into detail on the analysis that was done to reach the conclusions provided above about the data that was collected. Also, complete results of all the questions in the survey are summarized.
Information Technology Survey 2008
Angelo State University
INTRODUCTION
Angelo State University seeks to provide students with Information Technology facilities that will help students develop their educational goals. The Information Technology (IT) Department goal is to provide quality services to students, faculty and staff at Angelo State University and meeting their network needs.
The IT department counts with eight computer labs, seven of which are managed by IT for general student use, “…there are over 700 computers available for student use. The university has adopted a refreshment cycle of three years for all student use computers and peripherals located in student computer labs and specialized classrooms.” (ASU website) to offer customers the most up-to-date technology available. With the constant updates for technology equipment and services available came the development of a customer satisfaction survey that allowed IT to obtain information on how students perceived the new services and equipment available.
The IT survey started in 1997, and it has been conducted yearly since then. The results from this survey help the IT department to obtain students’ overall satisfaction level with the IT department’s service level and the new technology and equipment that is available. They also receive good feedback from the students on areas to improve and what new services to implement.
SURVEY DESIGN
The IT survey has been modified over the past ten years. It consists of three parts. Part I focuses on service quality and uses the SERVQUAL instrument that was developed by Zeithaml, Parasuraman, and Berry. Part II was developed by the IT staff and obtains students’ opinions on services provided and gauges students’ familiarity of available services. Part III is used to obtain demographic information of students. It also gives students an opportunity to make recommendations on what services they would like to see in the future.
Part one uses the SERVQUAL instrument developed by Zeithaml, Parasuraman, and Berry. It consists of twenty-four questions scaled from 1 (low) – 9 (high) scale. Each question has two scales, one measures the students’ desired service level and the other one measures their perceived service level. In their original study, Zeithaml, Parasuraman, and Berry developed seven dimensions. These seven dimensions were: Communications, Credibility, Security, Competence, Courtesy, Understanding and/or knowledge of customers, and Access. However, only five dimensions were noted in the end because the researches had difficulty maintaining distinction after two stages of scale purification on two of the dimensions.
In research done by David Hughey, three dimensions of service were identified using factor analysis. “With the limited number of samples returned in the ASU surveys it is not possible to maintain the distinction for the five dimensions noted above. Some manipulation of the author’s five dimensions is in order. So, as Parasuraman and company combined seven dimensions into five, five dimensions can be combined into three dimensions for use at ASU.”
To the twenty-four existing questions, question 25 was also added. Question 25 measured the students’ overall satisfaction with the IT department.
Part II of the survey has been modified according to the new services and product offering that the IT department makes available to the students. Each year, the IT staff determines what services they want to receive feedback from. Also, they provide the students with several ideas of future product offerings and services.
Part III of the survey contain questions for demographics. Also in Part III, there are questions that allow the students to provide feedback on what they particularly like about what IT has to offer and what they want to see improved.
DISTRIBUTION AND ANALYSIS
To gather data, a one stage cluster sample was administered. Surveys were distributed from March 4th to April 10th. There was a hard copy version of the survey that was distributed to classes throughout campus. Although classes were randomly selected, an attempt was made to include students from all the colleges. There was also an electronic version of the survey that was available online. Two different methods were used to promote participation using the online version of the survey. As in previous years, a link to the survey was placed on all computers on campus inviting students to participate and give their opinion. Also, we solicited the assistance of all professors that taught a class in a computer lab. We emailed those professors and urged them to direct their students to the survey link. The number of surveys completed was 534, with 261 paper surveys and 273 online surveys being returned.
All the surveys were input in Microsoft Excel. They were then imported into SAS 9.1 so statistical analysis could be run on the data.
The first statistical test done on the data was a factor analysis to determine what variables made up each factor. Although the existence of three dimensions of service has been established, the variables that make up each factor can change from year to year. When running factor analysis, only the perceived variable for each question was used. After running the factor analysis, the variables were grouped in the following manner to make up each factor.
Staff Factor 1 = A2+A4+A6+A8+A10+A12+A14+A16+A18+A20
Services Factor 2 = A38+A40+A42+A44+A48
Professionalism Factor 3 = A22+A26+A28+A30+A34
Based on the factor analysis results, the following shows what questions make-up each factor.
Staff (Factor 1) is based on the following questions:
Q1- The computer lab staff’s willingness to help me
Q2 - The staff’s ability to instill confidence in me to use the computer lab software and hardware
Q3 - Staff who has the knowledge to answer my questions about computer lab policies, operating hours, software, and hardware
Q4 - Computer lab staff who deal with me in a caring fashion
Q5 - Receiving a prompt response to my computer lab problems and/or suggestions
Q6 - Computer lab staff who have my best interest at heart
Q7 - Computer lab staff who understand my computing needs (Q7)
Q8 - Computer lab staff who give me personal attention (Q8)
Q9 - Staff who have the knowledge to answer my questions about software and hardware.
Q10 - Providing computing services as promised in the ASU catalog, computer lab brochure, or web page
The above questions are grouped into Factor 1 because they display the following characteristics:
· Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence
· Empathy – Providing care and individualized attention
· Responsiveness - Willingness to help customers and provide prompt service
Services (Factor 2) is based on the following questions:
Q19 - Counting on the computer labs to have the software and hardware I need to complete my assignments
Q20 - Relying on the computer lab software and hardware to operate smoothly
Q21 - The computer labs containing state-of-the-art computers and peripherals
Q22 - The variety of software available in the computer lab
Q24 - Disk space provided to me on the server (my P: drive) is adequate for my data storage
The above questions are grouped into Factor 2 because they display the following characteristics:
· Tangibles – Appearance of physical facilities, equipment, and personnel
Professionalism (Factor 3) is based on the following questions:
Q11 - Services provided at times listed
Q13 - Computer lab staff can be distinguished from other students and are easy to identify
Q14 - Ability to find an available work station in one of the six labs on campus.
Q15 - Being directed by staff to an open workstation at another computer lab if the first computer lab is full
Q17 - Computer lab staff who are dressed appropriately for their position
The above questions are grouped into Factor 3 because they display the following characteristics:
· Reliability - Ability to perform the promised service dependably and accurately
Three of the twenty-four questions were dropped. The following questions were dropped because the factor analysis assigned them to more than one factor.
Q12 - A visually appealing computer lab (i.e. neat, clean, and organized)
Q16 - Convenience of normal operating hours to personal schedule
Question 18 was not assigned to any factor and was dropped for that reason.
Q18 - Computer response time during log-in or application loading.
After running the factor analysis to determine what questions belong to what factor, the next step is to calculate the SERVQUAL score using Gap Analysis. The SERVQUAL score compares the difference between the students’ desired service level and their perceived service level. A score, depicted by variable Q, for each question is calculated using the model below:
Q = PERCEIVED – DESIRED
Once each question has been assigned a score, the next step is to obtain a mean score for each factor. The model below shows how the means were calculated:
Factor 1: Score 1 = (Q1+Q2+Q3+Q4+Q5+Q6+Q7+Q8+Q9+Q10) / 10
Factor 2: Score 2 = (Q19+Q20+Q21+Q22+Q24) / 5
Factor 3: Score 3 = (Q11+Q13+Q14+Q15+Q17) / 5
The mean for each factor, score x, was obtained by adding the questions for each factor and dividing the sum by the number of questions that made up that factor. The means for each factor are as follow shown in the table and graph below:
SERVQUAL Mean ScoresFactor 1 (Staff) / -0.498
Factor 2 (Services) / -0.558
Factor 3 (Professionalism) / -0.527
Another score that was calculated was ScoreT using the following model:
ScoreT = Score 1 + Score 2 + Score 3
ScoreT was then used to run a correlation with Question 25. Question 25, the last question in Part 1, measured the students’ overall computer lab satisfaction level. The values obtained using the Cronbach alpha are as follow: