Information Technology Survey

Information Technology Survey

Spring 2011

BA 6303

Business Research Methods

Prepared for:

Dr. Sudhir Chawla

Prepared by:

Daniel Kent

David Solomon

Aaron Nelson

Cody Cooper

Josh Augustin

Abstract

The Information Technology Department at Angelo State University has been distributing a customer satisfaction survey since 1997. It is used to identify popular IT trends in order to add, remove, or modify services and to gauge the student population’s satisfaction with the university computer labs and IT services.

Classrooms were selected randomly and surveys were distributed between March 24, 2011 and April 8, 2011. Surveys were collected in classroom based and online formats. From a total of 541 collected, 70 were not usable, leaving 471 which could be used for analysis.

The survey consisted of three parts. Part I remained unchanged from the prior year’s survey. It is based on the SERVQUAL instrument by Zeithaml, Parasuraman, and Berry which measures satisfaction with three dimensions of service, including Staffing, Services, and Professionalism. Part II was modified to bring the survey up to date with current service offerings. Part III consisted of demographic questions. One question was added, and the rest of part III remained unchanged.

Part 1 of the information technology survey measures gap five of service quality, the difference between customer’s perceptions and expectations. It consists of three SERVQUAL factors. These factors include the following:

Staff (Staffing) - This factor’s questions are based on the following dimensions:

· Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence

· Empathy – Providing care and individualized attention

· Responsiveness - Willingness to help customers and provide prompt service

Professionalism (Setting) - This factor’s questions are based on the following dimensions:

· Reliability - Ability to perform the promised service dependably and accurately

Services (Tools) - This factor’s questions are based on the following dimensions:

· Tangibles – Appearance of physical facilities, equipment, and personnel

Each of the SERVQUAL factors consist of multiple questions that are grouped and separated on the survey for convenient analysis. The groupings are listed as Staffing, Setting, and Tools on the survey. To determine the SERVQUAL score for each factor, the following equation is utilized:

Q = PERCEIVED – DESIRED

This equation measures the difference between the respondent’s perceived and desired level of satisfaction for a particular question. The scores for each question were then averaged and an overall average for each factor was determined. The following graph illustrates the results:

The factor with the largest discrepancy between the students’ desired level of service and the service they actually perceive is under the Professionalism factor (-0.95). The Staff factor showed the second largest difference (-0.80) followed by the Services factor (-0.68).

We have also compared the mean scores calculated in 2011 against the scores calculated in the five previous years. A summary of those figures is shown below:

Based on this graph, the Professionalism factor has decreased the most by 0.53 from last year. The Staff factor also decreased slightly by 0.03 from last year. Finally, the Services factor increased by 0.07 from last year. This shows clear justification for training in the factor professionalism.

Below are a few recommendations we feel will improve these scores:

· Increase Staff knowledge of hardware and software

· Become more proactive at directing students to open work stations particularly to other labs if the current lab is full

· Improve computer response time during log-in and application loading

· Continually update lab computers and peripherals as funds allow

Topics of interest this year:

The new RamMail system rated above average satisfaction (mean 4.30) and above average scores on all attributes on scales of 1-5.

· Ease of use: 4.27

· Mobile Access: 3.87

· Calendar: 3.80

· Increased storage space: 3.60

The largest percentage of students (43%) feel 1,500-2,000 printing pages a semester is appropriate (1,931 mean)

Possibly cut the following services if budget issues arise:

· Student owned laptop support (66% of students surveyed have not used)

· Email Express Kiosks (69% of students surveyed have not used)

· Camcorders/Digital Camera checkout (86% of students surveyed have not used)

· Learning Commons Equipment Checkout (86% of students surveyed have not used)

The following sections of this report will go into greater detail about the SERVQUAL instrument and the factors utilized in this survey; as well as providing additional detail and analyzed results of all three parts of the survey.

Introduction

Student satisfaction is an important part of the Information Technology Department at Angelo State University. The department strives to bring the most value possible to the services it offers the student body. It brings this value by constantly measuring performance and improving its services. With the rapid changes in this field, it is important to continually modify its offerings to stay up to date with the latest technology as well as current usage trends in the student population. Each year since 1997, a survey has been given to gauge student needs and to provide a basis for modifying its services.

There are currently four computer labs on campus for general student and class use with over 100 applications on each computer. The MCS 111 lab is available 24 hours a day while classes are in session. There are also 280 wireless access points provided around campus.

The survey measures the performance of the services being offered on campus, provides demographic information, and asks for feedback on possible new offerings the students would like to have available at the university. This year, questions were introduced to find student usage of laptops and to examine the possibility of adding laptop printing stations. Also questions evaluating the new RamMail system were also added. Continuing to provide services with the prospect of a reduced future budget was also a key point of interest this year.

Survey Distribution, Design and Methodology

During the spring 2011 semester, surveys were distributed to randomly selected classes between March 24th and April 8th. The majority of surveys completed were paper-based surveys passed out in classes. Professor approval was obtained prior to distribution of the surveys. A number of surveys were also taken by students directly online. A total of 541 surveys were completed. Of those surveys, 471 were valid and utilized for this report. Surveys were determined invalid if they were less than 70% complete, or if students’ answers were biased.

Part One: SERVQUAL

As previously discussed, part one of the survey uses the SERVQUAL instrument developed by Zeithaml, Parasuraman, and Berry. Part one consists of twenty-five questions scaled from 1 (low) – 9 (high). Each question has two versions. One measures the students’ desired service level and the other measures their perceived service level. SERVQUAL determines the mathematical difference between desired level of service and the level of service actually perceived. The initial SERVQUAL research yielded seven dimensions, but this was later scaled down to five dimensions to maintain distinction (Berry, et al. 26)

The five distinct SERVQUAL dimensions are:

§ TANGIBLES

Physical facilities, equipment, and appearance of personnel.

§ RELIABILITY

Ability to perform the promised service dependably and accurately.

§ RESPONSIVENESS

Willingness to help customers and provide prompt service.

§ ASSURANCE

Knowledge and courtesy of employees and their ability to inspire trust and confidence.

· EMPATHY

Caring, individualized attention the firm provides its customers.

It is not possible to maintain all five SERVQUAL dimensions due to the limited number of surveys typically returned from the I.T. survey. Therefore, the five dimensions have been combined into three. The three ASU survey service dimensions utilized, and the questions from the survey included in each dimension are as follows:

Factor 1: focuses on Staff and consists of questions concerning the following dimensions:

§ Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence

§ Empathy – Providing care and individualized attention

§ Responsiveness - Willingness to help customers and provide prompt service

The questions that made up this factor include:

· Q1 - The computer lab staff’s willingness to help me

· Q2 – The staff’s ability to instill confidence in me to use the computer lab software and hardware

· Q3 – Staff who have the knowledge to answer my questions about computer lab policies, operating hours, software, and hardware

· Q4 – Computer lab staff who deal with me in a caring fashion

· Q5 – Receiving a prompt response to my computer lab problems and/or suggestions

· Q6 – Computer lab staff who have my best interest at heart

· Q7- Computer lab staff who understand my computing needs

· Q8-– Computer lab staff who give me personal attention

· Q9 – Staff who have the knowledge to answer my questions about software and hardware

Factor 2: focuses on Professionalism (Staffing) and consists of questions concerning the following:

§ Reliability - Ability to perform the promised service dependably and accurately

The questions that made up this factor include:

· Q10 – Providing computing services as promised in the ASU catalog, or web page

· Q11 – Services provided at times listed

· Q12 – A visually appealing computer lab (i.e. neat, clean, and organized)

· Q13 – Computer lab staff can be distinguished from other students and are easy to identify

· Q14 – Ability to find an available workstation in one of the five labs on campus

· Q15 – Being directed by staff to an open workstation at another computer lab if the first computer lab is full

· Q16 – Convenience of normal operating hours to personal schedule

· Q17 – Computer lab staff who are dressed appropriately for their position

· Q18 – Computer response time during log-in or application loading

Factor 3: focuses on Services (Tools) and consists of questions concerning the following:

§ Tangibles – Appearance of physical facilities, equipment, and personnel

The questions that made up this factor include:

· Q20 – Counting on the computer labs to have the software and hardware I need to complete assignments

· Q21 – Relying on the computer lab software and hardware to operate smoothly

· Q22– The computer labs containing state-of-the-art computers and peripherals

· Q23 – The variety of software available in the computer lab

· Q24 – The variety of special equipment (i.e. scanners and color printers) available in the computer labs

· Q25 – Disk space provided to me on the server (my P: drive) is adequate for my data storage needs

Q19 -“What is your overall satisfaction level with ASU computer labs?’ was analyzed separately because the question encompasses all factors.

To determine the SERVQUAL score for each factor, the following equation was utilized:

Q = PERCEIVED – DESIRED

To ensure internal reliability of the scale, a Cronbach Alpha test was run on each of the three factors individually as well as running all three factors together. Cronbach’s Alphas will generally increase as the intercorrelations among variables increase, therefore, a larger number typically indicates greater internal consistency. A score of .70 or greater is considered sufficient for internal consistency. The following graph depicts the Cronbach Alpha results:

Based on these tests, we can conclude sufficient internal consistency and reliability of this scale.

Statistical Analysis

T-Test

A t-test is typically run in order to make inferences about the mean of a population. It will help determine if the means between two variables differ. The null hypotheses of these tests are that there is no difference in the means of the two variables being compared, while the alternative hypotheses are that there is a difference in the means. A significance level of .05 was used for analysis and normal distribution was assumed. It is important to note the types of data that can be included in a t-test. The dependent variables must be interval or ratio scale (in business, categorical data is treated as an interval scale). The independent variables must be nominal or two level categorical data. T-test results will determine if there is a significant difference between the responses of the independent variables. There are three questions in the I.T. survey that can be utilized as independent variables for T-test purposes. These are:

Q26: Do you own a personal computer or laptop?

Q60: Gender

Q65: I live (on/off campus)

Each of these questions were run against the three SERVQUAL factors, Q19 SERVQUAL (overall satisfaction of labs) and all scale questions (Q30, Q32, Q34, Q36, Q37, Q40-56, Q59). All significant results are listed below. Results were significant if the T Significance value was less than .05.

Q26: Do you own a personal computer or laptop?

No significant results found.

Q60: Gender

Q30: RamMail Increased Storage Sig .000

Males liked RamMail increased storage more (mean 3.94) then females (mean 3.27).

Q36: Overall satisfaction level with ASU technology services Sig .001

Females were more satisfied with the Overall satisfaction level with ASU technology services (mean 93%) than males (mean 91%).

Q45: Blackboard Satisfaction Sig .016

Females were more satisfied with Blackboard (mean 4.09) than males (mean 3.86).

Q55: ASU Alert Satisfaction Sig .013

Females were more satisfied ASU Alert (mean 4.11) than males (mean 3.82).

Q56: ASU Mobile Application Satisfaction Sig .018

Females were more satisfied ASU Mobile Application (mean 3.75) than males (mean 3.28).

SERVQUAL Staffing Sig. 004

Males were less dissatisfied with the Staffing factor (mean -0.53) then females (mean -0.98).

SERVQUAL Professionalism Sig .044

Males were less dissatisfied with the Professionalism factor (mean -0.76) then females (mean -1.08).

Q65: Live on/off campus

Q30: RamMail ease of use Sig .045

People who lived off-campus liked RamMail ease of use more (mean 4.37) then people who lived on campus (mean 4.13).