An Evaluation of E-Learning Program: a Case Study at Institute Of

An Evaluation of e-Learning Program: A Case Study at Institute of

Education Development

By

Syed Jamal Abdul Nasir, MohdRashidee Alwi ,Jamaliah Said

Institute of Education Development (InED)

Universiti Teknologi MARA, Shah Alam.

Abstract

This paper reveals the result of 322 distance learners’ perception towards e-learning program conducted by UiTM. Generally the respondents rated above average for all aspects of distance learning program irrespective of gender, program of studies, income and occupation. Students’ gender also did not show any difference in their perception. Similarly, semester of studies too, did not indicate any significant difference except their perception towards lecturer. However students’ semester of studies showed significant difference towards lecturer, module and physical. Gender, income and semester of studies did not show any relationship to students’ perception towards all aspects of distance learning program. However students’ program of studies showed significant relationship towards their perceptions of the program. Students’ CGPA showed negative relationship with all aspects of distance learning program.

Keywords: e-Learning and evaluation

Introduction

UiTM being one of the established public universities in Malaysia was formed in 1956 which was known as Dewan Latehan RIDA. After a few stages of restructuring, on 26August 1999 finally it was upgraded into a university status and was known as Universiti Teknologi MARA. The vision is to become a distinguished university aspiring to promote academic excellence in higher education. In its effort to assist those who do not have the opportunity to study fultime, UiTM under one of its department known as Institute of Education Development (InED) offers a variety of programs conducted via different modes. One of the major modes is distance education which first started the program in 1990.

Inline with the government vision in encouraging and promoting IT society especially when the Prime Minister launched Multimedia Super Corridor (MSC) in 1996, UiTM took the challenge by shifting its distance education program to electronic distance Education Program which known as Flexible Learning Program (FLP) in January 1998. The Ministry of Education officially launched the FLP program on 26 April 1999. In December 1998 there were only three (3) subjects being offered with seven lecturers conducted the programs with only 72 registered students. Today, the enrollment has increased to 5300 students.

E-Learning Model of UiTM

The e-PJJ/FLP is designed to take advantage of the developments and advancements in distance learning and education with Information and Communication Technology. These developments and advancements are harnessed into forms of support for students to enhance their learning. Thus InED uses the web as an adjunct of on-line learning which consists of eight main components that are:

Figure 1: The Eight Components of Flexible Learning Program (FLP)

Problem Statement

UiTM has conducted the e-learning for almost a decade. It has gone into vast development, issues and critics. Thus, it leads to the need for in-depth study towards its perception from the views of adult learners. These perceptions gathered based on feedback derived from studies conducted are crucial to the enhancement and improvement of e-learning. Guidelines and suggestion for improvement resulted from empirical study, will be able to be tailored the needs and interests of the institutions, system providers (servers) and the learners. Therefore several questions need to be answered such as:-

1. What is the student’s perception towards the program?
2. Are there any differences in perception between genders, program of studies, and semester of studies?
3. Is there any correlation between students’ perception and genders, program of studies, semester of studies, age and income?

Research Objectives

The specific objectives of the study are as follows:

1.To determine distance learner’s perception towards e-learning programs.

2.To determine differences in perception between genders, program of studies, and semester of studies.

3.To determine correlation between students’ perception and genders, program of studies, semester of studies, age and income.

Methodology

Research Design

This study is a descriptive research carried out among distance learners in the Institute of Education Development (InED), UiTM. A Self developed questionnaire was used to evaluate students’ perceptions towards e-learning programs. The data was gathered just once, making the research a cross sectional study. The secondary data was collected from reviews of prior researchers, journals, reports, articles relating to this study. The factors being evaluated are lecturers, the course, modules, IT infrastructure, physical facilities and support services.

Theoretical Framework

The theoretical frameworks explain the relationship between gender, programmes of studies, semester of studies, income and academic achievement to students perception towards course, lecturer, module, physical facilities IT infrastructure and support services.The above framework indicates that the focus of the study is to measure student perceptions towards six aspects of e- learning programmes those are:-

Lecture. level of lecturers’ commitment, competency and their quality of teaching delivery.

Course: Respondents were asked to evaluate on the quality of the course content and to what extent they perceived the syllabus content can help them to improve communication skills, creative skill, problems solving and other knowledge.

THEORETICAL FRAMEWORK

Figure 1: Theoretical Framework

Module: The respondents were asked to evaluate their perception towards quality of module in term of usefulness, organization, clarity, contents, accessibility and the price charge.

IT infrastructure: Refer to IT training sufficiency, accessibility of information, virtual library service and accessibility to web-pages.

Physical facilities: Refer to respondent’s satisfaction toward the quality of library facility, classroom environment, and teaching equipment.

Support service: Students were asked to assess their perception towards the quality of services provided by the institute. Among the support service covered in the questionnaire were enrolment process, quality of services provided by academicstaff, administration of examination and handling of students complaints.

Population and Sample

A total of 500 questionnaires were distributed to the distance learners of InED, UiTM. The distance learners were from diploma and bachelor level.. The students were randomly selected. Three hundreds and twenty two returned the questionnaires, which was 64.4 percent.

Reliability and Validity

The Cronbach Alpha values showed more than .07 which means the reliability of the instrument for each aspect is considered high.

Data Analysis

Three statistical procedures were involved in the data analysis in answering the research objectives as follow.

1.To determine distance learner’s perception towards e-learning program.

Means scores were used to determine the students’ perception towards six aspects of e- learning programs.

2.Independent t-Test was executed to determine differences in perception between genders, program of studies, and ANOVA forsemester of studies.

3To examine the correlation between students’ perception and genders, program of studies, semester of studies, age and income, Spearman Rank was used.

Research Findings

Distances’ Learners Perception Towards e-Learning Program

Table 1: Students’ Perception in General

Mean_Lecturer / Mean_Course / Mean_Module / Mean_IT / Mean_facilities / Mean_Service
Mean / 3.7867 / 3.8994 / 3.6372 / 3.4607 / 3.34163 / 3.6810
Rank / 2 / 1 / 4 / 5 / 6 / 3

Table 1 shows in general, respondents ranked highest for the quality of course content, followed by lecturer, support services, module and IT facilities. The finding shows that respondents agreed quality of physical facilities (such as classroom condition, teaching aid and library facilities) is the lowest in the rank with mean score of 3.34. This perhaps due to unavailability of own building whereby InED keeps shifting their classroom from one place to another. However the mean score for every factor indicates that respondents in general were satisfied with the e-learning program.

Students’ Perception by Gender

Table 2: Students’ Perception by Gender

Gender / Lecturer / Course / Module / IT / Physical / Service
male
Mean / 3.8348 / 3.9607 / 3.7281 / 3.5225 / 3.4363 / 3.7155
female
Mean / 3.7548 / 3.8619 / 3.5841 / 3.4198 / 3.3978 / 3.6559

Table 2 exhibits students’ perception by gender. Both groups indicate above average means score for every aspects of distance learning program. However it is interesting to note that male respondents were more satisfied as compared to female counterparts as indicated by higher mean score for every item measured in this study. This may be due to different level of expectation of the two groups as argued by Gurian (2001) that male and female have different set of expectation in mind.

Table 3 shows students’ perception by program of studies. The mean score clearly highlight that, degree students showed lower mean scores for every factor measured in this study as compared to the diploma students. This indicates that higher level students rated lower the services provided by the institution as compared to the diploma students. This may be due to higher expectation of bachelor students as the may be holding better position at their work place. Unlike the diploma students who are generally the supporting staff.

Schedul 3: Students’ Perception by Program of Studies

program / Lecturer / course / module / IT / Physical / Service
Diploma / Mean / 3.8289 / 3.9138 / 3.7104 / 3.4769 / 3.4639 / 3.6800
degree / Mean / 3.6884 / 3.8615 / 3.4793 / 3.4129 / 3.3033 / 3.6703

Table 4: Students’ Perception by Semester of Studies

Semester / Lecturer / course / module / IT / Physical / Service
1-2 / Mean / 4.0159 / 3.9423 / 3.7142 / 3.6438 / 3.6125 / 3.7313
3-4 / Mean / 3.7670 / 3.9145 / 3.6122 / 3.4299 / 3.3986 / 3.6158
5-6 / Mean / 3.8248 / 3.9462 / 3.6748 / 3.4509 / 3.3659 / 3.6408
7-8 / Mean / 3.7124 / 3.8389 / 3.6146 / 3.4522 / 3.4413 / 3.7820
>9 / Mean / 3.4400 / 3.6154 / 3.4200 / 3.2000 / 2.9600 / 3.6000

Table 4 showsstudents’ perception by semester of study. Regardless of which semester they were, respondents indicated above average mean scores for all aspects of distance learning program. It is interesting to highlight that junior students seemed to perceive the quality of distance learning program better as compared to their senior in semester 9. Again this may be to higher expectation of senior students as compared to their junior. All students, irrespective of their semester of studies ranked course content higher as compared to other factors. The high quality syllabus offered by UiTM, could explain this phenomenon.

Table 5: Students’ Perception by Income

income / Lecturer / course / module / IT / Physical / Service
<1000
1001-2000
2001-3500
3501-5000
>5000 / 3.7618 / 3.8601 / 3.6907 / 3.4241 / 3.3773 / 3.5410
3.7995 / 3.8853 / 3.6111 / 3.4757 / 3.4714 / 3.6962
3.7812 / 3.9678 / 3.6793 / 3.4427 / 3.3284 / 3.7395
3.7412 / 3.9690 / 3.4500 / 3.5364 / 3.3455 / 3.6673
3.7778 / 3.8547 / 3.6667 / 3.3889 / 3.3111 / 3.9011

Table 5 shows the students’ perception by income. The students from various income backgrounds showed average perception towards all the six aspects of distance learning program. The lower income students perceived better towards physical facilities as compared to higher income students. The higher income would have higher expectation as compared to lower income group that willing to accept lower quality services.

Table 6: Students’ Perception by Occupations

occupation / Lecturer / course / module / IT / Physical / Service
Public sector
private sector
own business
not working
others / 3.8186 / 3.8907 / 3.6880 / 3.4643 / 3.4569 / 3.7021
3.7664 / 3.8814 / 3.5504 / 3.4195 / 3.3413 / 3.6221
3.5950 / 4.1703 / 3.7929 / 3.5429 / 3.3857 / 3.8255
3.9068 / 3.9316 / 3.9667 / 3.7444 / 3.7333 / 3.9259
3.7944 / 3.8718 / 3.7167 / 3.7833 / 3.9000 / 3.9333

Irrespective of their type of occupations, the students showed above average mean score for every service offered to them. This indicates that they are satisfied with the service provided by InED.

The above findings showed that physical facilities were ranked lower as compared to other factors. This implies that, in order to improve image, number of enrollment and meeting learner’s expectation, InED needs to carefully investigate the classroom condition, teaching facilities, library facilities and other physical facilities. One of the factors contribute to this poor physical facilities perhaps is due to unavailability of own building.

It is important to highlight that respondents agreed that the course contents is high quality as they strongly felt that the course content is able to enhance their skills at the work place. This could be the quality of syllabus as UiTM keeps reviewing the syllabus to meet current needs of workforce market.

In general, respondents agreed that the e-learning program is at satisfactory level as the lowest means score is 3.3413. Likert scale of 1 indicate highly not satisfied and 5 for strongly dissatisfied. Score of 3 point or above indicate that respondents are satisfied with the services.

Differences in Perception between genders, program of studies, and semester of studies

Table 7: Differences in Perception between genders, program of studies, and semester of studies

Sig.(2 tailed) / Lecturer / Course / Module / IT / Physical Facilities / Service
Gender
(t-test) / .161 / .084 / .060 / .190 / .630 / .425
Program of studies (t-test) / .015 / .370 / .003 / .423 / .048 / .899
Semester of Studies ( ANOVA) / .015 / .388 / .825 / .502 / .246 / .407

Table 8: Multiple Comparisons Tukey HSD

Dependent Variable / (I) semester / (J) semester / Mean Difference (I-J) / Std. Error / Sig.
Lecturer / 1-2 / 3-4 / .24886 / .09763 / .083
5-6 / .19103 / .10131 / .327
7-8 / .30342(*) / .09975 / .021
>9 / .57587 / .23374 / .102
3-4 / 1-2 / -.24886 / .09763 / .083
5-6 / -.05782 / .07091 / .926
7-8 / .05456 / .06867 / .932
>9 / .32701 / .22226 / .582
5-6 / 1-2 / -.19103 / .10131 / .327
3-4 / .05782 / .07091 / .926
7-8 / .11238 / .07382 / .549
>9 / .38483 / .22390 / .424
7-8 / 1-2 / -.30342(*) / .09975 / .021
3-4 / -.05456 / .06867 / .932
5-6 / -.11238 / .07382 / .549
>9 / .27245 / .22320 / .739
>9 / 1-2 / -.57587 / .23374 / .102
3-4 / -.32701 / .22226 / .582
5-6 / -.38483 / .22390 / .424
7-8 / -.27245 / .22320 / .739

* The mean difference is significant at the .05 level.

Table 7 shows that male and female students have no significant difference in their perception towards lecturer, course, module, IT, physical and service. The students’ program of studies indicated significant difference in their perception towards lecturer, module and physical facilities but not towards course, IT infrastructure and service.

The ANOVA shows that students from different semester of studies showed no significant differences in their perception towards course, module, IT, Physical facilities, and service except towards lecturer.

The Tukey HSD shows that students between semester 1 - 2 and 7 – 8 have significant difference in their perception towards lecturer. The students from other semester have not showed any significant difference.

Ccorrelation between students’ perception and genders, program of studies, semester of studies, CGPA and income.

Table 9: Results of Spearman’s Rho

Spearman's rho / Sig (2 tailed)
Gender / Perception / .252
Program of Studies / .038
income / .877
Semester of Studies
CGPA / .176
.023

The result of Spearman’s Rho reveals that there is no significant relationship between students’ gender, income and semester of studies; and perception. However there aresignificant relationship between students’ program of studies and CGPA;and perception. The result shows that students’ perception depends on their program of studies and CGPA. However the Spearman’s rho does not provide which program and CGPA level have significant relationship to students’ perception.

The students’ perception towards distance program was found to have negative relationship to the students’ CGPA. This indicates that the higher achiever students lower perception towards the distance program.

Conclusion

From the findings of the study three conclusions can be made that are:-

1. Generally the students of InED perceive the e-learning program above average standard.
2. Students’ perception independent of gender, income and semester of studies.
3. Students’ perception depend on their CGPA

1. Generally the students of InED perceive the e-learning program at above average standard

Obviously all the six elements of the e-learning program rated average by all the respondents. Adult learners as been mentioned by Roger (1981) go back to school or university with high expectation. They expect everything is to be taught by the lecturers or everything is nicely in place. InED with all the limitations such as lack of facilities, like class room, library, computer labs, etc just couldn’t provide all the students’ expectation as they expected. As consequences InED receives a lot of complaints from the students with regards to the improper facilities set up. This could be one of the reasons why the students rated slightly above average of InED e-learning program.

2.Students’ perception independent of gender, income and semester of studies.

Though gender always been claimed to differentiate between male and female perception on something, but the study found that it did not in line with the claim. This could be the result of an equality of opportunity or treatment by the Malaysian government to the women. In Malaysia gender is not a factor of discrimination in education as in some developing countries. Thus, they have freedom to evaluate things as the male do.

The students from different income background too did not show any significant difference in their perception. It means that the six aspects of distance learning program have been genuinely evaluated based on academic perspective regardless of socio-economy background.

Seniority as a student also did not indicate any difference in their perception as compared to the junior students. Once again the evaluation was done purely based on academic and quality perspective.

3.Students’ perception depends on their CGPA level

Unlike gender, income and semester of studies, the students’ CGPA level did influence the students’ perception. It means the students’ academic achievement differentiates their perceptions. The high achievers showed higher expectation towards the programs as compared to the lower achievers. Logically, the smart students expect more than less smart students.

Recommendations

Based on the study, several recommendations can be made as follow:-

1. Recommendation for practice
2. Facilities such as classroom, students centers, quality of module, support service and even the commitment on the part of teaching need to look upon.
3. To improve the Learning Management System (LMS) environment to provide more structure and guidance to students in learning from interaction and group activities. Further research and development are needed to refine the LMS to support and enhance students’ learning using a problem-based model.

2. Recommendation for further research

a.To use multiple methods of study such as qualitative and

quantitative research to get genuine input.

1. To involve all students from branches in order to get representative

findings.

REFERENCES

Abdullah, Szarina. (2003). Understanding Adult Learners. Proceeding on the "Seminar Memahami & Memudahcara Pelajar-Pelajar dewasa e-PJJ UiTM". Kuala Lumpur

Bourne, J. R., McMaster, E., Rieger, J., & Campbell, J. O. (1998). Paradigms for online learning: a case study in the design andimplementation of an asynchronous learning networks course,

Carswell, L. (2000). Distance education via the Internet: The student experience. British Journal of Educational Technology, 31 (1), 29-46.

Collins, M. (2000).Comparing Web, correspondence and lecture versions of a second-year non-major biology course. British Journal of Educational Technology, 31 (1), 21- 27.

Corrent-Agostinho, S., Hedberg, J., & Lefoe, G. (1998).Constructing problems in a Web- based environment. Educational Media International, 35 (3), 173- 180.