A Staff Development Evaluation: Changing the Platform from a Mac to a PC Windows NT Environment
Heather Kinnear and Leslie Caul, Stranmillis University College, Belfast, Northern Ireland
Paper presented to the British Educational Research Association Annual Conference, University of Leeds, 13-15 September 2001
SUMMARY
This paper presents an evaluation of staff development in a Teacher Education Institution in the United Kingdom exploring the effectiveness of changing an existing Mac platform to a PC Windows NT environment. It incorporates how confidence, competence and use of ICT in teaching of all academic staff sampled, progressed during the changeover period. The methodology consisted of a baseline audit which was carried out to assess staff skills before the changeover began (Time 1), an attitudinal questionnaire administered half way through the changeover (Time 2) and a repeat of the original audit to monitor progress made after the changeover was completed (Time 3).
From Time 1 to Time 3 staff showed an overall decline in their levels of confidence with regard to general IT skills, word processing, databases, spreadsheets and core skills. Using a regression analysis it was found that the use of ICT in teaching and the attitudes of staff to ICT predicted 78.8% of the variation in baseline skill levels assessed when the changeover had been completed. A one way ANOVA found that staff in PE, History, English, Drama, Maths, Computing and ICT, Education, Geography, Business Studies and Early Years showed a significant increase in their levels of confidence and use of ICT in teaching as skill levels progressed between Time1 and Time 3. It was also found that staff from English, History, Education, Geography, Business Studies, Computing and ICT, PE, Maths and Early Years increased in terms of competence. Staff from RE, Science and Technology and Design showed a significant decrease in all 3 areas during the changeover period.
INTRODUCTION
Stranmillis University College has had since the 1980’s a partial network across its campus carrying a Macintosh environment. Developments within the University sector, the demise of the Macintosh computer and the future development of a PC environment led the University College in 1998 to change to a PC platform and to develop a complete network across the College. This change involved the development of an institutional plan, which aimed to provide access to ICT for all members of the College community and to promote the widespread use of ICT as the core communications framework. The College planned to provide each member of staff with an on-desk PC and to have a student computer ratio of 5 to 1 in each of the 7 open access areas. The staff development programme put in place was based on the intrinsic motivation of receiving a new computer. Consequently, staff had to attend a ½ day training course (on campus) and agree to complete 5 online training modules (Netg) in 1 year (by September 2001) with reference to the online training pack provided to each member of staff. These included MS Internet Explorer, MS PowerPoint 97, MS Windows NT 4.0 User Fundamentals, MS Word 97 Expert User and MS Word 97 Proficient User. “Netg” also covered 15 other modules including training in MS Access, MS Outlook, MS Excel, MS Office and ECDL (European Computer Driving License) modules and costs the University College approximately £6,000 per year to maintain across individual PCs. All developments to date have cost the University College approximately £200,000.
To aid with the completion of these modules, each academic member of staff was assigned a mentor, who was a more experienced and more advanced user of ICT. Their role was to monitor usage of “netg” and to offer support and guidance where necessary. Their role was one of facilitator. Each mentor had responsibility for between 6 and 8 individuals. During this stage it was the intention that the mentoring system would highlight areas where staff needed further training not sufficiently addressed by Netg.
It is clear that ICT provides access to rich sources of information in all areas of the curriculum well beyond each person’s relatively limited subject expertise. The technology is changing too fast to be able to operate as a skilled demonstrator of state of the art ICT facilities. Models of “good practice” in using ICT to promote learning have yet to be generated.
BACKGROUND
Stranmillis University College is one of two teacher Education Institutions in Belfast, Northern Ireland. It offers an exciting range of undergraduate programmes (BA Early Childhood Studies, BSc Health and Leisure Studies, BEd Primary and Secondary courses) and postgraduate programmes (PGCE Early Years, PGCE Educational Psychology and various MEd, MSc and MA courses). Currently, there are 61 academic teaching staff, 3 academic research staff, 17 administration staff, 5 IT staff, 4 learning support services staff and 7 library staff, together catering for the needs of approximately 1100 students enrolled in the College. The University College has a purpose of promoting excellence in the education of its students through teaching, scholarship and research of the highest quality, to contribute positively through the work of its student teachers to the educational experience of pupils and to help meet the professional needs of the community through a range of diversified programmes. In fact, higher education in the United Kingdom has one of the most technologically advanced networks in the world. In 1997, Professor John Arbuthnott (Chairman of the Dearing IT Working Group) suggested that most Higher Education Institutions are in “mid-leap”. ICT can be used to support the development of educational materials, support of teaching and learning, organisation of educational programmes, research, quality assurance, management and administration.
It has been suggested that the “the creation and use of information lie at the heart of a Higher Education Institution’s core functions of teaching and learning and of research” (JISC: Guidelines for developing information strategy, December, 1995).
Conducive to the present study was a study carried out by Murphy & Greenwood (1998) who looked at the main obstacles that limit the successful implementation of ICT for student teachers in the UK and strategies for overcoming these. Of interest to the present study it was found that ICT is significantly under-used by students and beginning teachers. Many explanations are offered for this including those of age, gender, age phase for which the students are being trained to teach and the subject specialism of the student teachers.
Another study related to the present study is that by Simpson et al (1997). This study offers an insight into how far teacher education institution lecturers have incorporated ICT into their own teaching and promoted it as a classroom teaching tool. It was perceived that the levels of ICT skills of students on entry to higher education are steadily rising (Lienard, 1995). This shows that there is a rapid advancement of use of computers at home and in schools. It was concluded that teacher education institution tutors must be more than routine, basic ICT users; they must be specialist educators who can make students secure in the range of pedagogical uses soon to be required of beginning teachers.
Dearing recognised that the exploitation of these information and communication technologies holds out much promise for improving the quality, flexibility and effectiveness of higher education (Dearing, 1997). The Dearing Committee envisage that ICT will overcome barriers to higher education, providing improved access and increased effectiveness, particularly in terms of life-long learning. Classes in a particular time will be replaced by access to resources that are time and place independent.
Over the last few years, there has been a paradigm shift in education, from teaching to learning. The role of teacher is changing to become more of a facilitator of learning rather than an expositor – students are encouraged to take much more responsibility and control over their own learning in an active way rather than in the traditional passive way. In fact, the European Commission (1996) formally recognised that the learning process must become more learner oriented, whereby the learner is actively acquiring knowledge and skills using not only the conventional tools such as books and teachers, but new technology based tools that enable learning at a distance and life-long learning.
Staff development involves 2 levels: first, understanding the technology and second, understanding and implementing the pedagogy. The relationship between technology and pedagogy is a complex and fascinating one. The interplay offers both possibilities and choices. Each use of a medium or product involves a pedagogical choice. Each medium of teaching and learning opens up certain pedagogical possibilities.
Teachers learn as they teach. They are no longer the prime possessors, controllers and transmitters of knowledge, but if they have a sound grasp of the principles of pedagogy and the processes of learning, they can plan educational contexts within which young people can engage innovatively with ICT to achieve valued educational goals. The development of ICT skills by educators will then result from, not pre-date, seeing the pedagogical power of the new technology.
In terms of the Stranmillis University College Policy for Information Technology, it is necessary to ascertain exactly what stage the College has reached with regard to its aims and objectives. In its mission statement; Stranmillis University College outlined 3 aims, which have all been accomplished. These were to provide access to computing and IT services for all members of the College community, in compliance with international standards wherever possible, and to equip them with the skills required to exploit those services, to promote the widespread use of IT in the educational process as part of a commitment to providing a high-quality learning experience and to provide IT based systems to underpin efficient and effective management and administration across the whole of the College. In addition, a student:computer ratio of 1:7 has been achieved in the open access areas. Plans are still under way to increase this facility to 1:5.
METHODOLOGY
The main investigation was conducted as 3 separate and semi-independent strands concerned with the range of ICT-related skills deployed by academic staff in Stranmillis University College, their attitudes towards ICT and their perceptions and use of ICT in teacher education. The data reported here were gathered using the following techniques:
1A Baseline Audit (April – May 1998) devised in-house to establish the level of ICT skills amongst academic staff. This was split into 3 separate sections with A devoted to general skills in different areas of computing, B devoted to the use of ICT in teaching and C devoted to the level of lecturer competence in ICT.
2A Questionnaire distributed to all teaching staff (September – October 2000) to establish attitudes to the use of ICT and perceptions of ICT in teaching and learning devised by the University of Strathclyde for a study of ICT in Teacher Education (1997).
3A repeat of the Baseline Audit (February – March 2001) to ascertain improvements in the level of skills amongst academic staff.
The Baseline Audit containing a significant number of statements was designed to ascertain the skill levels of academic staff in 9 areas of ICT usage. These included general ICT skills, word processing, databases, spreadsheets, internet, presentation software, core skills, use of ICT tools in own subject area and ICT tutor competence. The responses to each statement reflected whether staff felt they could carry out the task confidently, whether they though they could do with an update or whether they needed to learn the task from scratch. This audit was carried out at the beginning of the introduction of PCs in order to develop a baseline measure of skills.
The Questionnaires contained a number of open-ended questions for the purposes of ascertaining the attitudes of academic staff towards the use of ICT and their perceptions of ICT in teacher education. Responses were rated on a 5-point Likert scale ranging from “Strongly agree” to “Strongly disagree” and scored accordingly. This instrument was devised by the University of Strathclyde and adopted for the purposes of this study.
In addition, a number of semi-structured interviews were carried out in October/November 2000 to ascertain specifically when and how academic staff use ICT in their own subject area of teaching. The sample included 9 teaching staff of which 2 were newly appointed, 1 senior member of staff, 3 mentors and the Head of IT services in the College. These 9 teaching staff, 1 senior staff and 3 mentors were identified from the original baseline audit as possessing above average skills in relation to the use of ICT in their teaching (section B). As a result, the decision to interview these members of staff would provide a few case studies which could be examined in detail and subsequently related to staff on the whole in Stranmillis University College. The spread of subject interests and the focus of teaching was comprehensive and, coupled with the age and gender profile of the College population, suggested that we had a representative cross-section of academic staff.
It was identified that there were a total of 4 research questions. These were as follows:
1Controlling for baseline skill levels at Time 1, does the use of ICT in teaching and the attitudes of staff to ICT at Time 2 predict skill levels of staff as measured at Time 3?
2Is there a significant increase in the staff competence, confidence and the use of ICT in teaching when skill levels progress from Time 1 to Time 3? How is this increase related to the staff development programme that has taken place between Time 1 and Time 3?
3What is the strength of the association between staff competence and staff confidence both in the baseline audit (Time 1), the attitudinal questionnaire (Time 2) and the repeated baseline audit (Time 3)?
4What are the characteristics of staff approaches to self-directed learning (Netg)?
RESULTS
Staff in IT services within Stranmillis University College were responsible for monitoring the use that staff made of the Netg online training. Within this package there were many modules on offer of which academic staff had to complete 5 within 1 year. These were MS Internet explorer (completed by 1 Research staff), MS PowerPoint 97 (completed by 3 staff from Computing and ICT, Education and Geography), MS Windows NT 4.0 User Fundamentals (completed by 1 Research staff and attempted by 10 academic staff from History, PE, RE, Maths, Education, Art and Research), MS Word 97 Expert User (completed by 1 Research staff and attempted by 1 member of the Geography staff), and MS Word 97 Proficient User (completed by 1 Research staff and attempted by 3 staff from the Education, Art and Geography departments). This was continuously monitored although it was not possible to select individuals who had done a lot or individuals who had done very little. The only categories that were utilised were subject areas. User rates were taken definitively in February 2001 when the Baseline Audit was repeated.
The results from the original and repeated baseline audits do not show an increase in overall skill levels among academic staff in the college. When assessed at Time 3, staff reported less confidence in general IT skills, word processing, databases, spreadsheets and core skills. They did, however, appear to be more confident in using the internet, presentation software, ICT in their own subject areas and also in their subjective level of competence.
A correlational analysis using the non-parametric Spearmans rho test as the data was categorical between staff competence and confidence in the original baseline audit was found to be r = 0.967, in the questionnaire was found to be –0.444 and in the repeated baseline audit was found to be 0.892. This shows very high, positive, linear relationships in both audits. However, the correlation in the attitudinal questionnaire suggests that if staff are not confident they will not perform their work competently with regard to ICT. Reliability in the form of Cronbach’s Alpha was found to be 0.842 for all sections of the Audit together and 0.561 for the attitudinal questionnaire showing very high levels of internal consistency. This was regarded as being highly internally reliable by Bryman & Cramer (1997).
The Questionnaire results were directly compared to the results obtained by the University of Strathclyde who devised the attitudinal questionnaire. The results indicate that most of the academic staff have strong positive attitudes to ICT and its use in teacher education. However, there were significant signs of insecurity; only 64% of staff felt that they would be able to keep up with future developments and 26% felt that ICT skills should be taught by a specialist lecturer. Although the majority of academic staff (95%) indicated that the use of ICT gave insights into new learning environments and thought it was going to be essential for continuing professional development, and 88% felt that it enhanced and enriched courses or programmes of work, for about one quarter (24%), it had not reduced the time required to cover content in their lectures. The majority felt that ICT had improved the quality of the content of the students’ coursework or assignments and 78% felt that it improved the quality of the content of the curriculum materials students produce for their pupils. These results do not differ markedly from those obtained by the University of Strathclyde.