ECER 2004 Network 12Educational researchers use of information services on the web
Educational researchers' use of information services on the World Wide Web: a follow-up report on the PERINE survey of educational researchers in 8 European nations 2001-2004
Sam Saunders
British Education Index, Brotherton Library, University of Leeds, UK
Translation, data gathering and coding for national surveys was co-ordinated by Anita Monty, Elfriede Tajalli, Marisa Trigari, Alexander Botte, Peter Meyer, Katalin Varga and Gintare Tautkeviciene
Abstract
In the context of a Framework Five Research Infrastructure project, funded by the European Commission, eight partners from the Information Centres and Libraries in Educational Research Network of the European Educational Research Association distributed questionnaires between 2001 and 2002. These questionnaires sought information from educational researchers and related education professionals about their use of and attitudes towards the internet as an information medium in the course of their work. The questionnaires were distributed by post and on-line on an opportunistic basis, using the national languages and networks of each project partner. Direct questions were asked about which national and international resources were being used in relation to seeking and disseminating work related information. For 2001 596 complete questionnaires were received from UK and Danish respondents and in 2002 there were a further 975 questionnaires from German, Hungarian, UK, Swiss, Austrian, Italian and Lithuanian respondents. The results showed that among those who responded to the questionnaire, two thirds were daily users of the internet and that a majority had positive views about its value to their work, both now and in the future. Three quarters of respondents agreed that the internet had already transformed their work and 70% agreed that further transformation was likely. Respondents in all eight nations reported much heavier use of their native resources than they did for international resources or resources from other European nations. They also indicated that information retrieval was both more frequent and more successful. Dissemination of results and forging contacts with other researchers were much less frequently mentioned. This paper presents updated statistics following a final data gathering phase during 2004. While levels of successful use of the internet showed further increases in 2004, one interpretation of the data on researcher attitudes suggests a decreased level in confidence since 2002. In the light of the data and of the PERINE project in general, the paper discusses the need for a European information structure and the contribution to such a structure that the PERINE project might make.
Introduction
Partners from eight European nations have been working together on the PERINE project since 2001. The acronym PERINE stands for Pedagogical and Educational Research Information Network for Europe. It expresses a concept that emerged from Network 12 of the European Educational Research Association and which has been supported as a practical project since 2001 by the European Commission under its Access to Research Infrastructures activity within the Improving Human Research Potential and the Socio-economic Knowledge Base programme of Framework 5. One part of PERINE's work has been to gather data in each of eight partner countries on levels and patterns of use of internet information services and of educational researchers’ attitudes towards the medium. Reports on some of that work have been presented at previous EERA conferences (Larsen and Saunders 2001; Saunders, Tajalli and Monty 2003).
The original questions were simple. How are educational researchers using the internet in the conduct of their work, and what demands are they making of it? These practical questions were closely linked to services that PERINE partners were providing, and to their attempts to improve the information infrastructures that served both national interests and the interests of an emerging European educational research space. Starting with the general approach of a questionnaire written by Sabine Manning (Manning 2000) an evolving sequence of questions have been presented to a number of groups in the eight countries over a three year period. The general pattern revealed up to 2002 was of widespread and increasing internet use coupled with a broadly optimistic view of the developments in train. It was noted (Saunders, Tajalli and Monty 2002) that information retrieval was more universally adopted than information dissemination, and that making contacts and conducting collaborative work were much less commonly reported as internet activities. The need for a more qualitative understanding of the complexities of internet adoption was noted.
The aim of the present study was to repeat the data collection in the early part of 2004 in order to make comparisons that spanned the lifetime of the PERINE project. While no causality could be attributed, a measure could be made of the extent to which PERINE was moving with or against a general trend. Significant discrepancies or congruencies could be noted and used to guide future plans.
Data Collection
During the first half of 2004 eight versions of the same questionnaire were distributed by means and to target groups selected by each PERINE member. The framework of questions was retained from earlier versions and a small number of new questions were added. In particular, use of the internet in e-learning was investigated for the first time. On-line and printed versions were used, and in some cases questionnaire completion was done in the presence of the researcher and other respondents. 822 responses were collected for analysis. A copy of the English language questionnaire, distributed by post in the UK is appended. (Appendix 2)
Problems of generalisability
With the same questionnaire in seven languages, with no consistency of sampling strategy and with a variety of methods for distribution there has been no question of adopting parametric statistical analysis of the results. Any claim that the patterns within them have a general relevance are based on the fairly large numbers involved and on pragmatic comparisons of different sub groups within the results. In the three waves of data collection, a total of 2,400 sets of responses have been collected. The descriptions derived from this survey cannot be seen as scientific generalisations about European educational researchers per se, but they do serve as a pragmatic contribution to better informed decision making than would have been possible through intuition alone. Some consideration can also be given to comparable results from other studies. In particular, research for the UK Professional Associations Research Network by Friedman and Senior (2001) found a broadly similar pattern of level and pattern of use among UK professionals in general to the one described in the 2002 report. Within the surveyed group, some comparisons across groups or over time have been made on the basis of chi-square tests that do not depend on broader population characteristics being known.
Findings
Frequency
In most cases, findings from the 2004 survey of all eight countries will be presented in a series with data gathered in 2001 (UK and Denmark only) and 2002 (seven countries). The total numbers of respondents in each case are as shown in Table 1. Individual tables will not always match these full totals, as not every respondent answered all questions. Percentages are generally expressed in relation to the number of answers for a given question.
Table 1 Numbers of respondents in each year from each country
Year2001 / 2002 / 2004 / Totals
Germany / 172 / 101 / 273
Hungary / 139 / 52 / 191
UK / 187 / 121 / 57 / 365
Switzerland / 171 / 146 / 317
Austria / 71 / 81 / 152
Italy / 180 / 197 / 377
Lithuania / 128 / 148 / 276
Denmark / 409 / 40 / 449
Totals / 596 / 982 / 822 / 2400
The proportion of respondents reporting the frequency of their internet use as being daily has gone up over the period. Where 65% called themselves daily users in the 2002 sample, 2004's figure is 83%. The proportion of non-users has gone from just under one in a hundred to one in eight hundred and twenty. Daily use is the norm. Table 2 shows that reports of occasional or rare use have declined proportionately. Checking for differences between older researchers in senior positions on the one hand and younger, more junior researchers on the other reveals a similar pattern of change in each group. Tables 3 and 4 present data for these two specially selected sub groups. They are respondents who report research as a major part of their work and who are either aged under 35 in junior positions (the junior researchers) or aged over 50 and in senior positions (the senior researchers). It will be noted that the junior researcher group has a higher figure for daily use then the seniors. (94% rather than 83%)
Table 2 Reported frequency of internet use by year: all respondents
Year / Total2001 / 2002 / 2004
Internet use / on a daily basis / Count / 406 / 632 / 681 / 1719
% / 68.5% / 65.1% / 83.0% / 72.1%
occasional / Count / 152 / 268 / 123 / 543
% / 25.6% / 27.6% / 15.0% / 22.8%
rare / Count / 35 / 62 / 15 / 112
% / 5.9% / 6.4% / 1.8% / 4.7%
non-existent / Count / 9 / 1 / 10
% / 0.9% / 0.1% / 0.4%
Total / Count / 593 / 971 / 820 / 2384
% / 100.0% / 100.0% / 100.0% / 100.0%
Table 3 Reported frequency of internet use by year: older senior researchers
Year / Total2001 / 2002 / 2004
Internet use / on a daily basis / Count / 66 / 91 / 94 / 251
% / 60.6% / 60.3% / 83.2% / 67.3%
occasional / Count / 31 / 45 / 17 / 93
% / 28.4% / 29.8% / 15.0% / 24.9%
rare / Count / 12 / 12 / 2 / 26
% / 11.0% / 7.9% / 1.8% / 7.0%
non-existent / Count / 3 / 3
% / 2.0% / .8%
Total / Count / 109 / 151 / 113 / 373
Table 4 Reported frequency of internet use by year: younger junior researchers
Year / Total2001 / 2002 / 2004
Internet use / on a daily basis / Count / 33 / 47 / 60 / 140
% / 73.3% / 77.0% / 93.8% / 82.4%
occasional / Count / 9 / 13 / 4 / 26
% / 20.0% / 21.3% / 6.3% / 15.3%
rare / Count / 3 / 1 / 4
% / 6.7% / 1.6% / 2.3%
Total / Count / 45 / 61 / 64 / 170
% / 100.0% / 100.0% / 100.0% / 100.0%
Competence and Confidence
Respondents were asked for an estimate of their own competence level. They were also asked to respond to a number of opinion statements about using the internet. The 2004 results suggest that an increase in use and self-perceived competence has been accompanied by a reduced level of confidence in the internet's efficacy. Table 5 summarises the increased levels of reported competence. Where 63% of respondents in 2002 chose the "expert" or "capable" categories, the figure for 2004 was up to 75%. (rounded totals). A chi square test on this distribution confirms that such large increases are probably the result of something other than chance. (p=<0.01)
Table 5 Internet competence by year of completion: all respondents
Year / Total2001 / 2002 / 2004
Internet competence / expert / Count / 76 / 136 / 149 / 361
% / 12.8% / 14.0% / 18.3% / 15.2%
capable user / Count / 305 / 475 / 458 / 1238
% / 51.5% / 48.8% / 56.3% / 52.0%
user / Count / 171 / 293 / 190 / 654
% / 28.9% / 30.1% / 23.3% / 27.5%
novice / Count / 40 / 69 / 17 / 126
% / 6.8% / 7.1% / 2.1% / 5.3%
Total / Count / 592 / 973 / 814 / 2379
% / 100.0% / 100.0% / 100.0% / 100.0%
A short set of opinion statements invited respondents to agree or disagree, strongly or not, in each case. These opinions were drawn from frequently heard comments about internet use in the years before our first survey. The value of advanced search facilities, the value of simple search facilities, the volume of information, the quality of information, the difficulty of finding what was needed, and the potential of the internet for disseminating research were the dimensions addressed. Rather than treat each factor as an equal contribution in describing confidence and calculating a single "confidence" score, a hierarchical cluster technique (Ward's method using a squared Euclidian distance calculation) was used to identify sets of respondents with patterns of response that differentiated them from other groups. The most distinct groups were visible at the two cluster level, with patterns of response that can be readily interpreted as delineating a more confident group and a less confident group. The confident group tended to value advanced search facilities, mistrust simple searches, regard the internet as an environment where problems of quality and volume are not too great, which has potential for research dissemination and where things can be readily found. Members of the less confident group tend towards agreement with an opposite set of opinion statements. Table 6 shows that the confident group in 2004 was proportionately smaller in 2004 than the ones in 2002 or 2001. The same pattern pertained in the whole sample and in the two comparator groups of older senior researchers and younger junior researchers. Chi-square tests again indicated a very low probability that such differences were random outcomes. (p=<0.01>.
Table 6 Attitude to the internet by year of completion: all respondents
Year / Total2001 / 2002 / 2004
Attitude to the internet / broadly more confident / Count / 313 / 519 / 399 / 1231
% / 63.7% / 59.9% / 54.8% / 59.0%
broadly less confident / Count / 178 / 347 / 329 / 854
% / 36.3% / 40.1% / 45.2% / 41.0%
Total / Count / 491 / 866 / 728 / 2085
% / 100.0% / 100.0% / 100.0% / 100.0%
Specific uses
In the last report (Saunders, Tajalli and Monty 2002) a distinct pattern of researchers' successful use of the internet emerged. Uses connected with simple information retrieval were at the top of the ranking and more complex purposes to do with collaborative working and network creating were at the bottom. The same list of 17 items (based on Manning's original set of purposes: Manning, 2000) was presented in each year and respondents were asked whether they had used the internet "with some success", "with little or no success" or "not at all" for each of the purposes given. A summary of the results is presented in Appendix 1 as Table 7. Table 7 shows aggregate results from each year and all groups. It shows that over two thirds of all the researches have had some success in booking travel, finding information about conferences, finding news, getting information about institutions and finding resources for research, with three quarters reporting success in finding resources for research. At the bottom of the ranking, finding expertise, publishing results, sharing results in partnership, finding material for administration and finding research partners all feature successfully for less than a third of respondents. More than a half report that the internet had not been used at all for the last two items. As shown in Table 8 almost all purposes attracted more "with some success" responses in 2004 than in 2002 or 2001, a result that is congruent with the suggestion drawn from Table 2 that more use in general is being made of the internet.
Table 8 Use of the internet "with some success" for specified purposes by year of completion
Year2001 / 2002 / 2004
Finding sources for research / Count / 424 / 716 / 650
% / 73.2% / 74.5% / 80.0%
Getting information about institutions / Count / 420 / 703 / 626
% / 72.2% / 73.1% / 77.5%
Finding conference information / Count / 404 / 607 / 597
% / 69.3% / 63.2% / 73.4%
Finding news / Count / 389 / 610 / 576
% / 67.0% / 63.9% / 70.8%
Arranging travel / Count / 422 / 561 / 572
% / 72.4% / 58.9% / 70.3%
Getting information about researchers / Count / 401 / 584 / 555
% / 68.8% / 61.1% / 68.8%
Finding material for teaching / Count / 288 / 555 / 541
% / 50.2% / 58.4% / 67.2%
Getting information about courses / Count / 91 / 446 / 447
% / 49.7% / 46.8% / 55.7%
Getting information about networks / Count / 269 / 448 / 432
% / 47.0% / 46.8% / 53.7%
Getting information about other professionals / Count / 123 / 407 / 428
% / 67.2% / 42.9% / 53.3%
Finding project management sources / Count / 177 / 332 / 345
% / 30.8% / 35.0% / 42.9%
Other publishing / Count / 146 / 332 / 338
% / 25.6% / 34.8% / 42.4%
Finding expertise / Count / 178 / 279 / 280
% / 30.9% / 29.3% / 34.8%
Publishing results / Count / 140 / 250 / 277
% / 24.5% / 26.3% / 34.2%
Sharing results in partnership / Count / 130 / 258 / 272
% / 22.9% / 27.2% / 33.8%
Finding material for administration / Count / 120 / 226 / 257
% / 20.9% / 24.0% / 32.0%
Finding research partners / Count / 86 / 138 / 160
% / 15.1% / 14.6% / 19.8%
International and European dimensions
In gathering data from respondents about whether particular information services have been used through the internet, some indication might be given of the levels of interest in educational research resources that are European or International rather than domestic. ERIC, a US resource, and EURYDICE, a Europe-wide service, were included as choices in each of the eight national surveys (which, individually, also asked about levels of use of their own domestic services) Table 9 shows the patterns of reported use for these two services across the national surveys in each year compared with use of a national bibliographic service and use of a government web site. In each year, the use of the national service is much more frequent than the use of the Europe-wide or US service. More significantly, however, the reported use of the European service does increase over the two year period. The percentages for adoption of EURYDICE and ERIC are show in Figures 1 and 2, where the scale of the difference is more evident.
Table 9 Reported use of ERIC and EURYDICE by year, compared with use of a national bibliographic service and a national government web site.
more than once / once / neverCount / % / Count / % / Count / %
Year / 2002 / Use of ERIC / 273 / 29.0% / 166 / 17.6% / 502 / 53.3%
Use of EURYDICE / 135 / 14.6% / 210 / 22.7% / 582 / 62.8%
Use of a national bibliographic service / 185 / 53.8% / 58 / 16.9% / 101 / 29.4%
Use of national government web site / 519 / 55.4% / 218 / 23.3% / 200 / 21.3%
2004 / Use of ERIC / 210 / 27.3% / 146 / 19.0% / 412 / 53.6%
Use of EURYDICE / 190 / 24.7% / 199 / 25.9% / 379 / 49.3%
Use of a national bibliographic service / 396 / 61.5% / 170 / 26.4% / 78 / 12.1%
Use of national government web site / 654 / 82.8% / 97 / 12.3% / 39 / 4.9%
Figures 1 and 2 present the same data for use of EURYDICE and ERIC, with Figure 3 presenting the contrasting pattern for the national bibliographic services.
Figure 1 Reported use of ERIC by year
Figure 2 Reported use of EUYDICE by year
Figure 3 Reported use of a national bibliographic service by year
Use and potential of the PERINE database
In 2001 and 2002 an opinion question had asked whether respondents would find an organised collection of internet links helpful. While the aims of the PERINE project were to do with strengthening links and improving the infrastructure of information services supporting European educational research, a practical task of building a multilingual database of European resources for educational research had been planned as an important way of helping to realise those aims. Work on the database focussed on procedures and designs rather than extensive content, but a demonstration version of the database was presented at ECER 2003 and remained available for trial use and comment through the rest of 2003 and 2004. An opportunity for researchers to interrogate an organised and catalogued collection of internet resource descriptions in any of eleven European languages, using the multi-lingual European Educational Thesaurus was construed as a means to provide the sort of help that Table 10 shows was thought to be helpful by over 90% of respondents (53% strongly agreeing and 41% agreeing: figures rounded).
The 2004 survey asked if respondents had used the PERINE database, and whether they had done so or not, if they thought the idea was potentially useful. A minority of the 774 responses to the question about using PERINE (32%) reported one or more uses of the database, but a large majority of those users believed it would be useful. Among those who had used it more than once 71% believed it would be useful. Only 15 respondents believed it would not be useful, 12 of whom reported not having used it. It might be emphasised that the demonstration database has had only a very basic interface which indicated the provisional nature of the collection and did not match the sophistication or ease of use that PERINE partners would provide for their own national services.
Table 10 Year of completion by "an organised collection of education links would help"
"An organised collection of education links would help" / Totalstrongly agree / agree / disagree / strongly disagree
Year / 2001 / Count / 259 / 271 / 39 / 5 / 574
% / 45.1% / 47.2% / 6.8% / 0.9% / 100.0%
2002 / Count / 521 / 372 / 50 / 7 / 950
% / 54.8% / 39.2% / 5.3% / 0.7% / 100.0%
2004 / Count / 454 / 308 / 42 / 3 / 807
% / 56.3% / 38.2% / 5.2% / 0.4% / 100.0%
Total / Count / 1234 / 951 / 131 / 15 / 2331
% / 52.9% / 40.8% / 5.6% / 0.6% / 100.0%
Table 11 Utility of PERINE by use of PERINE
Use of PERINE / Totalmore than once / once / never
Utility of PERINE / useful / Count / 48 / 102 / 152 / 302
% / 71.6% / 56.7% / 28.8% / 39.0%
not useful / Count / 3 / 12 / 15
% / 1.7% / 2.3% / 1.9%
perhaps / Count / 19 / 75 / 363 / 457
% / 28.4% / 41.7% / 68.9% / 59.0%
Total / Count / 67 / 180 / 527 / 774
% / 100.0% / 100.0% / 100.0% / 100.0%
Discussion
There is clearer confirmation in the new data reported for this year that use of the World Wide Web has become a pervasive fact of daily life for the European educational researchers who responded to the questionnaire in the eight PERINE countries: Lithuania, Hungary, Austria, Germany, Italy, Switzerland, Denmark and the UK. It seems unlikely that other European nations would report differently. Younger and older researchers, in junior and in senior positions, tend to see themselves as expert, or at least competent, and regular users of the internet.