Webcef Is an Initiative of a European Consortium of Seven Higher Education Institutions

WebCEF: On-line Collaboration and Oral Assessment within the Common European Framework of Reference

Lut Baten, John Osborne, Yvette D’Silva Hymers

KULeuven, Université de Savoie

As mobility has become a regular feature of learning throughout Europe, a need has grown for commonly recognised benchmarking of language skills. In foreign language learning, the Common European Framework of Reference for Language (CEFR) is increasingly being used not only in the reform of national curricula but also by international consortia for the comparison of language certificates. The CEFR is widely recognised as a useful instrument but the CEFR scales are designed to reflect experienced language teachers’ perceptions of learners’ performance, and however carefully the scales have been formulated, overlaps and ambiguities remain. Consequently, for newcomers to the field and for daily classroom practice, there is a threshold for reliable implementation of the CEFR. Even experienced teachers may feel the need to check how their evaluations compare with those of colleagues in their own or other European countries. Hence, the question arises of how teachers, teacher trainees, and students can be helped to incorporate the CEFR when assessing language proficiency. A few concrete examples will illustrate some of the cases we have in mind.

Case 1: At a language institute, a young teacher would like to test the oral performance of her learners. As she is teaching Dutch as a foreign language, she is the only teacher in her section. How well can she rely on her own expertise as to the scores she gives? If she had access to a community of assessors, another teacher would be able to either sit in on the test or have a look at a recording and send her assessments and annotations.

Case 2: Teacher trainees need good models of what learners can do at a specific level for a specific task in a foreign language, and to be able to consult the descriptors of the CEFR that translate this performance into a commonly recognised scale. They can also learn from each other by comparing their assessments and discussing why and where differences occur. In a spiral movement of observing, listening, assessing, comparing, and observing again, they acquire a better grasp of what the ‘can do’ statements in the CEFR for productive and interactive performance actually mean. In team teaching, teachers may wish to set their standards more objectively by comparing each other’s intuitive assessments.

Case 3: In addition to trainee teachers, other students, especially those seeking an international test certificate, might like to record themselves, and be able to grade each other, not only themselves, as to their performance. Peer assessment has proved a highly motivating tool for raising performance. Students are often not aware of their own mistakes in presentations but are very open to criticism from their peers. In a community of assessments made by a teacher and peers, a balanced survey will give them the right support to become more autonomous and gain awareness.

Case 4: Comparisons across cultures and languages can raise awareness of diversity in assessments of performance levels. A student who is rated at a B2 level by some teachers may be rated lower by others, e.g., because of his ‘accent’. How much weight is given to pronunciation and intonation, to pausing and hesitation, or to grammatical accuracy? How explicit are we as an international community of assessors about our expectations of performances by non-native speakers of a language? Are there national characteristics in assessment or varying sensitivities to particular criteria?

Case 5: Students have to give presentations in a foreign language. They often lack confidence and thus motivation. Although there is a lot of truth in the adage that practice makes perfect, how can they know whether any progress has been made? When students can record themselves, get feedback, watch themselves, do the exercise again, and observe themselves again, they become better at self-assessment and gain insight into their own acquisition. The CEFR offers carefully formulated descriptors, but the interpretation of these ‘can do’ statements may still be opaque to students. Learners need to be able to link descriptors to specific samples illustrating performance at the corresponding level.

In these and in many other cases, the daily practice of oral assessment raises questions which call for a tool to facilitate online collaboration in the collective application of the CEFR. It is for this purpose that WebCEF is now being developed. WebCEF () is an initiative of a European consortium of seven higher education institutions: Katholieke Universiteit Leuven, Belgium (coordinator);the Dresden University of Technology, Germany; the University of Chambéry, France;Fontys University of Applied Sciences, the Netherlands; the University of Łódź, Poland; the University of Helsinki, Finland; and the Open University, the United Kingdom.Partners in this project have a long history of collaboration in computer-aided language learning and language assessment. The WebCEF project runs from October 2006 to September 2009 and is funded under the Socrates-Minerva programme of the European Commission (Directorate-General for Education and Culture), with the support ofthe Education, Audiovisual & Culture Executive Agency (EACEA).

How can WebCEF help?

First of all, the WebCEF site provides a ‘showcase’ with representative samples of video and audio recordings of learners performing comparable tasks, at each of the CEFR levels. These samples, recorded in various countries and educational settings, have been collectively assessed and annotated by members of the WebCEF team, and are offered, not as authoritative assessments, but as consensual examples of oral production at each level. The samples are accompanied by annotations indicating how features in the production relate to the descriptors for that level. This is the first goal of WebCEF: helping teachers and teacher trainees (but also students) to get a better grip of the descriptors in the CEFR. At present, a showcase is being developed for each of the languages used by the partners of the project: Dutch, English, Finnish, French, German, and Polish. Through the showcase, visitors to the WebCEF site can also see how the online assessment tool can be used for making assessments for these or indeed any other target language.

This is the second goal of the WebCEF project: to develop a self-sustaining community of practice, enabling collaborative assessment of oral language proficiency through a Web-based environment. The WebCEF community is a network of several national and/or language-specific communities linked into an international community. The community has been created in stages: in the initial phase it is formed by the project’s immediate partners in Belgium, Finland, France, Germany, the Netherlands, and Poland; then it is extended to small local and national networks in these and other European countries, and finally to a broad extended community of practice. This community is didactically sustained by a didactic model and a manual, the ‘Mindbook’, which is also available on the Website and is being developed as a pedagogically justified rationale for future foreign-language education assessment.

The notion of community also operates at the immediate level of the user. Every WebCEF user is a member of one or more groups; a group can be made up of language students and teachers (a student group) or can include apprentice teachers and teacher trainers (a mentor group). Within a group, members can upload recordings and can assess, annotate, and review their own and each other’s recordings. In this way, a recording of one student may receive several assessments, which are all displayed on the same screen in the form of bar charts and annotations. Comments may be linked to a particular part in a sample, thus providing the student with an assessment linked to an ‘official’ descriptor and to individual comments by assessors. If more than one assessment is available, a survey of assessments is shown as well. Hence, all users belonging to a group have access to multiple assessments, and are given feedback on their own samples (see fig. 1).

Fig. 1. An example of an assessment page

The technology used is low-threshold, to make it simple for users to create and upload their own video and audio samples, using basic equipment available in any institution. Once uploaded, the samples are automatically converted to Flash format and can be viewed and assessed through the online assessment tool. The interface for managing groups and for uploading and assessing samples is being developed in each of the languages listed above, with the possibility of other languages – notably Italian and Spanish – being added subsequently.

In this way, local groups and communities can be formed, without need for technical support, and can be incorporated into a wider national or international community. For example, an Italian WebCEF community is now being formed at the initiative of language teachers at the University of Bologna. Like all national communities, this will develop two types of collaboration: language-specific, with teachers and learners of Italian as a foreign language inside and outside Italy, and country-specific, with groups devoted to other languages within Italy. These in turn can link with communities devoted to particular languages in other countries. Another example of a cross-national community: at the K.U.Leuven (Belgium), trainee teachers of English as a foreign language assessed samples made of EFL students at the University of Chambéry (France). The trainees improved their skills in the use of the CEF scales and had their assessments validated by the teachers in France. The potential for forming similar communities of language learners or teachers at a local level or across Europe increases rapidly as the project develops.

Briefly then, WebCEF offers the following services:

-A didactic model for using the CEFR in oral assessment

-The display and joint analysis of oral proficiency samples supported by new and easily accessible technologies

-The creation and maintenance of a European community of practice of teaching staff, trainees, and students for assessing oral production and interaction

-Opportunities for self and peer assessment for language learners

-Access to a constantly growing database of samples and assessments for language researchers

The WebCEF community is entirely open; information on how to become an active participant is given on the ‘WebCEF community’ page of the Website.