EUROPEAN UNION

Project financed under Phare

MoERY/ncdtvet-PIU

Activity 09

Training in Assessment methods and procedures

Training and Advice for Further Development of the TVET Sector, Romania,

PHARE TVET RO 2006/018-147.04.01.02.01.03.01

2008

“The content of this material does not necessarily represent the official position of the European Union”

Report on the evaluation on existing assessment methodologies and procedures

Contents Page

Glossary of terms4

Acronyms6

Summary7

  1. Introduction8
  2. Context9
  1. Evaluation of the currentassessment methodologies13
  1. ECVET and assessment 18
  2. Recommendations for assessment and accreditation methodology 20
    for TVET
  3. Conclusions23

Bibliography24

Appendices 1 Romanian TVET Quality Indicators25

Appendices2 Assessment Manual27

Section 1Glossary of Terms

Active Learning / Learning environments that allow “students to talk and listen, read, write, and reflect as they approach course content through problem-solving exercises, informal small groups, simulations, case studies, role playing, and other activities -- all of which require students to apply what they are learning” (Meyers and Jones -1993)
Assessment / It is the judgement of evidence presented by learners, and the process of making decisions about the extent to which learners’ evidence meets the assessment criteria for a qualification or unit, and demonstrates learners’ competence.
Criterion-referenced assessment can be formative and summative. During formative assessment the assessor establishes and gives feedback as to how many of the criteria have been achieved and what still needs to be done. Summative assessment occurs when a module has been completed.
Competence / The ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development. In the context of the European Qualifications Framework, competence is described in terms of responsibility and autonomy.
Credit for learning outcomes (Credit): / "Credit " individuals' learning outcomes which havebeen assessed and which can be accumulated towards a qualification or transferred toother learning programmes or qualifications;
Curriculum / A document describing the modules that will be taught for a qualification, including their delivery.
ECVET points / A numerical representation of the overall weight of learning outcomes in a qualification and of the relative weight of units in relation to the qualification.
European Qualification Framework (EQF) / A system that defines knowledge, skills and the wider personal and professional competences to 8 levels and matches them to existing national qualifications across Europe
eqf_en.pdf
Key competencies / Represent a transferable, package of knowledge, skills and attitudes that all individuals need for personal fulfilment and development, inclusion and employment and should act as a foundation for further learning as part of lifelong learning (EQF)
Learning outcome(s) / The set of knowledge, skills and/or competences an individual acquired over a period of time and/or is able to demonstrate after completion of a learning process, which are defined in terms of knowledge, skills and competence
Life-long-learning / All learning activity undertaken throughout life, with the aim of improving knowledge, skills/competences and/or qualifications for personal, social and/or professional reasons.
Performance criteria / Qualitative or quantitative descriptions of what the learner must be able to do that allow an objective judgement to be made about whether or not an individual has achieved a learning outcome
Skill / The ability to do something, consistently, confidently and accurately
Portfolio / Within criterion-referenced assessment this is a collection of evidence, including formative and summative assessment records, which is cross-referenced to the standards, usually in a file or binder, by which a learner claims competence.
Qualification / A formal outcome of an assessment and validation process which is obtained when a competent body determines that an individual has achieved learning outcomes to given standards
Recognition of learning outcomes / The process of attesting officially achieved learning outcomes through the awarding of units or qualifications
Validation of learning outcomes / The process of confirming that certain assessed learning outcomes achieved by a learner correspond to specific outcomes which may be required for a unit or a qualification

Acronyms

Name / Acronym
Technical and Vocational Education and Training / TVET
Sector Committees / SC
European Qualification Framework / EQF
National Quality Assessment Framework / NQAF
European Credit System for Vocational Education and Training / ECVET
National Centre for the Development of Vocational Education and Training / NCDVET,
Ministry of Education Research and Innovation / MERI
National Adult Training Board / NATB
Council for Occupational Standards and Accreditation / COSA
Training Standards / TS
Romanian Agency for Quality Assurance in Pre-University Education / ARACIP
Student Centred Learning / SCL
Lifelong Learning / LLL

Summary

The existing Romanian Technical and Vocational Education and Training (TVET) system is shaped on learning outcomes. They form the basis of the aims and objectives of the modularised curriculum. The curriculum is described in terms of key transferable competences as well as the more specific technical knowledge, skillsand attitudes covered by each of the 16 Sector Committees (SC). According to the qualification specifications, the learning outcomes need to be assessed against their associated performance criteria and achieved in totality to certificate a learner with a qualification.

However at present there is an over emphasis on assessing discrete pieces of knowledge:

  • teachers give an excessive number of short theory tests, in many cases, to provide a mark against a learner every 3 weeks
  • there is a lack of student centred learning and practical work experience that allows the development and assessment of key competences

The culture in Romanian education is for all learners to achieve.Teachers overestimate the ability of their learners and let them move onto the next stage of learning without a thorough understanding of the previous stage. This culture benefits few people and generates little trust by the employers in the education and qualification system. The learners attempt to gain qualifications above their current ability level and teachers have a difficulttask in delivering the curriculum.

The European Qualification Framework provides standard reference levels for qualifications across Europe. The Romanian VET qualifications need to be referenced to this framework through the development of a national qualification framework that clearly describes the key competences. With this system in place and the implementation of the existing quality assurance system, transparent assessment can take place and so allow for the introduction of the European Credit System for Vocational Education and Training (ECVET). Learners will be able to progress seamlessly from one level to the next.

To achieve transparent assessment against clear learning outcomes based on knowledge, skills and attitudes,the Romanian educational culture needs to change from a hierarchical system between inspectors, school directors, teachers and pupils to that of a learning community working towards common and achievable learning goals. The change may be achieved by:

  1. Developing an assessment manual[1] to support the TVET system
  2. Providing clear assessment guidance within qualification and curriculum documentation
  3. Promoting assessment for learning within the classroom
  4. Training teachers, inspectors, school directors and parents in assessment and its associated procedures
  5. Clarifying levels by specifying a national qualification framework in terms of the key competences
  6. In the medium term, creating an educational system that is more flexible and includes partial certification of units that allows for the easy implementation of the ECVET system.

  1. Introduction

This paper explores current assessment methodologies in Romania, within the context of EU systems and current assessment and learning theories; reviews current practice and suggests recommendations to improve the assessment systems.

Section 2 gives some background to the paper, and thensection 3 describes the current nature of the Romanian assessment methodology and evaluates it against a checklist based on the National Quality Assessment Framework (NQAF) quality principles. Most of the material from this section is presented in table-format.

Section 4 goes on tosuggest improvements in the assessment methodology. Much of the detail of these suggestions have been included in the format of an Assessment Manual, whose content is found in Appendix 1

Section 5 makes recommendations for assessment and accreditation methodology in the light of ECVET.

Section 6 concludes the paper with an outline of how the general concepts of the system might be changed in the future.

  1. Context
  2. Changing global society

The pace of change in society generally is accelerating and the future will be very different from the present. People need to acquire new knowledge and skills and embrace different attitudes and ways of working so that they can adapt to change.

  1. European
  2. European Qualification Framework

The European Parliament and the Council of the European Union has adopted the establishment of a European Qualification Framework (EQF) for lifelong learning.[2] The adoption of the EQF with its eight reference levels and associated definitions provides a mechanism to create transparency between national and European qualification systems. The Recommendation specifies that countries should relate their national qualifications systems to the EQF by 2010. By 2012, every new qualification issued in the EU should have a reference to the appropriate EQF reference level, so the benefits to mobility and lifelong learning that the EQF brings will be visible and available to every EU citizen.

The European Qualification Framework includes a glossary that explains the terminology on learning outcome and the meaning of: knowledge, skills and competencies. Practical level referencing is determined to be based on learning outcomes and therefore elaboration of the learning outcome approach is part of the future curriculum and qualification development process.

The establishment of an EQF Advisory Group supports further development and implementation of the EQF and comparing the corresponding National Qualification Frameworks policy,development and application. Comparing learning outcome based qualifications calls for a clear description of what is “in it” and what are the demands of learning outcomes used for referencing the EQF levels or appropriate learning outcome descriptions.

The process of developing the EQF has inspired and influenced the way qualifications have been described during the last decade. Experts now concentrate attention on the relationship between output and input centred approaches. Even with the shift to learning outcomes the teaching process still calls for some guidelines in terms of input – the curriculum, to support the local planning and contribution to setting the context for the learning outcomes.[3]

One of the main conclusions from the research undertaken by CEDEFOP in 2007 on the concept and impact of implementing learning outcomes in Europesummaries:

The focus changes, but the other components of the process do not disappear. Learning outcomes are the focus, and provide a key role in organising systemic aims, curriculum, pedagogy, assessment and quality assurance. These other factors remain significant in planning and implementation. p.41

  1. European Credit System for Vocational Education and Training (ECVET)

The European Commission (EC) agreed on the establishment of the European Credit system for Vocational Education andTraining (ECVET)[4] in April 2008. The system facilitates learners’ mobility within the EU by recognising their achievement from the result of studying qualifications in other EU countries. The system is based on memorandums of agreement between institutions where they agree the credit value for a particular unit of learning within a qualification. The credit value is based on a full time vocational qualification course having the equivalence of 60 credit points in any one year. The EC expects that countries to be gradually implementing the system from 2012, by which time they will have developed tools and users guides to facilitate the process.

  1. EU Key Competences for Lifelong Learning (LLL)

In 2006 the EU parliament identified and defined the key competences[5] necessaryfor personal fulfilment, active citizenship,social cohesion and employability in a knowledge Society. The EU suggests that the inclusion of these competences in the curriculum for initial education and training ensure thatby the end of school youngpeople will have developed the key competences toa level that equips them for adult life and thatforms a basis for further learning and working life. Another aim of the LLL key competence frameworkis to provide a European-level reference tool forpolicy-makers, education providers, employers, and learners themselves to facilitate national- and European-level efforts towards commonly agreed objectives

The Reference Framework sets out eight keycompetences:

1) Communication in the mother tongue;

2) Communication in foreign languages;

3) Mathematical competence and basiccompetences in science and technology;

4) Digital competence;

5) Learning to learn;

6) Social and civic competences;

7) Sense of initiative and entrepreneurship;

8) Cultural awareness and expression.

The four competences: digital, learning to learn, social and civic and sense of initiative and entrepreneurship are skills related and cannot be taught and assessed using traditional teaching methods and have to be developed and assessed using proactive instruments of assessment that generate evidence of attitudinal competence over a number of weeks eg projects, practical exercises, role play simulation and case studies.

iv European Good Practice

With the EQF inclusion of the development of cognitive and practical skills and competence rather than just knowledge, many countries are putting more emphasis on holistic practical assessment tools. Finland assesses the learners’ skills demonstration within the work place as the final assessment of each module. Germany, Austria, Ireland and the UK all include skills demonstration as a part of the final assessment of an individual module.

This form of assessment promotes the development and assessment of the LLL Key Competences as well as motivating the less academic learner to achieve.

Frank Coffield[6], Professor John Hattie[7], Black and Wiliam[8] all advocate the increase use of:

  • shared learning goals
  • assessment criteria developed in partnership with the learners
  • self assessment
  • peer group assessment

to develop motivation for learning, effective learning and assessment for learning inside the classroom.

Buttler (1988) cites the use of marks within an assessment system as a strong demotivator of learners, particularly the weaker learner. He suggests that learners should be given comments in the form of constructive feedback rather than with marks. Butler’s research has shown that this may lift achievement by as much 33%.

  1. Romanian
  2. History of TVET system

Romania was the first country in Central and Eastern Europe to start developing competency-based occupational standards. Since the second half of the 1990s, vocational education has been using these occupational profiles to develop broadly formulated,outcome-based courses and qualifications that will provide young people with access to a range of occupations. The links with representatives of both sides of industry and with companies have also been improved gradually through the establishment of Sector Committees.

Occupational standards were developed, with the assistance of the World Bank, by a specially formed Council for Occupational Standards and Accreditation (COSA) which provided technical assistance to the working parties which developed the standards. This body was later absorbed into the National Adult Training Board (NATB) which is currently developing its role as the National Qualifications Authority. NATB has continued to provide technical assistance in the development of Occupational Standards and Training Standards, working with the emerging Sector Committees, which have the responsibility for validating both kinds of standard and the qualifications which they compose. The NATB also keeps the national register of qualifications including both pre and post university qualifications and continuing professional qualifications. The National Centre for the Development of Vocational Education and Training (NCDVET) has supported this work through a series of PHARE projects

Romania is developing a National Qualifications Framework (NQF) within a national system of quality assurance. The body that assures the quality system is the Romanian Agency for Quality Assurance in Pre-University Education (ARACIP).This body will ensure the relevance and consistency of qualifications and the standards of which they are composed.

  1. Educational system

The Romanian educational system is described in the table below.

  1. Existing 2 Methodologies for developing curriculum from qualifications
  1. Existing assessment legislation

The TVET curriculum methodology was developed in 2003 and refined in 2006 from validated Qualifications formed from Unit Training Standards based on learning outcomes and performance criteria with range statements.

  1. Linkages with other projects and activities

This report makes use of outputs from the PHARE 2005 TVET project, in particular the curriculum development activity.The suggested assessment methodology has also been shared with the PHARE 2006 NATB project, NATB and ARACHIP.

  1. Evaluation of the current assessment methodologiesfor pre-university TVET education

The tableson the following pages evaluate the current curriculum and assessment methodologies and their impact on learning; achievement and certification against the Evaluation Checklist[9]. The checklist is based on the National Quality Assessment Framework (NQAF) quality principles:

Quality Principle 4 – Learning Programme Design, Development, and Review
The organisation is responsive to the needs of all stakeholders in developing and delivering learning programmes
Quality Principle 5 –Teaching, Training, and Learning
The organisation offers equality of access to learning programmes and supports all learners
Quality Principle 6 – Assessment and Certification of Learning – Individual Modules
The organisation operates effective assessment and monitoring processes to help learners progress

The details of these principles may be found in Appendice 1. The evaluation checklist goes into more detail