Partners for Change

PARTNERS FOR CHANGE

Annex 2 Hornby School: An Introduction to Designing Communicative English Language Assessment

1. General Information
What is the title of the school? / An Introduction to Designing Communicative English Language Assessment
Which is the lead country? / Bangladesh, Iran
Which other countries are involved? / Nepal, Sri Lanka, India, Pakistan, Afganistan
Which venue will be used? / Colombo, Sri Lanka (Iran School) and
Dhaka, Bangladesh
(the details of the venue will be confirmed)
On which dates will the school be held? / Two consecutive schools: Iran school will be held in 4th week of November 2016 and Bangladesh school will be held in 1st week of December, 2016
What topics will the school cover? Please include a brief description of each topic. / This workshop aims to improve the assessment skills and develop a solid understanding of standardised testing and the key steps in the test development process of English language test developers, test administrators and teachers.
It will cover testing and assessment methodology and principles but will be also very practical in nature. Participants will get ample opportunities to design tests throughout the course to apply the theory. It will cover the following topics to achieve its aim:
·  Factors Influencing the Assessment of English Language Learners
This section will cover factors to consider when developing assessments including students' language level, age, educational background, and culture.
·  Key Testing Principles
This will highlight the essential qualities of test that are integral to proper measurement.
·  Planning the Assessment
This section will focus on different steps within the planning process including understanding the purpose of the test: whether it is summative or formative, types of learners, length of tests, language skills, language elements and so on.
·  Developing Test Items and Scoring Criteria
It will cover the steps of developing test items and how the items will be linked directly to the test specifications and additionally the content and skill that the items are supposed to measure.
·  External Reviews of Test Materials
This section will cover the effective techniques of reviewing test materials to improve the quality of assessments and how the insights external reviewers provide can help test developers understand how students are likely to interpret test materials.
·  Evaluating the Tasks Through Trailing
This part will focus on how to try out or pilot test items which can provide extremely useful information to understand how appropriate the items are and to evaluate the clarity of instructions given to examinees during the test development process.
·  Using Statistics to Evaluate the Assessment and Scoring
This section will emphasise on how the empirical evidence should be gathered to evaluate the fairness of assessments and scoring and how to investigate whether scores are sufficiently reliable to support their intended interpretations.
2. Audience, objectives and Impact
Who are the target group(s) from which participants will be selected? / The main target group from Bangladesh will be technical experts from the Education Ministry including test developers and test administrators, teacher educators from teacher training colleges, and master trainers who are responsible for designing and reviewing curriculum, textbooks, assessment and teacher training and cascading new knowledge, skills to practicing teachers.
Iran – The main target from Iran will be teachers, teacher trainers and educators, curriculum developers and test writers from the state sector.
There will also be a representation of a similar participants form Sri Lanka, India, Nepal, Pakistan and Afganistan,
How will you identify and communicate with participants? / The opportunity will be circulated through the Ministry of Education, Teacher Training Colleges, the National Curriculum and Textbook Board, the Board of Intermediate and Secondary, Bangladesh English Language Teachers Association, British Council trainer/teacher database and Hornby alumni of each country.
Iran - The opportunity will shared with our contacts in the MOE, Teachers Associations and other teachers in our network.
Participants Selection Criteria:
·  At least five years’ experience in English teacher training/curriculum/test designing and material development.
·  Willingness/commitment to cascade the learning from the workshop to the practicing teachers
·  Experience in developing/implementing national strategy/policy for education reform/assessment reform
What are the immediate objectives (deliverables) of the school? / What are the learning outcomes of the school? What will the participants take away with them in terms of new skills, products, etc.?
By the end of the School participants will develop
·  A better understanding of methodology and principles underpinning testing and assessment in ELT
·  An increased ability to create tests that accurately assess language and skills
·  An increased capacity to write quality test items for assessment purposes
·  An increased ability to design a reliable and valid test
·  An increased ability to use information gained from testing and assessment to evaluate the fairness of assessments and scoring
What are the long term outcomes (impact) of the school? / How will the participants who attend the school make a difference when they return to their working context?
Bangladesh:
Participants will
·  be able to contribute in improving the standard of current exams
·  be able to contribute to the implementation of the joint initiative of introducing speaking and listening assessment in high stake exams by the British Council and the Ministry of Education
·  will be able to join in the test developers team who will be trained in all aspects of listening and speaking task development under the joint initiative of introducing speaking and listening assessment in high stake exams by the British Council and the Ministry of Education
·  present their learning to the Ministry of Education at a round table meeting
Iran
Participants will
·  be able to deliver cascade training in their regions
·  be able to contribute to the national test bank this improving the standard of tests currently available
·  be able to share their learning at regional and national meetings.
3. Local / Regional Relevance
Explain the relevance of the school to country needs. / Relate to the objectives and impact in section 2.
Bangladesh
Across South Asia there is a clear need and demand for assessment reform which can provide a clear picture of pupils’ progress of learning and the effective ways of measuring it. There is also a need to hold the system accountable for improving student learning outcomes. Together with most of the countries in South Asia, Bangladesh has given importance to education reform, demonstrated through the recent development of the national education policy and the revision of curriculum and textbooks across all subjects at both primary and secondary levels. The Ministry of Education has already prioritised introducing school and classroom-based formative assessments in order to improve the quality of teaching and learning. In addition, British Council has been doing the groundwork in partnership with the Ministry of Education in order to include speaking and listening skills in the public examinations, which are currently not assessed.
Assessment is a serious and sensitive issue in the context of Bangladesh because lives are affected either positively or negatively through its outcomes. There are four public examinations in primary and secondary level: the Primary Completion Examination (PSC) at the end of year 5, the Junior Secondary Certificate (JSC) at the end of year 8, Secondary School Certificate (SSC) at the end of year 10 and Higher Secondary Certificate (HSC) at the end of year 12. These are Norm referenced tests which show how a student has performed compared with other students in the group. Successful completion of these examinations enables students to move on to the next phase. The result of SSC and HSC is a pre-requisite for admission into tertiary level institutions.
According to the objectives of the curriculum, development of four language skills of learners is equally emphasised. Therefore, the current tests are supposed to be testing communicative competence but in reality they are far from being a test of communicative ability. All examinations focus on predominantly reading and writing skills, which have a severe ‘washback’ effect on teaching practices. This often leads to a narrow focus on knowledge and skills necessary for learners to get through the exams rather than on developing their actual language skills as stipulated in the curriculum. Test anxiety among learners and test dominated lessons thus appear to be a common phenomenon in Bangladesh. However, the aims and objectives of this communicative curriculum can never be achieved without an assessment system that assesses the communicative ability of the learners (Haider, 2008).
According to Khan (2010) the marked weakness of the testing system of Bangladesh is the absence of listening and speaking components on the test, lack of test specifications, examiner training and proper guidelines for scoring and grading. The predictive nature of questions on the test and the dependence on private tutors is also highlighted as a major weakness in her report. She argues that the current tests are not meeting two fundamental qualities of good tests- reliability and validity. She also states that there needs to be an equal focus on the four skills and listening and speaking must be tested if achieving communicative competence is the goal.
The analysis of the 2015 and 2016 SSC English test papers also indicates that urgent training is needed in language test development (Green, 2016)
In response to the need of developing capacity in the field of assessment, the British Council and A S Hornby Education Trust jointly hosted a Hornby Regional School in Dhaka in February 2016 on the theme Assessment for Learning where 32 participants from the National Curriculum and Textbook Board and Government Teacher Training Colleges of Bangladesh, who play a key role in developing and implementing education policy in Bangladesh, attended the five day training. In addition, 7 participants from Sri Lanka and Iran with a similar level of expertise attended the school. The training covered the area of assessment for learning, including best practice in assessment, how to design assessment tasks, effective ways of assessing learners’ performance and how to provide timely and quality feedback to learners for improvement. This training was considered a well-timed initiative and proved a huge success. The participants who attended the training reported that it has sharpened their understanding of how to integrate assessment into curriculum and classroom practice. The National Academy for Educational Management (NAEM) has already started to produce assessment literacy materials for use with schoolteachers by their trainers who attended the training. Green (2016) states in the Executive Summary & Assessment Framework that the materials they have developed so far indicate that a lot of thought has gone into them and that they would make a good foundation for a course in assessment literacy. This willingness to put what they have gained from such courses into practice is to be applauded and is illustrative of what can be accomplished from such training initiatives. However, she also recommends that the materials be expanded to include units on each of the four skills, with particular emphasis on listening and speaking, as these are the areas in which teachers feel they need the most support.
This proposed training will develop the capacity of test developers, testing program administrators, teacher trainers and teachers to design a more widespread, systematic and consistent strategy for assessment which will be fair, valid and reliable. This training will also encourage them to keep essential criteria in mind throughout the development, administration, scoring, and interpretation of assessments, and it will ultimately lead to better assessment practices for all learners of primary and secondary level in Bangladesh.
Iran
English as a Foreign Language is not taught formally in schools until Grade 7 (age 11-12) and is compulsory until Pre-University Level (age 18-19). Usually English is taught for a maximum of 3 hours a week and classes are often run in L1.
After more than 30 years, in September 2013, a new curriculum for EFL was introduced focussing on Communicative Language Teaching (CLT) rather the dated Grammar Translation Method (GMT). The Curriculum Development Centre (CDC) set a target of A2 CEFR as the benchmark for completion of lower-secondary and B2 CEFR for completion of higher-secondary level.
The new Prospects Series brought about a much needed change but there has been resistance from the teachers who are expected to use the new methodology. This radical shift from the GMT came as a shock to many teachers (Foroozandeh & Forouzani, 2015) as they have had little training and support and in fact many classes are still conducted in Farsi. The Prospects pack includes a student’s book, a teacher’s guide and extra materials are available on the CDC website for download. However, there is still no formal assessment. Teachers are expected to devise their own end of semester tests for Grade 7 and Grade 8, with a formal ‘national standard test’ at Grade 9, which is still being developed by the CDC. The CDC provides some sample tests on their website but little or no support is given to teachers on test development, its theories and practice.
In the Hornby Regional School of February 2016 the topic was ‘Assessment for Learning’. The Iranian participants on this course were senior teacher trainers and educators from various regions of Iran. The areas covered by the training provided the participants with better understanding of how to design assessment tasks as well as knowledge of how to integrate this into classroom practice. On their return to Iran, they presented a cascade plan to policy makers at the MOE and are currently running cascade training in their regions.
The proposed course will enable us to provide much needed training for a larger group of practitioners and teacher educators and thus greater opportunities for cascade and knowledge sharing with EFL teachers across Iran. They will develop their knowledge and skills in the development, administration, scoring and interpretation of tests enabling them to provide a higher standard of delivery and performance. The CDC encourages teachers to contribute to their test banks which are made available to other teachers across Iran. By contributing to this bank, the participants will be able to have an impact on the standard of testing and therefore the standard of the teaching of English in Iran.
4. Evaluation
How will you assess whether the school objectives are being achieved during delivery and on completion? / During delivery – this will be assessed initially stating the overall objectives of the school and then having day end reflection slots which assess the learning of the day and relate it to the overall objectives
On completion - this will be assessed by trainer & participant questionnaires as well as by a final reflection session where groups evaluate their learning from the Hornby Regional School (HRS) and relate this to the initial objectives.
How will you assess the long term impact of the school? / Long-term impact will be assessed by
·  Contribution in implementing the joint initiative of introducing speaking and listening assessment in high stake exams by the British Council and the Ministry of Education
·  participation in the test developers team who will be trained in all aspects of listening and speaking task development under the joint initiative of introducing speaking and listening assessment in high stake exams by the British Council and the Ministry of Education
·  The quality of the test items developed for the speaking and listening assessment
·  Follow-up feedback from participants after 6 and 12 months to assess how they feel the School influenced their teaching/training, what impact the ideas had on their teaching/training and whether/how often they tried out the activities and ideas with their trainees
·  Comments on country/group Facebook pages (across the SA region)
Describe any follow-up events that you have planned to ensure that participants are achieving long term objectives. / ·  Sample test items developed by participants
·  Regional and national policy dialogue on assessment
·  Facebook group page for Hornby School participants to share ideas and activities after the School
·  Conduct a more detailed M&E evaluation of the impact the HRS has had

Reference