EXECUTIVE SUMMARY: UFH AIMSSEC ACE Programme Evaluation / PART 1

University of Fort Hare

African Institute for Mathematical Sciences Schools Enrichment Centre (AIMSSEC)

Advanced Certificate in Education (ACE)

Programme Evaluation

PART ONE REPORT: EXECUTIVE SUMMARY

Kelello Consulting (in collaboration with Jill Adler and colleagues)

July 2011

Contents

Contents

1.Introduction

2.Summary of Part One Report Findings

2.1How does the FHA ACE aim to improve the quality of teaching practice in delivering the mathematics curriculum and to impact on learner performance?

2.2Which elements of the FHA ACE 2011 course inputs are viewed as most important?

2.3Which elements of the FHA ACE 2011 course inputs are viewed as strengths?

2.4Which elements of the FHA ACE 2011 course inputs are viewed as weaknesses?

2.5What changes are recommended for the programme, based on these initial perceptions?

3.Emerging Recommendations

List of Tables

Table 1: Descriptors of envisaged current and future practice

Table 2: Envisaged shifts and evidence to look out for

List of Figures

Figure 1: Framing the problem

1.Introduction

This report is the executive summary of Part One of the series of four evaluation reports reflecting on the Advanced Certificate in Education in Mathematics offered to 72 teachers, all of whom are from the Eastern Cape.

The programme is offered by the University of Fort Hare in partnership with the African Institute for Mathematical Sciences Schools Enrichment Centre (AIMSSEC). The programme is funded through contributions from the Eastern Cape Provincial Department of Education, Zenex Foundation and Datatec.

The FHA ACE was a formal teacher education programme for mathematics teachers, accredited at level six on the National Qualifications Framework (NQF). The programme was planned to run from January 2011 to December 2012 and involve approximately 75 teachers in three Phase groups: Intermediate, Senior and FET.

This evaluation has been commissioned and funded by the Zenex Foundation.

The evaluation comprises the following elements:

  1. Evaluative comment on the model being used, as this is thought to be distinct from other teacher education models being adopted in South Africa. This includes reflections on the strengths and weaknesses of different elements of the model, from the programme designers and teachers in the programme. This has the following as its primary research questions:
  2. How does the FHA ACE aim to improve the quality of teaching practice in delivering the mathematics curriculum and to impact on learner performance? This question focuses on the intended curriculum of the FHA ACE. Answering it will be in the form of a detailed project description including articulations from programme managers, designers and lecturers on project inputs and intentions.
  3. Which elements of the FHA ACE 2011 course inputs are viewed as strengths and weaknesses of the programme, and what suggestions are made to improve the training design for 2012 from the perspective of:
  4. Service providers (programme managers and lecturers)
  5. Participating teachers?

This question focuses on the participants views (service providers and teachers) on the implemented curriculum of the FHA ACE

  1. Measurement of the impact of the FHA ACE in relation to the learning gains in the mathematics content knowledge of the teachers participating in the programme. This has the following as its primary research question:
  2. What shifts, if any, are evident in teachers’ subject matter knowledge considering their attainment in the pre-test and subsequent examinations:
  3. Pre-test results (January 2011);
  4. 2011 examination results (December 2011);
  5. 2012 examination results (December 2012).
  1. Measurement of any shifts in teaching practice for nine case study teachers (three teachers per phase) from the FHA ACE. This has the following as its primary research question:
  2. Focusing on nine case study teachers, what evidence is there of shifts in their classroom practice in relation to:
  3. Shifts in classroom practice evident from the type of content and curriculum coverage in a selection of learner books;
  4. Shifts in classroom practice evident from school visits (by evaluators, by tutors and or colleagues in the programme); and
  5. Shifts in learner attainment as evident from their school assessments;

The Part One report focuses on the first element.

2.Summary of Part One Report Findings

This summary of findings provides brief answers to the primary research questions for element one, above.

2.1How does the FHA ACE aim to improve the quality of teaching practice in delivering the mathematics curriculum and to impact on learner performance?

In order to improve the quality of teaching in delivering the mathematics curriculum, to impact on learner performance in schools, a formal accredited academic programme for mathematics teachers has been developed. There are seven key inputs into this programme, and the outputs of the programme are considered in relation to teacher knowledge, classroom practice and learner attainment. This is illustrated in the following figure:

Figure 1: Framing the problem

The aims and purpose of the programme as well as it curriculum design, curriculum delivery model, collegial support networks, and its assessment choices are described in brief below.

2.1.1Structure

The FHA ACE is a formal academic qualification registered as an Advanced Certificate of Education at level 6 of the national qualifications framework. It is of two years duration and comprises 120 credits. The programme is accredited by the University of Fort Hare. The programme is implemented through a partnership between the University of Fort Hare and AIMSSEC. As a result of the various funding sources, from the Eastern Cape Provincial Department of Education, Zenex Foundation, and Datatec, all of the teachers on the programme are able to undertake this academic programme at minimal charge to themselves.

The programme coordinators considered the programme to be unique in South Africa, because:

  • It makes use of the blended approach to learning, that is, it has telematic sessions as well as face to face residential sessions. Teacher students submit monthly assignments for which they receive feedback. The programme uses Moodle course management system and e-learning.
  • The programme is the first initiative between UFH and AIMSEC.
  • The programme is designed to include (and make it more convenient for) teachers from disadvantaged rural areas to participate.
  • There is a high academic standard/rigorous approach to mathematical subject content knowledge, though only school level maths is covered. There is an activity based approach to learning mathematics.
  • The intention is to have a network to provide on-going communication and support for teachers during and beyond the course, and to develop a culture of mutual support between the teachers for sharing ideas.
  • Teacher students are trained to be subject leaders, e.g. to be able to facilitate teacher workshops at their schools.
  • It has international lecturers helping on courses, who are very experienced and enthusiastic.
  • The programme has multiple funding, and gets funding from ZENEX, DoE and other funders.
2.1.2Programme Aims and Purpose

There are several articulations of the programme aims and purpose. The official programme documents include descriptions of purpose which emphasize the UFH link to AIMSSEC, training subject leaders, subject knowledge and didactic skills, knowledge and understanding of mathematics leadership, good subject knowledge, and the quality of lessons taught. The potential impacts on learners in relation to improved learner performance and increased uptake in Mathematics are suggested. The programme coordinators view mathematical content knowledge as a very important component of the programme. They describe the programme as innovative, and as developing or improving presentation and teaching skills. It adopts a blended learning delivery model, involving an annual residential session, 10 hours per week time commitment, the monthly assignment submission, and end of year examinations. The programme coordinators agree that the following aim statements were most important or pivotal to the programme (listed alphabetically) - the programme aims to:

  • Expose participating teachers to good mathematics teaching practice;
  • Expose teachers to many content rich mathematical problems, and sources of ideas and problems, which they can use in their classrooms;
  • Improve the mathematical subject matter knowledge of the teachers in the programme – they should learn the mathematics they are required to teach;
  • Improve the pedagogical content knowledge of the teachers in the projects – they should learn how to teach the mathematics they are required to teach;
  • Motivate teachers to create exciting and interesting mathematics lessons.

Equal status is given to mathematical subject matter knowledge and pedagogical content knowledge. The programme coordinators value the problem solving approach and consider exciting and interesting mathematics lessons to be a priority. In contrast to these pivotal aims, far less emphasis is placed on curriculum coverage, use of pace setters and the academic research aspects of mathematics education.

There are three key areas in which the programme coordinators hope to see shifts in behaviour: subject matter knowledge; teaching practice; and subject leadership roles. The envisaged shifts in behaviour of the participating teachers are articulated slightly differently by each programme coordinator. Nevertheless some common themes emerge.

In relation to subject matter knowledge there are expectations that the teachers in the programme will widen and deepen their mathematical understanding, and that this will be reflected in shifts in results from the pre-tests to the final examinations. A target of 50% shift in results from the pre-test to the examination has been suggested as a target. The dominant bench mark is considered to be competence at the level at which teachers are expected to teach: Grade 6 for Intermediate phase teachers, Grade 9 for Senior phase teachers, and Grade 12 for FET teachers.

In relation to teaching practice, the programme coordinators expect teachers to adopt a style of teaching that promotes guided discovery or enquiry-based learning. Teaching should be transformative rather than instrumental. Evidence would be to observe learners engaged in rich activities both practical and pen and paper, asking questions but also discussing the mathematics and wanting to find their own solutions, expecting to give reasons for answers they give to the teacher, working in groups, presenting their own solutions to the class on the blackboard or on posters. Teachers are not expected to introduce mathematical facts without reasons and proofs suitable for the learners, and should not be substituting into formulae without understanding the derivation of the formula. Problem solving, cooperative, collaborative, inquiry inspired, and re-invention activities are emphasised. There is also some emphasis on use of learning materials (including the use of technology where available) and designing materials.

With regard to subject leadership, the programme coordinators view the programme as setting the teachers on a path towards subject leadership (and are aware that they probably will not have reached this destination by the end of the programme). Helping and sharing explorations with colleagues, potentially through facilitating workshops seems to be the main emphasis given to the subject leadership role. Developing mathematics subject matter knowledge and improving teaching practice (arguably necessary preconditions for subject leadership) are more valued than developing subject leaders.

The following summarises the key descriptors used by lecturing staff to explain envisaged current and future practice:

Table 1: Descriptors of envisaged current and future practice

Descriptors of envisaged current practice / Descriptors of envisaged future practice
Didactic, instrumentalist, teacher talking, transmission / Activities, listening to learners, teacher questioning learners, learners partners in learning
Procedural, many examples, closed questioning / Investigate, problem solving, open questioning, generalising, predicting, giving reasons,
Dominant use of black board and text books / Use of concrete materials (paper, counters,)
Possible use of ICT software such as geogebra
Deeper subject matter knowledge, making connections between sections

The programme coordinators provided examples of their envisaged shifts in teaching practice as of what they would hope to see in the classrooms of the participating teachers. They also indicated what evidence evaluation researchers should collect during school visits, to observe such shifts:

Table 2: Envisaged shifts and evidence to look out for

Envisaged shift in teaching practice / Evidence researchers should look for
Take ownership of activities based cooperative teaching and learning. / Move away from teacher-tell chalk n talk, text book to words problem-based investigation and collaboration
Use activities to enhance understanding of learners. / File in teacher's file.
Making/planning their work and daily lesson plans / Look for lesson plans in their master portfolios.
Hope to see learners engaged in active learning, prepared to tackle non-standard problems and do some work themselves using their own initiative. / Look for teacher doing most of talking at end of the lesson rather than the start, summing up what pupils have learnt from their own efforts.
Use of technology. / Geogebra use for designing tests, exams and worksheets.
Design appropriate and standard assessment tools. / Look for question papers and memorandum and mark lists.
Hope to see teachers asking probing questions to encourage learners to give examples and answers rather than teachers telling learners too much. / Note open questions, teachers expecting learners to give reasons for answers and teachers asking rather than telling.
To be self-confident to prepare on their own. / Show work solutions.
Go through the learners' books to confirm the practices. / Learners' books.
Hope not to see formulas, procedures and results given without explanation, reasons and proof. / Look in learners' workbooks for evidence of explanations, reasons and proof (justification) of mathematical processes and results/facts.

The coordinators also provided anecdotes of impact of the programme which they formulated from their prior experience in the programme. These anecdotes related to:

  • Teaching practice: Adopting ICT software in teaching;
  • Ongoing professional development: Continuing to learn mathematics and engage in professional development;
  • Career progression: Promotion from a teacher to a Head of Department, involvement in subject leadership initiatives in the district; promotion to a lecturer position;
  • Improvement in subject knowledge: awareness of what is not known, and improvements evident in attainment in the pre-tests and final examinations of the programme.

These envisaged shifts, the suggestions for evidence, and anecdotes of impact, will be used to inform the design of the school visit research tool for the school visits planned for the case study teachers.

2.1.3Curriculum design

Curriculum design is described in relation to the selection process for the course, the phase differentiation within the progamme, the course modules offered, and monitoring and evaluation activities planned internally to support the programme.

The original requirements for entry into the programmes were that the teachers were predominantly from the Eastern Cape, had access to a mathematics classroom in a public school in which to teach, and had completed a ‘Mathematical Thinking’ short course offered by AIMSSEC. The final requirement had to be relaxed however and teachers were allowed entry into the programme, provided they completed the Mathematical Thinking course in parallel to the programme. As such, 72 teachers were enrolled in the FHA ACE programme, of which the majority had completed the Mathematical Thinking course prior to commencing it, and 32% were expected to complete the short course in parallel to it.

The programme coordinators opted to organize the participants in the ACE programme into three groups: Intermediate Phase, Senior Phase and FET. Although this is referred to as a single programme, in fact the offering consists of three distinct streams – one for each phase. The assessments, content and expectations for each group are different. The three groups enrolled in the programme for 2011 and 2012, consisted of 26 Intermediate phase teachers, 19 Senior phase teachers, and 27 FET teachers.

The programme was comprised of ten modules with credit weightings ranging from 8 to 16 credits. It was expected that all teachers undertook five modules in the first year, and five in the second year. The programme amounted to a total of 120 credits, as required by the University of Fort Hare.

The programme designers included several activities to monitor and reflect on the programme. These included soliciting information about the participating teachers’ schools and community contexts, teacher feedback questionnaires following residential sessions, and planning and reflection sessions between the collaborating training provider partners

2.1.4Curriculum delivery

The curriculum delivery model used for the FHA ACE programme is both innovative and ambitious.

A blended delivery model was adopted which includes elements of both traditional face-to-face tuition, as well as distance education practices. Teachers attend residential sessions annually. They are then expected to work through learning materials and assignments independently. They are supported in this endeavour through monthly assignments, interactive telematic education sessions, an e-learning environment using Moodle, and collegial support networks.

Two residential sessions were planned by AIMSSEC – the first in January and another in June/July. However this was subsequently reduced to only one residential session in the first year as a result of student concerns about the second session being introduced late, and not being covered by the bursary funds.

The first residential component was undertaken as planned in January 2011. This commenced for staff on the 1st of January 2011. Teachers arrived on the 2nd of January at Stellenbosh High School. A total of 71 contact hours per student was provided. This comprised 42 sessions per student. Considering the three different groups, this amounted to 90 teaching sessions in total. The teaching team comprised of both UFH lectures, AIMSSEC lecturers, and AIMSSEC volunteer international lecturers.

Real time Interactive Telematic Education (ITE) workshops were only a small part of the course. The original intention was that activity in these ITE sessions would be coordinated by the Fort Hare staff, who were to be physically present to monitor and give support to teachers. However, this did not take place.[1]The course coordinators from AIMSSEC conducted and taught the workshops via broadcast link. These took place in four centres: East London, Post Elizabeth, Queenstown and Umtata, on six Saturdays each year of the programme. In Mthatha a graduate from the ACE subject leaders courses acted as a peer mentor during the ITE sessions. The long term intention was to “involve course graduates in this capacity which in terun provides professional development for them”.[2]