Working group on evaluating quality in educational facilitiesFirst meeting:18-19 September 2006, OECD, Paris

At the PEB Governing Board meeting in February 2006, Governing Board members agreed to establisha working group to advancePEB’s work on evaluating quality in educational facilities, initiated in 2005.This document describes the objectives, format and themes of this working group, to be held in Paris from 18-19 September 2006, within the context of the outcomes of two previous experts’ group meetings on evaluating quality in educational facilities.

Background

In 2005, the OECD Programme on EducationalBuildingco-organised two international experts’ group meetings in Lisbon, Portugal and Telchac-Puerto, Mexico to discuss how countries define and evaluate quality in educational facilities. More than 40international experts− architects, social scientists, academics, facility evaluation specialists, and representatives from state and national administrations− from 14 countries, and UNESCO and the Organisation of American States, participated in the meetings.

The meetings’ objectives were to define a set of international principles and criteria for evaluating quality in educational facilities; to discuss methodologies used to measure quality in different countries; andto consider options for international methodologiestoevaluating quality in educational facilities.The principal outcome of the meeting was a draft organising framework for evaluating quality in educational facilities (see Annex 1). This framework comprises the following:

  • Definition, principles and criteria of quality in educational facilities.
  • Examples of successful methods for evaluating quality in educational facilities.
  • Parameters within which quality in educational facilities must be defined and evaluated.

A matrix was developed (see Annex 2) that can be used to relate methods of evaluating quality in educational facilities to educational policy, the principles and criteria of quality,and the stage (i.e. pre-occupation [pre-design, design and construction] and occupation) at which the building is evaluated.

Objectives of the working group

The objectives of the working group on evaluating quality in educational facilities are threefold:

To validate the organising framework on evaluating quality in educational facilities, described above (seeAnnex1).

To explore and assess the suitability of methodsfor evaluating quality in educational facilities(e.g.user-centred tools; statistics and indicators; and quality benchmarks) for the purpose of international comparability, within the context of the organising framework (see Annex2).

To make concrete recommendations to the PEB Governing Board on an appropriate“method package”– incorporating a range of suitable methodsthat respond to specific research questions and policy issues – that could be implemented at an international level.

Participation in the working group

This group will comprise a small number of experts, many of whom participated in one or both of the experts’ group meetings on evaluating quality in educational facilities in 2005. PEB Governing Board members are asked to support the participation of the expert in the working group meeting.

Format of the meeting

The meeting will be divided into four sessions. Each session will commence with a short issues paper (10 minutes) delivered by one or two of the experts, followed by a group discussion. If necessary, experts will form smaller groups at the end of each session to formulate recommendations to the PEB Governing Board.

Issues

The meeting will address the following topics and related focus questions:

  1. PEB Organising Framework and Matrix for Evaluating Quality in Educational Facilities
  • Is the existing organising framework and matrix an acceptable conceptual model for evaluating quality in educational facilities? Can existing methods fit into this matrix?
  • Which educational policy issuesshouldaspects of thisframework address?
  • How can the results of evaluation feed back into the design process?
  1. Developing Common User-Centred Tools for Evaluating Quality in Educational Facilities
  • Given the range of effective user-centred methods for evaluating quality in educational facilities– interviews, walkthroughs, focus-groups, questionnaires, etc. – can a common approach be developed that measures the needs and satisfaction of stakeholders in differentcountries? For example, can a facilities-related questionnaire for 15-year-old students developed within the framework of the Programme for International Student Assessment (PISA) demonstrate a positive relationship between the quality of educational facilities and student performance?
  • How could such an approach be designed, in terms of its objectives (e.g. to collect examples of best practice on facilities that respond to users’ needs; to collect comparable data on building performance and functionality), research methods and tools used, actors in the study, development and implementation time and cost, outputs, etc?
  • What are the potential benefits and constraints in implementing such an approach?
  1. Developing Quality Indicators on Educational Infrastructure
  • What are the existing international data sources on educational facilities (e.g. PISA, administrative-, state-/province- and local-level data collections)?
  • What (potentially internationally comparable) indicators can be generated from existing sources that address issues of quality in educational facilities?
  • What policy issues could such indicators address?
  1. Developing International Quality Benchmarks on Educational Infrastructure
  • Is it feasible or desirable to establish benchmarks of “quality” in educational facilities(e.g. school construction and maintenance standards, student capacity regulations for classrooms and schools, space standards, comfort and health norms (i.e. acoustics, hygiene), security regulations)?
  • What are the constraints in developing such benchmarks? What type of international benchmarks (if any) could be established?
  • What is the real and potential policy impact of establishing standards, guidelines or benchmarks of quality in educational facilities at an international level?

Issues papers

To initiate, stimulate and focus discussion, experts will be asked to prepare a short (500-word) issues paper. The paper will addressone or more of the focus questions in the sessions described above.

Working language

The working language for the meeting is English.

1

Annex 1. Organising framework for evaluating quality in educational facilities

Defining “quality” in education facilities

Experts agreed on the following definition of “quality” in educational facilities:

“All individuals have a right to a quality educational facility, a physical space that facilitates the learning process and demonstrates cost-effectiveness over time; one that respects and is in harmony with the environment; and one that encourages social participation, providing a healthy, comfortable, safe, secure and inspirational setting for its occupants.

Universal parameters– reflecting temporal, local and concrete quality dimensions– such as flexibility, sustainability, and preventive and corrective maintenance can be used to evaluate the physical space that defines a school.”

Identifying principles and criteria for evaluating quality in educational facilities

The seven broad principles, or base guidelines, of “quality” in educational facilities identified in the first experts’ group meeting were refined in the second experts’ group meeting as follows:

  1. Fit for purpose

Objective. Does the facility meet the general requirements of a school (i.e.all facilities should be flexible, multi-purpose, inclusive for people with mild to moderate special educational needs, age-appropriate, and effectively and holistically planned and managed)?

Subjective. Does the facility meet the needs of the users? Does it encourage community involvement/access? Are the school community and others involved in the design, planning, and day-to-day management, maintenance and use of the facility?

  1. Inspirational and symbolic

Does the facility have character and display architectural, educational, socio-cultural (i.e.impact of local culture) or technological ambition (i.e.ICT)/innovation?

  1. Environmentally sustainable

Does the facility demonstrate environmentally responsible site planning and use of environmentally sustainable materials and construction methods? Does the building demonstrate efficient use of water, energy, recycling and waste management, and daylighting?

  1. Healthy and comfortable

Does the facility meet health standards? Does it meet with visual, acoustic, ergonomic and thermal standards- is it comfortable?

  1. Safe and secure

Does the facility meet security standards? Is the facility resistant to natural disasters?

  1. Cost-effective

Are initial investments in capital, maintenance and repairs, and operations and staff cost-effective in the long term?

Identifying effective methodologies for evaluating quality in educational facilities

Experts also identified a number of successful approaches, involving both qualitative and quantitative research methods, for evaluating the quality of the built environment:

Post-occupancy evaluation, which has been implemented in Australia, Brazil, NewZealand, Portugal, the UnitedKingdom and UnitedStates, uses a variety of qualitative techniques such as interviews, observation, walkthroughs and focus groups by one or several evaluators, with a high degree of stakeholder involvement.

Inventories of school accommodation have been established and indicators generated to identify areas of greatest need in countries such as Greece and Portugal.

Standards or norms are being developed to ensure adequate standards of quality in new building construction, maintenance and equipment in countries such as El Salvador and Mexico.

Although there is a growing amount of literature on the relationship between pedagogy and design of learning environments, further research is required to better understand how to measure the educational effectiveness of the built environment.

While there is no single “best approach” for evaluating quality, experts stressed that several broad guidelines must be followed when defining and implementing an evaluation methodology. The approach used must be:

Multi-disciplinary and inter-disciplinary, bringing together experts and others from the fields of education, architecture, engineering, child psychology, anthropology, environmental psychology, sociology and public policy.

Participatory, ensuring early and continuous feedback with policy-makers, students, teaching and non-teaching staff, parents, educationalists, financial bodies, architects, the media, facilities and asset managers and researchers.

Holistic, providing a systemic and integrated vision of a built environment throughout all phases of the facility’s− from planning to project delivery to occupation and management− and society’s life cycles.

Multi-method, incorporating aspects of appropriate qualitative and quantitative methods into the research design.

Purposeful and valid, having a process, a timeframe and research questions that are clear and objectives that are tangible and ever-present. Evaluators must be neutral, trained and accredited by external authorities.

Policy-oriented. Evaluation must respond to and influence policy.

Recognising parameters

An understanding of the relationship between architecture, educational effectiveness and educational policy is fundamental to the establishment of any framework or methodology that addresses the evaluation of educational facilities. To investigate how these three domains inter-relate, a number of variables must be considered:

Audience. i.e.How do students, teachers, non-teaching staff, school administration, facility and asset managers, the community, the media, educationalists, academics, architects, researchers and policy makers define a quality (built) learning environment? What is their role in evaluating the design quality of a facility?

Existing design standards, guidelines and data. i.e.Facilities should also be evaluated against existing design principles, standards, norms, legislation, regulations and guidelines established by educational authorities. Existing data or inventories of educational facilities should also be analysed against these standards.

Temporal factors. i.e.Could the framework and methodology be applied for both existing and new educational facilities, and for all phases of the building’s life cycle?

Spatial factors. i.e.How can a framework and methodology consider the use and function of educational space (fixed vs flexible infrastructure, specialised vs multi-purpose teaching space and shared vs non-shared teaching space)?

Educational factors. i.e.Would the framework and methodology consider all levels of education and institutions: pre-primary, primary, secondary, post-secondary and tertiary?

Other contextual factors.

Pedagogical/organisation of learning factors (e.g.student-centred vs teacher-centred learning, didactic vs interactive learning, fixed curriculum vs flexible subjects).

Economic factors (i.e.level of capital investment).

Historical (e.g.historical buildings, heritage areas) andphysical/geographical (e.g.urban vs rural, local vs regional vs national) factors.

Political factors (e.g.centralised vs decentralised systems).

Socio-cultural factors.

Human/community factors (e.g.levels of community commitment).

1

Annex 2.Matrix for evaluating quality in educational facilities

Point at which quality will be evaluated
Policy issue / Principle / Element / Pre-occupation / Occupation
Pre-design stage
(i.e.involving architects’ brief, design-safety guidelines, national educational policies and programmes,etc.) / Design stage
(i.e.participatory stage) / Construction phase / New buildings (in use less than 5 years) / Older buildings (in use more than 5 years)
(involving local, regional and local level upgrading, remodeling, refurbishing)
Educational / Fit for purpose: / Does the facility meet the general requirements of a school? Are spaces:
effectiveness / Objective / Flexible?
Multi-purpose?
Inclusive for people with mild to moderate special educational needs?
Age appropriate?
Effectively and holistically planned and managed?
Fit for purpose: / Does the facility promote social cohesion? Does/is it:
Subjective / Satisfy the needs of its users?
Open to the community?
Involve school community and others in the design, planning, and day-to-day management, maintenance and use of the facility?
Inspirational and / Architecturally innovative?
symbolic / Educationally innovative?
Technologically innovative (ICT)?
Socio-culturally innovative?
Environmentally sustainable / Environmentally responsible in terms of site planning?
Use water, energy, recycling, waste management and daylighting effectively and efficiently?
Use sustainable construction methods and building products?
Healthy and comfortable / Hygienic?
Ergonomically comfortable?
Thermally comfortable?
Visually comfortable?
Acoustically comfortable?
Safe and secure / Secure?
Resistant to natural disasters?
Cost-benefit / Long-term cost-effectiveness of initial facility investments, in terms of / Capital costs?
Maintenance and repairs?
Operations and staff costs?

1