Resisting the Seduction of the Global Education Measurement Industry:
Notes on the Social Psychology of PISA
Gert Biesta
Department of Education, Brunel University London
e-mail:
Author details
Gert Biesta
Department of Education
Brunel University London
Uxbridge
UB8 3PH
United Kingdom
Resisting the Seduction of the Global Education Measurement Industry:
Notes on the Social Psychology of PISA
Gert Biesta
Department of Education, Brunel University London
e-mail:
Abstract
The question I raise in this paper is why measurement systems such as PISA has gained so much power in contemporary education policy and practice. I explore this question from the bottom up by asking what might contribute to the ways in which people invest in systems such as PISA, that is, what are the beliefs, assumptions and desires that lead people to actively lending support to the global education measurement industry or fall for its seduction. I discuss three aspects of what, in the paper, I refer to as the ‘social psychology’ of this dynamics, highlighting the seductive nature of numbers, measurement and comparison, the persistence of technological expectations about education and its workings, and the reference to social justice as a key motivator for wanting to know how systems work and perform. I raise critical questions with regard to each of these aspects and, through this, suggest ways towards a more grown-up response to the difficult question of providing good education for everyone rather than engaging in an unsustainable race for the top.
Keywords
PISA, Measurement, Social Psychology, Education Policy, What Works, Social Justice
Introduction: The Global Upscaling of Education Policy
In his 1987 book The Struggle for the American Curriculum (Kliebard 1987) Herbert Kliebard presented the forging of the American curriculum as the outcome of a struggle between a range of groups and parties that all had different interests in what the curriculum was supposed to represent and bring about. He thus showed that the curriculum should never just be understood in rational terms – as an answer to Herbert Spencer’s question ‘What knowledge is of most worth?’– but always also as the result of social and political struggle. If Kliebard still worked in a tradition that saw the curriculum mainly as a national project, the contemporary dynamics of curriculum making are increasingly taking place at a transnational and global level (see, for example, Priestley and Biesta 2013). It thus becomes important to ask what the dynamics of the struggle for the ‘world curriculum’ are, also because at the level of official politics, curriculum matters tend to remain the prerogative of national governments.[1]
One important factor in the recent global ‘upscaling’ of education policy is the role of transnational players (see also Rivzi and Lingard 2010). Whereas some of these players – such as, for example, the World Bank – tend to intervene quite directly in national educational policy, often because policy and money come as one package (see Ball 1998 for an early analysis of these dynamics), the more remarkable impact has been the result of measurement systems that, at least in their stated intention, did not seek to influence or change education policy directly but ‘just’ wanted to provide information upon which national education policy makers could make decisions about the desired shape, form and direction of their education system. Although such systems have been around for a while – IEA, the International Association for the Evaluation of Educational Achievement responsible for such studies as TIMMS, PIRLS and CIVED, emerged in the late 1950s and became a legal entity in 1967 – PISA, The Programme for International Student Assessment, run by the OECD since 2000, has not only become the most visible of these systems but most likely also the most influential (see Hopman, Brinek and Retzl, 2007; Pereira, Kotthoff and Cowen 2011; Pons 2011; d’Agnese 2015).
The OECD, the Organisation for Economic Development and Cooperation, was founded in 1961 with 18 member states plus the USA and Canada, and has over time grown to its current size of 34 member states. Its stated ‘mission’ is “to promote policies that will improve the economic and social well-being of people around the world” (OECD website; accessed 17 July 2015).[2] The following statement provides a comprehensive overview of what the OECD is, how it sees it remit, and what its main areas of concern are.
The OECD provides a forum in which governments can work together to share experiences and seek solutions to common problems. We work with governments to understand what drives economic, social and environmental change. We measure productivity and global flows of trade and investment. We analyse and compare data to predict future trends. We set international standards on a wide range of things,from agriculture and tax to the safety of chemicals.
We also look at issues that directly affect everyone’s daily life, like how much people pay in taxes and social security, and how much leisure time they can take. We compare how different countries’ school systems are readying their young people for modern life, and how different countries’ pension systems will look after their citizens in old age.
Drawing on facts and real-life experience, we recommend policies designed to improve the quality of people's lives. We work with business, through the Business and Industry Advisory Committee to the OECD, and with labour, through the Trade Union Advisory Committee. We have active contacts as well with other civil society organisations. The common thread of our work is a shared commitment to market economies backed by democratic institutions and focused on the wellbeing of all citizens. Along the way, we also set out to make life harder for the terrorists, tax dodgers, crooked businessmen and others whose actions undermine a fair and open society.
(OECD website; accessed 17 July 2015).
As significant issue for the focus of this paper is that the OECD sees itself as a forum where governments can work together to share experience and seek solutions. In this regard the OECD) has no direct power to set policy but works through a logic that is sometimes referred to as ‘soft law’ (see, for example Abbott and Snidal 2000; see also footnote 2).
Referring to soft-law as a ‘logic’ is meant to highlight that, unlike hard law which, amongst other things, can be enforced and comes with a system of sanctions, soft law ultimately depends on ‘investment’ from the bottom up. It depends, in other words, on the willingness of actors to actively invest in the suggestions, communications, interactions, and information generated and orchestrated by such organisations as the OECD. The power of soft law, to put it differently, is predominantly rhetorical, and although rhetoric can be described as the art of persuasion, persuasion is never a one-directional process but depends as much on the skills of the rhetoricians as on the contribution of their ‘audience’ – that is the degree to which audiences wish to invest their beliefs in what is on offer and act on those beliefs. It is, therefore, as much as matter of identification as it is a matter of persuasion.[3]
In this paper I would like to look at the phenomenon of PISA – which I take to stand for similar phenomena in the global upscaling of education policy – precisely with regard to the question why actors are willing to invest in PISA and everything that comes with it, thus giving it the kind of power it appears to exert on current education policy in many countries around the world. I refer to my particular interest as that of the social psychology of PISA and similar phenomena, as I’m interested in the perceptions, beliefs and actions of those who in some way are at the receiving end of PISA. I want to shed some light, in other words, on the psycho-social dynamics around PISA, particularly because I assume that the power of PISA has a lot to do with expectations about education and its manageability that have been around in education for a long(er) time – expectations that we could also see as desires about education and its manageability (on this see also Meirieu 2007). It is the persistence of some of these expectations and their underlying desires that is interesting as well in explaining the dynamics around PISA and similar systems (see d’Agnese 2015; see also Au 2011; Alexander 2011).
The paper is organised in an introduction, three sections and a conclusion. In the first section to follow I discuss questions about the purpose – or as I prefer: telos (Biesta, 2015a) – of education, which has to do with the wider theme of values and measurement. Next I look at the issue of educational improvement, and particularly the enduring existence of technological expectations about education. Thirdly I raise some questions about the social justice rationale for systems such as PISA, highlighting the tension between beliefs and social structures and arrangements. In the concluding section I return to the idea of the social psychology of PISA and the socio-psychological dynamics surrounding it, suggesting the importance of distinguishing between what Bainbridge and West (2012, 6) have called the syntax and the semantics of education, and making a case for a grown-up response to PISA and similar systems.
Measuring What We Value or Valuing What We Measure?
The most visible way in which systems such as PISA are seductive is in that they seem to provide clear, unambiguous and easy to digest and to communicate information about the apparent quality of educational systems, particularly with regard to their ‘performance’ (put in quotation marks because the very idea that the quality of education systems has to do with its performance is already problematic – see below). Quantitative measures that can easily be transformed into league tables and into clear statements about gain and loss between different data-trawls which, in turn, provide a clear basis for policy makers to set targets for ‘improvement’ (see below) – such as gaining a higher place in the league table than apparent competitors, increasing national performance by at least a certain number of points, or articulating the ambition to score ‘at least above average’ – give PISA a simplicity that is absent in complicated discussions about what counts as good education. Yet there are obviously a number of issues here that need exploration and that have been discussed fairly widely in critical literature about PISA and similar systems.
One discussion concerns what I have suggested (Biesta, 2010a) to refer to as the technical validity of the measurements of PISA, which is the question whether what is being measured is an accurate representation of what is supposed to be measured. Next to the technical flaws that can be found in PISA’s methodology – a topic of ongoing discussion (see, for example, Goldstein 2004; Kreiner and Christensen 2014) – there is the in my view more important issue of what I have termed the normative validity of the PISA measurements, which has to do with the question whether what is being measured is focusing on an acceptable and justifiable operationalization of what education is supposed to be and do. One obvious issue here is that PISA only measures performance (or achievement) in a small number of school subjects (mathematics, science and reading) and only focuses on performance at one particular point on the educational career of young people (at the age of 15). While it may be useful to have such information available, even the suggestion that what PISA measures provides a valid indication of the quality of education – both as system and as practice – is difficult to accept. The fact that PISA seems to function in this way and is accepted as a good indication of the quality of education thus indicates, in my view, the seductive power of numbers and of league tables.
While the argument I am putting forward here (and have put forward elsewhere; see most notably Biesta 2010a; 2015a) is not an argument against measurement in education – albeit that there are issues about how strict we define measurement and what it is we seek to measure (see below) – the big question is whether we are measuring what we value about education, that is, whether we seek to establish whether education is indeed doing what we hope and expect from it, or whether we have created or are creating a situation in which we have come to value what is being measured. The power ascribed to PISA suggests that the latter situation is the reality we are currently in (see also d’Agnese 2015). In terms of the ‘social psychology’ that is at stake here, there is not only the seduction of numbers – that is, the idea, that numbers are more accurate and objective than say narrative accounts of education quality – but also, so I wish to suggest, the role of fear, and particularly the fear of being behind and the fear of being left behind. The latter fear explains the belief and investment in league-tables which, just in their form, seem to communicate the idea that some (some education systems, for example) are better than others and that – and the ‘and that’ is key here – those who do not perform in the way those at the top do, are lacking and are lagging behind. The question whether it would be desirable to perform as those who end up at the top of the league table is a question often not asked – having a league table seems to make it ‘obvious’ that the top position is the most desirable position – but should, of course, be asked, not only with regard to the question which measures identify a certain system as ‘top’ but also with regard to the question what it takes to end up at the top of the list and whether this makes it worth striving for the top position.[4]