Support our Nation today - please donate here
Opinion

Wales’s Pisa school test results have declined – but it’s not a true reflection of the education system

16 Dec 2023 5 minute read
Exams

Alma Harris, Professor of Leadership in Education, Cardiff Metropolitan University

Every three years, an early Christmas gift arrives for the global education community from the Organisation for Economic Co-operation and Development (OECD).

The Programme for International Student Assessments (Pisa) is an international test in which 15 year olds are tested on their knowledge and skills.

It relegates those far below the Pisa top ten as poor performers in desperate need of improvement, which this time includes Wales.

The Pisa scores for participating education systems around the world are unquestionably significant. But since its inception in 2000, Pisa has sparked much debate, especially among experts and policymakers, with many viewing it as a flawed assessment of educational outcomes. In 2018, around 600,000 students took part in the standardised Pisa tests, which measured their performance in maths, science and reading, and also looked at wellbeing.

Predictably, the 2023 Pisa results captured the negative impact of COVID on learners and learning, with some downward trends in performance visible across the data set.

The results signalled mixed fortunes for the UK. The BBC headline, reporting the results starkly stated: “Wales slumps to worst school test results.” Such sweeping statements are by now an anticipated byproduct of Pisa that ignore how the tests are often highly contested and controversial.

Pisa in Wales

Every three years, Pisa measures the ability of 15 year olds to use their reading, mathematics and science knowledge and skills to meet real-life challenges. In Wales, 2,568 pupils from 89 schools took a two-hour computer-based exercise. To put this in context, there are approximately 33,000 pupils in Year 11 in 178 secondary schools and 27 middle schools in Wales.

The subsequent OECD report acknowledges that “the sample for Wales, and for many other countries, did not meet some of the Pisa standards”. It is important to reflect on how a test taken by a sample of 15-year-old students, every three years for two hours, can possibly be a valid and reliable measure of a system’s performance even in a relative sense.

Pisa’s statistics show that Wales’s average score for mathematics in 2022 was significantly lower than the average across OECD countries. Wales’s average scores for mathematics, reading and science have all declined significantly since 2018. This was also the case, on average, across OECD countries for mathematics and reading. Although for science, the difference between the OECD average in 2022 was not significantly different to that in 2018.

It also noted that the gap in performance between pupils from the most disadvantaged backgrounds and the least disadvantaged backgrounds was smaller in Wales than it was on average across OECD countries for all subjects.

The important thing to observe is that Pisa deals in averages. In the latest results, those averages are derived from the 81 countries that took part, which is a huge range. The report notes a relative fall in Welsh performance against an aggregated average of OECD countries.

It then highlights that this decline was also the case on average across OECD countries for mathematics and reading. In other words, this is a trend. It also suggests that Wales has been more successful in closing the achievement gap between the most disadvantaged and least disadvantaged pupils than most other OECD countries. Yet this important indicator of success has been overshadowed by the blanket headlines of abject educational failure.

If all countries participating in Pisa now recover fully after COVID and improve their educational performance across the board, it is highly likely that Wales will “underperform” on Pisa yet again, whatever it does. If all countries in Pisa continue a steady trajectory of improvement, the country differentials will remain largely the same. Some countries may move up or down, but that movement will be marginal.

A game of relatives

There were no real surprises in the latest results. Countries like Singapore, Taiwan and Japan have retained their comparative advantage and will probably continue to do so, because Pisa is a game of relatives. The complexity and dynamic of any education system cannot (and should not) be at the mercy of a single measure of assessment however compelling or lucrative.

But what does that mean for Wales? First, it should encourage us to look at Pisa as one data set only and to not be obsessed by its findings. It is important to put Pisa in perspective by looking far beyond the simple headlines and delving into the detail of the report.

Second, it should be a reminder that Pisa is a snapshot of performance at a particular moment in time. It takes no account of the possibilities and potential of ongoing reform – deep contextual detail is not on its global radar. Hence, the danger is that every three years, Pisa fuels doubt, dissent and concern, when education systems need certainty, confidence and consensus about the reforms they are putting in place.

Third, no education system is perfect. Getting great Pisa scores is certainly no guarantee that the wellbeing and mental health of children and young people is not compromised or sidestepped along the way. The potential of human collateral damage in achieving high Pisa performance over two decades, unsurprisingly, does not feature in the OECD reports.

Wales has a choice, to either let this global compass direct its educational pathway, accepting that every three years it will derail and disrupt the reform agenda, or to hold its nerve. We cannot ignore Pisa, but we can put it in perspective and continue to focus on the learning and wellbeing of all children and young people in Wales. This is what matters most.


Support our Nation today

For the price of a cup of coffee a month you can help us create an independent, not-for-profit, national news service for the people of Wales, by the people of Wales.

Subscribe
Notify of
guest
8 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Another Richard
Another Richard
2 months ago

The PISA process may be flawed – what complex human-designed system is not? – but it provides the best benchmark we have for assessing the quality of education in Wales. This article fails to note the human collateral damage associated with poor performance in reading and mathematics, where a lack of proficiency impedes full participation in society makes people vulnerable in many ways. Children’s “wellbeing” cannot be separated from their educational progress. I fear we will see another lurch downwards in three years’ time following the introduction of the new curriculum.

Alwyn
Alwyn
2 months ago

I would agree with ‘another Richard’ IF the new curriculum is trsted by the old assessment system

Alwyn
Alwyn
2 months ago

If Pisa reflected what is taught, and how it is taught, in Welsh schools it might show better results. It doesn’t. That is because Pisa tests problem-solving, reasoning and thinking, basic numeracy and literacy, whereas the Welsh system is geared towards passing examinations/ tests at 5, 7, 11, 14, 16, 17 and 18. Teachers by now are excellent at getting pupils to jump through hoops that the examination boards set them -AND/ OR examination boards consistently reduce pass mark for different grade boundaries, especially in numeracy. NO other countries in the world apart from England and Wales test as often… Read more »

tulip
tulip
2 months ago

welsh education massively underperfomring england last twenty years. as has the nhs. wonder what has caused that???

max wallis
max wallis
2 months ago

All scores were hit by Covid. But the Welsh Govt’s disregard of the harm to children’s schooling was severe, almost absolute. “Safety First” was Drakeford’s mantra, no balancing with lockdown harms. Applied to children even though they hardly suffered from the virus. .

Thompson
Thompson
2 months ago

The journey of Nation Cymru from a semi critical news outlet to the Welsh version of Pravda is consistently proved by its desire to project every abject failure of the Welsh Government in a positive light. This article by Alma Harris is a classic handwringing from a person wholly invested in the current system and some might argue partly responsible for the mess. She again wheels out the classic excuses to justify abysmal performance drawn from very PISA denier manual that circulates Welsh schools as the test results are published. But as the Pisa lead Andreas Schleicher states “It’s not… Read more »

John Williams
John Williams
2 months ago

Comparisons between Welsh and English school (and indeed NHS) performance are largely meaningless unless adjusted for differing levels of disadvantage. More validity would be provided by comparison with regions of England sharing similar levels of disadvantage.

Peter
Peter
2 months ago

So the rest of the world is wrong and Wales is right, is that why we are the poorest, country struck on benefits with the worst nhs?

Our Supporters

All information provided to Nation.Cymru will be handled sensitively and within the boundaries of the Data Protection Act 2018.