Were English pupils’ scores artificially inflated in international PISA tests?

Martin Shipton

Doubts have been cast on international test results that show English teenagers performing significantly better at reading, maths and science than their counterparts in Wales.

Earlier this month the 2025 international PISA results were published, allowing comparisons to be made about performance in these key subjects in schools across the world.

According to the results, which are collated by the Organisation for Economic Co-operation and Development (OECD), Wales was the worst performing country among the four UK nations.

But a prominent academic has now published a critique of the results for England, suggesting the system may have been gamed by excluding pupils with lower attainment levels.

Professor Paul Frijters, a research economist who has worked at various universities including the London School of Economics and is the co-founder of an academy in Belgium called Academia Libera Mentis (ALM), has published a provocative essay on Substack called Lies, Damn Lies and Education Statistics.

In it, he writes: “A good rule of thumb to discover data manipulation is to look at something that seems too good to be true and delve into that case. For the 2025 PISA results, the ‘too good to be true’ prize must go to England.

“How would anyone know whether the schools and education authorities in England have manipulated the data? Of course, a first instinct many have is to first ask AI, which I did. Google AI duly reported that nothing is amiss and England has achieved a wonderful success. That is what AI (roughly) said within two hours of releasing the PISA reports no less, which suggests a wonderful marketing effort behind the scenes to pre-empt criticisms. Other AI was not much different.

“But we don’t trust AI at ALM with anything politically or commercially sensitive, so I dug deeper. In order to minimise the effort I had to make to get onto the right trail, I asked AI who had been critical of the previous PISA results in England in 2022. I got a name: Professor John Jerrim. Ok, then I looked up what he said just after the release of the 2025 results (I put his name into X). He tweeted the Table below, which is a table appearing on page 184 of the England-specific report, meaning you will not find it in the 377 page main report of the OECD, but you really have to go to the government webpages of the UK and then fork out the England report on its website.

“The key data in the report that John noted are in the table below, which gives two main indications that, indeed, the England PISA results are completely unreliable.

“For one, look at the last row of this table, saying that 1.4% of the participating students had an ‘Education, Health, and Care plan’ (EHC), versus 7.1% of the non-participating students. Below the table it is clear that there were 5,209 participating students, 1,651 non-participating ones, and 410 students who could not be matched. Ok, what do these two bits of information tell us? Well, EHC students will be the worst performing students: those are the ones who can’t sit still, have severe learning difficulties and have major health problems. They will normally score very low. In the group of 1,651 students who did not do the final PISA tests, 7.1% was in this low-performing group, five times more than the 1.4%: the final group doing the PISA tests has far fewer of the low performers than the group that for ‘some reason’ did not do the final test. Convenient, no?

Background check

“Of course, one then does a background check. Does the 1.4% line up with national data? No. It turns out that in 2025 5.3% of UK school children have EHC plans, hence more than three times more than in this PISA data. Something similar holds for the broader category of children who have difficulties: those with SEN support, where SEN means ‘Special Education Needs’. The country has 19.5% of those in schools, whilst the final sample in the PISA only has 10.7% (and the non-participating sample 25%). So the English schools in the PISA see a very heavy selection against poorly performing kids. Basically most of the bad performers were not doing the tests …Perchance the teachers told the worst performers it was time to be sick on the days of the test? It is an old trick, but it works.

“A final preliminary test then: how does the bottom in England match up against other countries? It turns out that the bottom 25% in England did as well as the middle persons in Wales. And in Wales, if I look for selectivity of a poorly performing group, I find on page 176 of their report that the proportion of particularly needy students (called ALN in Wales) is 8.6% for the final group of participating schools, compared to about 10% for Wales as a whole, which is a much smaller difference than 1.4% versus 5.3%. Hence, either Wales is full of particularly poor performing students, or the English data is particularly massaged at the bottom. No prizes for what seems most likely!

“I decided to look deeper into the issues. To start with, I dug a bit more into that England-specific report, finding a few very fishy things indeed.

“For one, I asked myself who actually put the data together. In several countries it is a national statistical agency or something close to that. Those are usually the best ones because they have a lot of reputation at stake across a huge number of statistics, and of course they have a lot of professional statisticians in their employ.

“In the UK, although the final reports come from the Department for Education, the data was gathered by Pearson, a private company. Oh oh. Worse, Pearson is a private company that sells a lot to education authorities in England (about 90% of the UK population is in England, so the English are the client to please, not the 10% in Wales and Scotland).

Red flag

“That is a major red flag: the people gathering the data cannot reasonably be considered an uninterested party. The dangers of having a private company gather sensitive data for a government department and the whole system it does other work for is of course widely known in the literature on corruption (on which I have written two books): a lot of dirty work is by companies that do deals with governments. Deniability and defamation laws make it an attractive avenue. Of course I wouldn’t dare to accuse Pearson of anything, just noting possible motives known from the literature.

“Secondly, I crawled a bit more into the data the report provides at the back about sample selection of schools and pupils. I noted that 24.1% of the participating schools had a missing ‘attainment band’ (Table A5), whereas in principle every participating school should have one. This is a huge degree of missing data, always a red flag if one is looking for data manipulation because the person setting up the data can play with that missing category, such as by selecting top schools and pretending their attainment band is unknown (which you can always justify by saying it is not known for some particular year). Also possible is to rummage deliberately in the set of schools with missing attainment bands and select the ones you like. The 24.1% thus suggests an enormous amount of discretion on the part of the commercial company doing the data gathering. I note that in Wales only 9% of the PISA schools had a missing attainment band (the unnumbered table on page 173).

“Then I noted something similar with the students. It is the case that the participating schools in the final sample have better students than those who dropped out of the sample (Table A.4), that these schools did better than the average in England as a whole, and that the better schools had more pupils doing the actual PISA tests than the less good schools. How can you see this? Well, you look for the national average from the government itself and compare it with what is in the PISA 2025 report. In the latest reported data from GOV.UK (for 2022/2023), 45.4% of pupils achieved a grade 5 or higher in both English and Maths; versus 47.1% with grades at least 5 in the weighted final PISA sample. It is not a huge difference, but a significant shift somewhat on top of the EHC selectivity.

Rule-of-thumb

“Let us then do a bit of ‘magical’ rule-of-thumb statistics to see how big the effect of selectivity might have been. If I interpret the 1.4% versus 5.3% EHC selectivity as a 3.9% shift upwards in the distribution of students, and add the 47.1% versus 45.4% selectivity as a 1.7% further shift upwards, I get a 5.6% upwards shift in the distribution, basically as if I took out the worst 6 students of every 100. If you then note in the report itself that the range between the 10th percentile and 90th percentile in science is 264 (averaged over the genders), meaning that an 80% shift in the distribution is worth 265, then that 5.6% shift is worth 18.55 points in those PISA scores. An 18-point shift downwards would put England on the same level as Scotland and in line with the OECD trend.

“Are there reasons to suspect that more than 18 points might be involved in the sample selectivity? Yes, indeed. Whilst the above is about selectivity of the bottom students, there is also the question whether the data involved particularly good schools that shifted the top. In this regard, I note that the 90th percentile results in England show a spectacular increase in science test results (making them better even than 2018!), though not an increase in reading and mathematics results for that same 90th percentile. Very fishy.

“My suspicious mind then wonders how one could smuggle in a few top science schools into the sample. If I take out the additional 10 points specific to the miraculous increase in (science) results between 2022 and 2025, on top of taking out the 18.55 points for sample selectivity at the bottom, then the schools in England suddenly look just as good as …. the schools in Wales. Well, I never! Isn’t that a coincidence?”

Dataset

We asked the UK Government’s Department for Education and the Welsh Government to comment on the article.

A spokesperson for the Department for Education said: “OECD decides which datasets to use for PISA comparisons and they are responsible for how the comparisons are made across countries, so this isn’t one for us to comment on.

“Our data on GOV.UK isn’t just used for PISA; it’s used for various things as you would imagine, so I’m confused how it can possibly be artificially inflated when that isn’t the sole purpose of datasets…”

A spokesperson for the Welsh Government said: “International assessments apply common standards across participating countries. Our priority is to understand the evidence and drive the improvements that will help every learner in Wales reach their potential.”

---

Get more trusted Welsh news

Choose Nation.Cymru as a preferred source in Google News to see more of our journalism.