Is this the worst education research ever?

Probably not, but it’s pretty bad.

Every so often my interest in exclusions leads me to some piece of “research” that mentions them. Usually, I can spot a few false or misleading statements that have apparently not concerned peer reviewers. Sometimes there’s something bad on every page. Most recently I found a peer-reviewed paper where it seemed like almost every sentence I read had something wrong with it. I thought I’d share some of the highlights with you.

The paper is about internal exclusion. It’s entitled

Students’ experience of isolation room punishment in UK mainstream education. ‘I can’t put into words what you felt like, almost a dog in a cage’

This should give you some idea of the level of objectivity maintained by the authors. But the problems go far beyond bias. The authors appear to be struggling with some pretty basic aspects of how the education system works and how to write research. For instance, they repeatedly write “UK” when referring to material relating only to England. They struggle to decide how to refer to what it is they are studying. Again and again, they could use the term “exclusion room(s)” or “use of exclusion rooms” but instead use a term that sounds more sinister. We see:

• “isolation room punishment”
• “isolation rooms”
• “internal classroom exclusion programmes”
• “isolation room discipline”
• “Isolation room exclusion”
• “internal isolation rooms”
• “Isolation units”
• “internal exclusion to an isolation unit”
• “the utilisation of isolation units as a form of punishment”
• “isolation”
• “internal isolation units”
• “isolation punishment”
• “isolation room exclusion as a form of school discipline”
• “internal exclusion policy”
• “internal exclusionary practices”

These are authors who know 15 ways to make being sent to sit at a desk sound sinister but don’t know that England isn’t the UK.

But the main problem isn’t the bias and ignorance. It’s the indifference to factual accuracy, particularly in the use of cited sources. The only way to show how bad it is is to go through some of the text in detail, checking references, and looking at the claims that are made. So let’s look at the introduction.

This is how it begins:

Fixed-term and permanent exclusions were introduced in the UK in 1986 to be used as a last resort solution to disruptive behaviour in schools in the face of threats of serious harm to the pupil or others (DfE Citation2017).

This citation is the entire exclusion guidance. It makes no mention of fixed-term and permanent exclusions being introduced in 1986, because, of course, exclusion from school existed before then.

Heads of schools can remove a student for 45 days each time; however, there is no limit on the number of multiple exclusions a student can receive.

No source is given for this, but it is incorrect. 45 days is the total number of school days a pupil can excluded over the course of a year, not the maximum length of a single exclusion (see page 11 here).

In 2019, the Timpson Review was commissioned by the Secretary of State for Education to review exclusion practices in schools in the UK and found that across three cohorts 71 children had received more than 50 exclusions during their school life (Citation2019).

The link in this citation is to a search page, not the Timpson Review which can be found here. Literally, the first thing said in the Timpson Review is that it was commissioned in March 2018. And, of course, it did not cover the whole of the UK. The 71 children statistic itself seems correct, but if anything it shows how rare it is for a pupil to receive that many suspensions.

The Department for Education (DfE) report an increase in students identified with behavioural difficulties rising from 1.7% in 2004–2.1% in 2011 (DfE Citation2011).

“This page cannot be found” when you follow the link. I think it’s possibly this document, but I can’t find that statistic.

However, a clear picture of the extent of behavioural problems in schools appears to be distorted due to the broad spectrum of behavioural challenges and the lack of available data (Ball et al. Citation2011).

That article doesn’t seem to discuss the availability of behaviour data.

The only national data available is an Ofsted evaluation of student behaviour where 92.3% of schools in England were given a ‘good’ or ‘outstanding’ grade for student behaviour with only 0.3% judged at ‘inadequate’ (Ofsted Citation2018) and 70% of teachers surveyed reported generally good behaviour (NFER Citation2012).

Obviously, this was not the only behaviour data available. Surveys of teachers about behaviour are fairly frequent. Many do not present such a rose-tinted view of behaviour.

In spite of this, there has been an increase in Government policies and initiatives addressing disruptive behaviour in UK schools (Shaughnessy Citation2012).

This article is obviously from 2012, so I am not sure how it could support such a claim made in 2020. I am not aware of any period where behaviour was not a big issue in schools.

These policies are concerned with the development of a culture of respect and maintaining authority and control as demonstrated in the Education Act (Citation2011) and the Steer Report (Citation2005) (Ball et al. Citation2011).

The Ball citation here seems essentially random. Does nobody reference specific pages any more?

However, there appears to be no clear evidence of the efficacy of this abundance of behaviour policies, in fact, current thinking suggests that UK Government guidelines on behaviour are exacerbating the situation for some schools (Nash, Schlösser & Scarr, Citation2016).

Is there a more weasel word than “suggests”? Is it true, or isn’t it? In this case, the “current thinking” appears to be the opinions of some educationalists who are concerned that too many teachers in a self-selecting online survey disagree with them.

And that’s the end of the introductory paragraph. One might be wondering what it has to do with “isolation room punishment”, but the big issue for me is how so many inaccuracies and dodgy citations can get into one paragraph. The obvious answer is that the process of peer review is utterly meaningless. Nobody cared whether the article was factually correct or whether the links worked.

Obviously, I could continue checking every fact and looking up every citation, but I doubt it is likely to improve after the first paragraph. Some links are almost amusing in how inadequate they are for the claims they are meant to support. For instance, the article makes this dramatic claim:

“The implications of perpetuating an unjust system of social exclusion are far reaching, potentially impacting students’ psychological and physical health”.

This is followed by a link to an experimental neuroscience article in which social exclusion is simulated by playing a computer game where the player didn’t have the ball passed to them often enough by “virtual players” and was then given a brain scan. Did the author truly believe this was relevant? Did they just do a search for “effects of social exclusion” and link to the first thing they found? Did a peer reviewer actually follow the link? Did the peer reviewers check any of the citation links?

I know it’s nothing new to comment on the quality of education research, but every so often I find something that still scrapes the bottom of the barrel. This has pretty much all the flaws you could hope for. As well as a quotation in the title that would be more appropriate for a tabloid newspaper; statistics quoted from websites; citations that cannot be found, and a complete lack of objectivity, it also credulously repeats implausible anecdotes from pupils:

It’s terrible because I missed out on a GCSE because of it. We were learning about the Albert Dock for geography and because I didn’t go there ‘cos I weren’t allowed to go down to the Albert Dock with the teachers and with the class and that like … ..when they came back I didn’t know anything about the Albert dock not into that much detail for the GCSE and that was like half of the course.

It also makes no practical suggestions for what could replace internal exclusion:

Actively listening and tuning in to how children feel about the everyday practices that impact their emotional well-being can be the first step for educators to shift the dominant discourse because ‘truths don’t just happen, they are produced in our struggle to decide the meanings of our actions, thoughts and feelings’ (MacNaughton Citation2005, 21). For educators, this begins with a reflection on how we ‘struggle over and against what it is we have become, what it is that we do not want to be’ (Ball Citation2016b, 1143) to ensure that all children, especially the most vulnerable, have access to a fully inclusive education. Changing the practice of unethical disciplinary measures may be difficult within a culture of standards and performance (Ball Citation2016b).

And, of course, there’s no new knowledge in it, just opinions. If you want to show that the study of education falls well below the standards of scholarship you’d expect from undergraduates in an academic discipline, this would be the perfect teaching tool. One can only wonder how this stuff gets written and then published.