h1

Another look at exclusions and race

November 17, 2019

About a year ago I wrote this post looking at the data on permanent exclusions and ethnicity.

I had often seen it claimed that the differing rates of permanent exclusions in different ethnic groups were evidence of racism and grounds to reduce, restrict or stop exclusions.

In that post I observed the following:

  • The numbers involved are actually tiny. This means that a) no ethnic group is affected massively by permanent exclusions, and b) if somebody claims there is racism driving these figures, they are talking about the actions of far less than 1% of heads, which could be individually scrutinised, not “institutional racism” or “unconscious bias” throughout the system.
  • While individuals should always be judged as individuals, there is absolutely no reason to assume that the extreme behaviours that result in exclusion will be constant across different demographics. Different ethnic groups do not have the same history, social class, religion, geographical location, type of school attended and so on. To assume the same behaviours from all demographics would be wishful thinking.
  • The differences between ethnic groups above, conform to no obvious pattern of widely held prejudices. Does anyone believe that black African children experience far less racial prejudice than black Caribbean children? Or that Pakistani Muslim children are subjected to less racial prejudice than white children?

Since then I’ve had a few more things to think about.

The first is that anecdotally, at least one teacher has told me of their experience of exclusions being affected by prejudice. This does not mean that it is common, but it is something that should not be dismissed even if it is unlikely to be affecting the averages. However, if it is the case that some school leaders let prejudice affect their decisions on exclusions, then this is not necessarily a case to make all heads less able to exclude. It would be a reason to make grounds for exclusion clearer and more objective. It should be possible to say, “Well, you didn’t exclude a white child for the same thing.”. However, if schools did set clearer grounds for exclusions, it is likely to drive exclusions up, not down. It is easy to make exceptions to keep kids in school, far harder to push them out with no good reason. Greater consistency in the use of exclusions would be a “no excuses” approach, not an “a lot more excuses” approach. There is also no reason to assume a reduction in exclusions would change relative rates of exclusions.

A further development is that the Timpson Review tried controlling for all other factors when looking at the different rates of exclusion between different ethnic groups.

The discrepancies between ethnic groups were massively reduced, with only 3 groups being more likely to be excluded than white children. This suggests my argument from the previous post was correct, it also reduces still further the number of heads whose decisions might have contributed to the disparities from a few hundred (out of tens of thousands) to a few dozen, making talk of “institutional racism” even less plausible.

Does the remaining disparity provide good evidence of racism? Not really. We still wouldn’t expect total uniformity of exclusion rates after controlling for all those other factors, even in the complete absence of prejudice. This research controlled for the data schools keep, but not for all possible factors. It won’t control for peer group or for crime in the local area, or any other factor schools don’t keep records on. Any imperfections in the data will also have an effect as controlling for other factors requires accurate data.  We still cannot assume racism is the explanation for the remaining disparities. What we do know from this, is that prejudice can, at worst, only be a marginal factor in permanent exclusions and cannot be a reason to restrict or reduce exclusions.

And finally, there is the issue of just how subjective attempts to prove prejudice from data can be. In a completely different context (football), research by Silberzahn and Uhlmann looked at how the same data can lead to drastically different conclusions.

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 (Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions.

How could teams of researchers differ in their conclusions, when looking at the same set of data? A lot came down to what controls were considered relevant and what statistical tests seemed appropriate. I don’t suggest that this makes it impossible to ever demonstrate prejudice. But if professional statisticians can differ as to what data shows, we should perhaps be more sceptical at the certainty with which the politically motivated draw simple conclusions about racism from raw figures. And that applies whether the conclusions are being reached about school exclusions, the level of diversity of speakers at a conference or in arguments about whether people’s lists of favourite tweeters are representative or not.

If people wish to believe that exclusions are evil, or that schools are institutionally racist, they will. But the evidence is simply not there in the figures for permanent exclusions, and there is absolutely no reason to think reducing, restricting or banning permanent exclusions will address any injustice.

h1

The Difference – An education charity that gets it very wrong about exclusions

November 10, 2019

As you may know, I have delivered a talk at some researchED events about the evidence that permanent exclusions are necessary and, if anything, they are a bit rarer than they should be. After the national researchED conference, my attention was drawn to the charity The Difference. Their Twitter bio now says:

Bringing the best in education to those children who need it most. Training specialist leaders in Yorkshire, North West, London & South England.

However, their original emphasis seems to have been on exclusions, and their bio used to say:

Fighting to break the link between social exclusion and school exclusion. Our mission: to place excellent teachers in alternative provision schools.

In July, the NAHT made it clear The Difference are trying to reduce exclusions.

The podcast describes “exclusions” as “top of the government’s agenda” and explains that the charity gives school leaders a placement in AP with, “the ultimate aim of reducing school exclusion”. (It also claims that, “every cohort of excluded kids costs the state an extra £2.1 billion” which is around £300 000 per child, something I’d love to see the source for). It is assumed throughout that permanent exclusion is something that happens to, “the most vulnerable students”, rather than in order to protect the vulnerable, and that exclusions should be reduced.

As a school teacher, I have experienced what happens when schools try to avoid excluding – teachers and students are abused, attacked and intimidated while those in charge congratulate themselves for not excluding.  I shared what other teachers experienced in this post “What happens when schools don’t permanently exclude?”. It makes terrifying reading.

The Difference website is even clearer about what they are up to. Their one and only blogpost is an article originally from the TES entitled “How to teach challenging pupils and never exclude them” and it quotes David Whitaker, who is the director of learning at a trust that doesn’t exclude, and who supports the work of The Difference:

He recalls how at both his first placement at the Priory School in Barnsley and his first job at Kingstone School in the same South Yorkshire town that he had found his calling.

“When I had that job in Kingstone, I was teaching 30 kids in a class. You very rarely had a teaching assistant and there was no behaviour structure or system that was there to rescue you. When the door closed behind you you were on your own. You didn’t have a behaviour structure to support you.

“What I had to do is earn the respect of the class, teach really, really well. And if those kids were running around on the desks I had to get them down on my own with my personality and they needed to want to get down off the desks.”

He describes the experience as being daunting but he also believes it is where he learned what he considers is essential to managing behaviour – building up relationships with the pupils. And he worries that some of this is being lost in mainstream education.

“What happens now is that if a kid runs around on the desk there is a button you can press and someone comes and takes the kid out.  At the risk of sounding controversial I think we are at risk of deskilling.”

Incredible as it seems that anyone could suggest that it is, “deskilling” to support teachers faced with a attempted riot, I am not surprised as I had previously read a Guardian article about Whitaker; in it he described the approach he used (not at a mainstream school) to his staff being repeatedly abused:

One 11-year-old girl, Whitaker recalls, responded with a curt “Fuck off!” every single morning for a year. That particular response would be met with instant isolation, detention or expulsion in many schools – but not at [this school]…

Instead of disciplining her, teachers paid the girl more positive attention in an attempt to understand the angst she was bringing from home. Within a year, she had stopped her morning outburst and got along with school staff….

….[the school’s] teachers vow to approach every child with what they call, “unconditional positive regard” – or in Whitaker’s words they, “batter the children with kindness”. It’s in sharp contrast to the “no excuses” approach used in a growing number of schools…

At [this school], Whitaker says, it means rewarding children for the smallest things – like being kind to fellow pupils – and not punishing bad behaviour. “I could have a kid that spits in my face today and tomorrow I’ll be OK with them,” he says. And if a pupils throws over a table and swears at the teacher? “The teacher would be really nice to them, talk nicely….

Now, I realise that in AP teachers have to accept there may be more challenging behaviour than at a mainstream school  But The Difference exists to take approaches from AP and apply them in mainstream schools in order to reduce exclusions. It matters if any AP providers are tolerating abusive behaviour, refusing to punish and then mainstream SLT are meant to learn from this and apply it. As a teacher who does not want to be told to accept abuse and disruption, I am concerned about which approaches from AP The Difference may be promoting in mainstream education.

Worse though, it has been accepted that The Difference are experts. Kiran Gill, the Chief Executive of The Difference was part of the Timpson Review of Schools Exclusions and gave evidence to the Education Select Committee. She made her beliefs clear to the committee.

Thelma Walker: Would you suggest then that, if the curriculum in mainstream were inclusive and met the needs of children in mainstream, there would be fewer children in AP? Obviously, if it was more inclusive and the staff were appropriately trained, then that early intervention in mainstream would avoid the high numbers that are going into AP.

Kiran Gill: I think that is ideal. …. if we know exclusion levels were lower, then we know we can return to lower exclusion levels.

Thelma Walker: If you have senior leaders that buy into inclusivity, the holistic whole-child curriculum and positive behaviour management, then that will prevent—

Kiran Gill: Absolutely. That is our theory of change. As David said, we need to attract people and there need to be incentives. The Difference programme seeks to recruit existing mainstream teachers who are committed to becoming specialists, to upskill them in a placement in alternative provision, and also to broker them for future leadership opportunities back in the mainstream sector. It is that leadership that is an incentive.

Now, there is no reason progressives who believe that we should stop excluding should be silenced, but we need to stop treating people with this ideology as “experts” rather than campaigners. This blogpost was prompted by the fact that there was a conference run by The Difference yesterday, and there were a couple of things claimed at the conference and then shared by their Twitter account.

 

This was later deleted

Fuller context, to the second tweet was given by the TES:

Obviously, neither of these claims are true. (If anyone disagrees I’m happy to discuss it in the comments.) I am going to suggest that schools, policy makers and politicians simply stop listening to groups like The Difference. They are willing to spread information that I think most school leaders would dispute. This serves a clear ideological commitment to reducing exclusions. They should be free to disseminate their political opinions as an ideological campaign, but we need to move on from letting this anti-teacher, anti-discipline agenda be promoted in schools.

 

h1

13th Blogaversary

November 9, 2019

I haven’t been blogging as much lately, and I do hope to get back to more regular blogging. However, I don’t want to miss marking the fact that this blog has now existed for more than 13 years.

My last “birthday” post is here: 12 years a blogger.

Since then, the education debate seems to have focused more and more on behaviour. As traditionalist ideas on pedagogy and curriculum have become mainstream, progressives have focused on a campaign against discipline, arguing that everything secondary schools do to keep order is cruel or selfish and should be banned or regulated. My posts of the last 12 months reflect that shift.

These two posts covered the debate about exclusions.

These two posts covered the campaign against internal exclusion (called “booths” by those who wish to ban or regulate it) which has gained the support of a host of politicians, bureaucrats and charities, but has staggeringly little public support.

These posts cover the ideology and values behind this anti-discipline campaign.

These two posts discussed other aspects of behaviour:

As well as behaviour, I’ve also written about the bullying behaviour that MATs, schools or individuals receive, particularly for enforcing rules or arguing against educational progressivism. One notable event of the year was that I left the NEU, having been a member of it or its predecessor, the NUT, for 17 years, because when it comes down to supporting their members or advancing their ideology, they consistently chose the latter, even if it means supporting hate campaigns against schools and teachers.

For most of the last 12 months, I worked at a school that was subjected to a shaming, and saw first hand the effect it has on teachers and how unrelated to reality the accusations can be. I will probably wait a bit longer before I blog about this, but it confirmed exactly how little outsiders, or in some cases a tiny number of hostile parents, actually know about the schools they criticise and how vile the behaviour is that teachers can be subjected to as a result of a hate campaign.

I’ve also written a few posts about ideology in education.

Then there were two posts about teacher autonomy:

And finally, in a class of its own, this is what teachers told me about work scrutinies:

In other news:

  • the level of abuse on edutwitter is probably at a all time high, so I’m aiming to adjust the balance towards spending more time on blogging and less on tweeting.
  • I have prepared a talk on the exclusions issue which I have now given three times at researchED events, and am keen to repeat elsewhere if you want to invite me.
  • I wrote a chapter about “attachment myths” in the researchED book on Education Myths, available here.
  • I got engaged.
h1

Teacher autonomy is the most difficult issue in the education debate. Part 2

October 27, 2019

When should teachers not be allowed to do their own thing?

I recently wrote a blogpost which summed up most of what I’d previously said about teacher autonomy, which emphasised how teachers should not be obstructed by their managers. When I had finished, I thought about what needed to be added and realised that I could probably continue writing any number of additional circumstances in which it was best to leave teachers alone. However, the tricky part of this issue, for me, is the extent to which teachers should have their freedom limited and be told what to do or how to teach.

The most obvious constraints are those that arise from a teacher’s duties. Teachers are contractually obliged to keep their students safe, turn up to teach them and so on. In matters of pedagogy and curriculum, practices that are so flawed that they amount to a failure to do the job of teaching effectively should be stopped. While I’ve no time for performance related pay, student outcomes that show a marked failure to learn are a justification for finding out what a teacher is doing, and if the cause of those outcomes is clear, to ask them to stop doing it. If a lack of assessment, or a lack of time, means there is no evidence of student outcomes, managers may need to intervene where the teaching methods themselves seem likely to be ineffective. However, it is probably worth noticing that managers are not necessarily good judges of this, and intervention in the absence of evidence of student outcomes should only take place where this is good reason to believe teaching is ineffective, i.e. where methods are obviously flawed, or where there is solid research evidence to show how something should be taught (e.g. use of systematic phonics for early reading) rather than where a manager simply has a different opinion to a teacher. Even then, it would help if both evaluation of student outcomes and judgements about ineffective teaching methods were open to debate.

The limited nature of this grounds for constraining teachers, and all the arguments for autonomy in my previous post, might suggest that I see little role for leaders in shaping what teachers do outside of extreme cases. However, while I do favour a lot of autonomy for teachers, this is not the end of the story. In schools, leadership is necessary to establish what is normal. By this, I don’t mean that there should be a list of “non-negotiables” that are forced on every teacher. I mean there should be default behaviours that would be expected from staff (and students) whenever there is no specific and explicit reason to do something different.

So while I don’t think managers should say, “All desks should be in rows.”. I think it’s fine for managers to say, “All desks should be in rows unless there is a specific reason for them not to be.” While I don’t think managers should say, “No groupwork”, I think it’s fine for managers to say, “You must have a reason for doing groupwork”. While I don’t think managers should say, “All classrooms should be silent”, I think it’s fine for managers to say, “Only let students talk in lessons if there is a reason for them to be talking”. It is fine for school leaders to set out what is normal in their school in this way. INSET and CPD should be based around how best to do things in that normal way. All staff will be informed of what the default way to do things is. School rules and routines will be based on enabling teachers to teach in the way that is normal for the school.

These defaults should be more of a help to teachers than an imposition; they should make it easier to make decisions about what to do in the absence of strong reasons either way and they should ensure that students have a clearer idea of what to expect from lesson to lesson. If anything, I would argue that school leaders should be more explicit about these defaults than they might otherwise be. It is absolutely fine to teach all staff some shared routines (“This is a good way to give out books.”; “This is a good way to dismiss your class.”; “This is a good way to assess learning.”; “This is a good thing to remind students of when they are using worksheets.”) and shared language to describe classroom practice (“Do nows”; “Check for understanding”, “Modelling”). This kind of detail does not have to be enforced on staff, just so long as it is well known enough to teachers that they will find these ways of doing things to be obvious options to take on board, and to use when needed. So, for instance, there is more than one way to ask a class to stop and pay attention, but if a teacher is torn between saying “SLANT” or “Pens down, eyes on me” or “Excuse me”, they should know which option is normal for the school.

This gives rise to the greatest necessary limit to teacher autonomy: teachers should not be doing things that give rise to new defaults for the school. You may allow your own class to talk when you say so, but you shouldn’t give the impression that it is generally okay for students to chat in lessons, unless that is the chosen default for the school. You may allow students to hand their books in at the end of every lesson, but you shouldn’t give them the expectation that they don’t need a school bag, unless that is the chosen default for the school. You may show a video relevant to what you are teaching; you may not given the impression that you just watch movies in the last lesson of term, unless that is the default for the school. Most of all, you might not have quite the same rules in a drama lesson as in a maths lesson, but you don’t get to pick and choose which school rules are to be enforced and which are to be ignored for an easy life.

School leaders make a huge difference, not because they can control everything that happens in every classroom, but because they set the culture in the school. I am all for teacher autonomy, but teachers should not have the freedom to create a different culture for the school, particularly not if it’s one of lower expectations.

h1

Teacher autonomy is the most difficult issue in the education debate. Part 1

September 28, 2019

How much freedom should teachers be given to do their own thing?

Few things are more irritating to educators than knowing what needs to be done, and being stopped from doing it by those in charge of you. Whether that’s the actions necessary to keep order in the classroom, or keep students safe from each other, or the way of teaching that will make the most difference. At times, one is tempted to ask for complete autonomy. Freedom to make whatever decision one wants to. However, this principle becomes less appealing when one realises that what happens in other classrooms affects expectations of behaviour and effort in one’s own. It also becomes less appealing when one considers the consequences for children of the worst classroom practices, both educationally and with regards to their safety and well-being. A related issue is workload. An assumption that every teacher plans their own lessons, may undermine collaboration in the production of resources, leaving teachers to duplicate work their colleagues have already done. Conversely, decisions imposed on teachers by those less familiar with their classes or year group, may have adverse workload consequences for teachers who find they constantly have to make adjustments for a poorly sequenced curriculum or to prepare for badly designed assessments.

It’s easy to make rhetorical arguments in favour of teacher autonomy. Teachers should be trusted. Teachers are professionals. Teachers know their classes best. It’s easy to make rhetorical arguments against teacher autonomy. Teachers must be required to teach effectively. All teachers should have high expectations. All teaching should be based on how students actually learn. Students need consistency. Once you accept that all these arguments are sometimes true, the debate becomes about where you draw the line, and that’s tricky.

In this post I will summarise my past posts on the topic, which can be found here:

In those posts I concluded a number of things.

Managers should try to avoid giving any of the following:

  • Instructions contradicted by other instructions.
  • Completely idiotic instructions.
  • Instructions that no manager would ever subsequently admit to giving.
  • Instructions which, if followed, will be used against the teacher following them.

Some of this might seem obvious, but none of these things are uncommon. Dysfunctional management is by all reasonable accounts a problem in teaching and it is worth considering where managers should definitely leave well enough alone. However, attending to this only narrowly restricts the places where the line can be drawn.

Moving on from the day to day decisions of managers to the systems used to manage teachers, I suggested the following should be avoided in any system of holding teachers to account.

  1. Trying to achieve multiple aims simultaneously and without a clear indication of priority;
  2. Holding teachers accountable for methods and outcomes simultaneously;
  3. Enforcing, and creating paperwork for things that would happen anyway;
  4. Creating work that does not have to be done;
  5. Measuring and judging things that don’t matter;
  6. Measuring and judging things unreliably;
  7. Encouraging behaviour that is actually counter-productive;
  8. Wasting money, particularly on management salaries.

Again, this stuff might seem obvious, but it is all incredibly common. I believe almost every large school would gain from applying these principles to all of its rules and systems for holding teachers to account.

The point about the problem of trying to achieve multiple aims simultaneously is one that applies at many levels and across the public services. The philosopher Onora O’Neill, when talking about accountability described the following problem:

Traditionally, the public sector exercised control by process. We often call it bureaucratic process. The private sector allegedly exercised control by targets. When the target setting was imposed on the public sector, the process controls were not removed, hence the problem of having to be responsive to and responsible for two completely different sets of controls whose coincidence is not guaranteed.

Teachers should never be held accountable for outcomes if they were not given the freedom to affect them. In this era of workload concerns, I would add that if they can affect those outcomes, but only by taking on more work, that should also be considered unreasonable.

One helpful way of looking at restrictions on autonomy can be found in this blogpost by Doug Lemov which appeals to the concept of “positive and negative variance”.

… one of the strongest ways a school can make a difference in student achievement is to have a coherent approach to teaching, one that outlines a shared understanding of “how we do it”—things that comprise a schools core approach that everyone is expected to do. The school should name the things that are part of “how we do it” and then provide training  so predictable implementation errors are reduced. That’s a way of both aligning and implementing a philosophy but also of reducing negative variance.

But it’s super-important to balance that reduction of negative variance with an understanding of the benefits of “positive variance”… the idea that people who have achieved proficiency with a skill should have the freedom to personalize and adapt.

The example he gives is centralised lesson planning. Preparing lessons centrally will reduce negative variation in that it will make it harder for teachers to be under prepared. However, in order to encourage positive variation, teachers will need to be allowed to adapt the lessons and be progressively given decision rights that can include dropping the centrally prepared lesson entirely.

I suggested the following principles might help with ensuring there is less negative variance and more positive variance.

  1. Outcomes must be considered before processes.
  2. Schools should be upfront about what they want.
  3. If you can’t write down clearly, concisely and objectively what you want, you have no right to ask for it.
  4. The best justification for restricting autonomy is where a teacher’s behaviour will undermine colleagues. e.g. differing expectations for behaviour across the school.
  5.  Don’t take the piss. i.e. don’t have systems that can be harmful to teachers in themselves by adding to stress or encouraging bullying.

That’s about 1000 words summarising what I’ve already said on this topic. I am fully aware that everything I’ve said only suggests some constraints on where to draw the line, and doesn’t give any easy answers to the question of teacher autonomy. In my next blogpost I hope to add a few more considerations that I haven’t covered previously.

h1

Has new research on exclusions solved the problem of causation?

September 21, 2019

Earlier this week, Siobhan Benita, the Liberal Democrat candidate for London mayor made a speech where she declared that:

My “feel safe, be safe” plan for Londoners will give every young person a voice, activities and the security of good schooling… No child will be permanently excluded from mainstream schools.

Naturally plenty of teachers, ex-teachers and people whose kids go to mainstream schools were able to point out on social media that it might not be best to keep young people in schools if they are dangerous or out of control. But in the online discussions that followed, a number of people who work in education, but don’t teach in schools, seemed very convinced that there was great new evidence that exclusions were bad.

It turned out to be this conference paper by Bacher-Hicks et al which, while largely having the same glaring inadequacies as any other research in this area, did have a few innovations in its methodology. It found a link between suspensions from school and various negative outcomes. It is about suspensions rather than permanent exclusions. It is based in the US not the UK. It is based on a student population where 23% of students are suspended at least once per school year, and 19% go on to be arrested between the ages of 16 and 20. However, even given the general irrelevance of this research to the debate in England about permanent exclusion, it would still be interesting if it had solved a big problem with research about school discipline.

I’ve written before about how a big problem in reaching conclusions from data is being able to identify causation from correlation, i.e. being able to identify what is an effect and what is a cause where two statistics seem to be related.

The accepted method for researching causation is to use a Randomised Control Trial, where the proposed cause is assigned at random to some part of a sample, so that the effect can be isolated by comparing the “treated” and “untreated” parts of the sample. From a practical point of view it would be very easy to test policy on exclusions in this way, you could design an exclusion process where after the decision to exclude had been made it was only carried out after being confirmed by the toss of a coin. Unfortunately, while practically easy, it is ethically beyond the pale to apply punishments by chance. So there is no body of RCT based evidence on exclusions. This is not an isolated problem, similar ethical considerations also leave us with huge gaps in our knowledge of the effects of other sanctions used in schools, not to mention criminal justice policy and parental discipline.

Where there is no chance of an RCT, researchers tend to look at existing data, and try to draw conclusions from it. This is not a futile endeavour where there are clear reasons to limit which hypotheses you are testing. If it seems reasonable to think that falling out of an aeroplane without a parachute is a cause of death, but that being about to die is not a cause of falling out of aeroplanes, then looking at the death rate of people who have fallen out of planes could provide good evidence for that hypothesis. People do use correlation evidence in order to reach conclusions about causation quite often and quite reasonably in cases where there is only one reasonable hypothesis about causation that can be made. Unfortunately, this can trick us into thinking that deducing causation from correlation is a reliable “second best” method to be used in all cases where an RCT cannot be used. It isn’t. If there are multiple competing hypotheses about how causation works in a particular instance, then correlation evidence can be, not just less reliable than RCTs, but utterly useless.

This is the situation we have throughout research on sanctions and behaviour (where RCTs are ruled out). We usually wish to know whether particular sanctions improve behaviour. We wish to test the hypothesis that applying sanction X, improves behaviour. Unfortunately, because the sanction is a result of poor behaviour, there will always be a correlation between poor behaviour and sanctions or the poorly behaved and sanctions. We can pick groups to compare or we can vary the timescale, but almost every empirical claim that sanctions don’t work, or that harsher sanctions are less effective than lenient ones, runs into the problem that the punishments were a result of bad behaviour, therefore the punished were always more likely to behave badly than the unpunished. There may also be other variables that cause both bad behaviour and sanctions, which will allow for further hypotheses about causation.

The problems with causation don’t stop ideologues from making pronouncements (“non-custodial sentences are more effective than prison sentences” or “the best way to manage children’s behaviour by explaining to them why their actions are wrong”) that the evidence cannot actually support. However, we simply cannot tell whether correlations between stricter sanctions and worse behaviour are a result of behaviour being negatively affected by harsher sanctions, or sanctions being made harsher due to worse behaviour (except perhaps by applying common sense). The same problem exists where we try to find the the effects of sanctions (or systems of sanctions) on other outcomes. Do those outcomes result from the sanctions, or from the punishment that results from the sanctions? The most egregious example of being unable to separate cause and effect we have seen recently has been over the supposed link between school exclusions and knife crime where some well-intentioned people seemed unable to consider the possibility that a propensity to violent criminal behaviour is a cause of exclusions, rather than the possibility that upstanding members of society are being excluded and stabbing people as a result.

There are various techniques that can be used to solve some problems involving correlation and causation. Sometimes the timing is a clue. Where young people are both permanently excluded and convicted of possessing a knife, it is far more common for a permanent exclusion to swiftly follow the conviction than for the conviction to swiftly follow the exclusion, suggesting that causation does not run from exclusion to involvement in knife crime. Sometimes alternative chains of causation can be eliminated by using multi-variate statistical methods (although this paper makes a pretty good argument that this doesn’t work as well as we think it does). Sometimes “natural experiments” occur, where just by luck, we have data that should resemble what we might expect from RCTs. Where people think that they have grounds for comparing the effects on two groups, without randomisation, research is often referred to as “quasi-experimental”.

And this brings us to the Bacher-Hicks et al paper mentioned earlier. A change in the boundaries of school districts allowed them to eliminate some variables that might confound efforts to identify causation involving suspensions. This enabled them to create a measure of schools’ willingness to suspend that controlled for student background, and to see what effect that had on outcomes for students who went to the school after the boundaries changed and, therefore, came from different backgrounds. Being able to control for some variables, allows the paper to improve on papers that couldn’t control for those variables. Unfortunately, the paper does not even begin to address the problem of controlling for behaviour. It describes schools with high conditional suspension rates (i.e. suspension rates after controlling for student background) as “strict” and concludes that strictness results in negative outcomes. What justification there is for this approach is limited. The point is made that where principals have changed schools, conditional suspension rates have also changed in ways that suggest leadership is important. However, this does not go very far to disprove the obvious hypothesis that high suspension rates may be a result of bad behaviour that also leads to the negative outcomes the researchers found. To conclude that suspensions, not bad behaviour, are the cause of the negative outcomes that are correlated to high conditional suspension rates, requires that one controls perfectly for bad behaviour. Anything less, can result in a correlation that does not prove causation.

One additional point I’m going to make is that research that assumes you can control for behaviour by attributing suspension rates entirely to a mix of pupil background and schools’ willingness to suspend, rather than, say, school culture, makes assumptions that I think few teachers will agree with. We will get better education research when researchers start getting better at listening to teachers when theorising about whether correlation indicates causation.

h1

Tough questions about behaviour

June 29, 2019

I originally wrote most of this as a Twitter thread, but it’s long enough that it really deserves to be turned into a blog post. 

There have been recent news reports about a school that has staff striking due to the perceived threat of violence from their students. I won’t link to it as I’m not sure naming schools always helps (not that I condemn anyone for reporting this as news). But I do want to draw attention to the underlying issue of schools where staff and students do not feel safe.

The background to this is that, in my experience, any secondary school that makes sure rules are clear and followed at all times will be labelled “zero tolerance” and demonised. Any school that doesn’t have clear rules constantly enforced will have a certain amount of kids out of control for at least some of their school day. This is, I regret to say, normal in secondary schools as far as I can tell. It doesn’t mean that every teacher faces the same situation; most at least ensure senior staff have some power and the difference between, say, being a supply teacher and being an established member of staff can be extremely vast.

However, once the normal situation in a school is for some kids to be out of control for some of the time it really doesn’t take much for things to tip towards a greater level of anarchy.  The number of staff who are ignored by kids can suddenly increase from unfamiliar faces, to any member of staff who can’t count on the full support of managers. The type of out of control behaviour can also suddenly escalate, particularly when students become aware of incidents not being dealt with adequately. It doesn’t take much to reach a point where fairly extreme behaviour, whether that’s verbal abuse to staff; violence; sexual harassment or sexual assault, or even bringing in weapons and drugs becomes normal. That’s where some schools end up.

Because the response to out of control kids when they were just disrupting lessons or ignoring some staff or some rules, was to ignore it, or cover it up or blame the teachers, some schools end up taking the same approach to more serious behaviour. At such a point, schools become dangerous to staff and students. The biggest problem in the culture of secondary schools is that there are still incentives for schools to cover up, rather than deal with, endemic behaviour problems. People still talk about “too many exclusions” or “too many detentions” as if doing something about behaviour was more of a problem than the behaviour itself.

‏Schools need to be honest. How many of your students are “outlaws”, i.e. ones who largely do what they like knowing the schools systems will never effectively constrain them? How clear are the systems for dealing with defiance? How much incentive is there for staff to ignore some behaviour? There are schools where challenging behaviour would lead to having to giving up time to sit detentions, having a confrontation or, worst of all, being blamed for the behaviour. What would actually happen if a kid was doing something wrong in front of a member of staff who didn’t know who they are? How many pseudo-rules are there, i.e. rules that can explicitly be broken without any actual punishment?

The biggest question to ask about a school is this: who would have a harder time at your school, a new member of staff who enforced every rule according to the behaviour policy or a new member of staff who didn’t even know what the behaviour policy was?

Edited 29/6/2019

Since I wrote this, more has come to light about the school where staff are on strike. According to The Sun they used the “Pivotal Approach” to behaviour. This has been condemned in the past by my union, the NASUWT, and is named after the behaviour consultancy previously (although I believe not presently) run by the consultant I wrote about here.

%d bloggers like this: