I’m not a huge fan of the Education Endowment Fund, partly because they’ve allowed some pretty shoddy research in the past, and partly because they have a history of being partisan on certain issues. However, they do fund RCTs that test certain education initiatives and, at the very least, that should enable them to spot some popular initiatives that have no effect or even a negative effect.
The latest emperor with no clothes is Achievement For All, which according to the EEF website
…is a whole-school improvement programme that aims to improve the academic and social outcomes of primary school pupils. Trained Achievement for All coaches deliver a bespoke two-year programme to schools through monthly coaching sessions which focus on leadership, learning, parental engagement and wider outcomes, in addition to focusing on improving outcomes for a target group of children (which largely consists of the lowest 20% of attainers). The programme has cumulatively reached over 4,000 English schools.
Their evaluation of the programme found that:
In this trial, Achievement for All resulted in negative impacts on academic outcomes for pupils, who received the programme during five terms of Years 5 and 6 (ages 9-11). Children in the treatment schools made 2 months’ less progress in Key Stage 2 reading and maths, compared to children in control schools, in which usual practice continued. The same negative impact was found for children eligible for free school meals. Target children (those children the intervention specifically aimed to support) also made 2 months’ less progress in reading, and 3 months’ less progress in maths. The co-primary outcome finding (whole-group reading, and target children reading) had a very high security rating, 5 out of 5 on the EEF padlock scale.
Given the size of the effects and the consistency of negative findings, these results are noteworthy. Of particular importance is the impact that the programme had on target children, and children eligible for free school meals.
A report in Schools Week filled in some details.
The findings rank AfA as the joint worst-performing of more than 100 projects reviewed by EEF since 2011, with only three other projects earning an impact rating of negative two months.
Of these it is the only one to have the highest possible evidence strength of five – which indicates EEF “have very high confidence in its findings”.
They also reported the laughable response of the founder of AfA, Professor Sonia Blandford:
Blandford pointed out that disadvantaged pupils within the AfA trial schools still “achieved above national expectations, which was our key aim in the intervention”.
She added “it was an error to agree to a trial that attempted to evaluate the effectiveness of our broad and yet bespoke approach through the narrow lens of two school improvement parameters”.
Does this matter? I think it does. Since it started in 2011, it’s entirely possible that 4000 schools have harmed their students’ learning or at least wasted resources on something that is more likely to be harmful than helpful. And it’s worth asking how. Probably the single biggest reason this disaster lasted for so long is because the DfE endorsed it with a report assessing its positive effects on SEN children with data collected through:
- teacher surveys
- academic sources
- interviews with strategic people
- longitudinal case studies of 20 AfA schools
- mini case studies of 100 pupils and their families
- AfA events
In other words, the kind of “research” that costs money but nobody can reasonably believe is a fair way to evaluate an initiative of this kind. So the first thing we can learn from this is that the DfE should not be endorsing projects in this way. Particularly when the chances are some teachers, like Mrs S below, could have given a more accurate evaluation.
But another point, is the extent to which the people who run organisations like this become a vested interest, eagerly telling politicians and the public that schools are getting it wrong. There is a huge amount of expertise in the system in teachers and school leaders. Yet, it is staggering how often the AfA’s Professor Blandford was a voice in important debates. I have particularly noticed how such people, whose position means they don’t have to deal with the consequences of dangerous and out of control schools, seem to dominate the debate on exclusions. Professor Blandford was a particularly loud voice on this issue:
Calling for fewer exclusions in response to the Timpson Review.
In the TES claiming schools could do without exclusions
Talking on exclusions at Kinsgston University
Addressing LA conferences on reducing exclusions
Any one of these would have been far better used as an opportunity for a successful school leader to explain why exclusions are necessary. But our education system as a whole promotes the voices of “experts” whose ideas don’t work over the voices of practitioners with a proven track record.
And I won’t ever forget, as I reported here, back in the days when the Chartered College Of Teaching was still pretending it was going to be teacher led, Professor Blandford was one of the first non-teachers to be given a leadership role that, if promises had been kept, would have gone to a teacher.
I’ve always defended the right of non-teachers to help and advise schools, but we need a system where schools look first to a) practitioner expertise and b) what has been proven to work. Not a system where it’s only after 4000 schools and 9 years that we actually realise that we’ve been listening to the wrong people.
Scenes From The Battleground 
Achievement For All is bad for kids
June 9, 2020I’m not a huge fan of the Education Endowment Fund, partly because they’ve allowed some pretty shoddy research in the past, and partly because they have a history of being partisan on certain issues. However, they do fund RCTs that test certain education initiatives and, at the very least, that should enable them to spot some popular initiatives that have no effect or even a negative effect.
The latest emperor with no clothes is Achievement For All, which according to the EEF website
Their evaluation of the programme found that:
A report in Schools Week filled in some details.
They also reported the laughable response of the founder of AfA, Professor Sonia Blandford:
Does this matter? I think it does. Since it started in 2011, it’s entirely possible that 4000 schools have harmed their students’ learning or at least wasted resources on something that is more likely to be harmful than helpful. And it’s worth asking how. Probably the single biggest reason this disaster lasted for so long is because the DfE endorsed it with a report assessing its positive effects on SEN children with data collected through:
In other words, the kind of “research” that costs money but nobody can reasonably believe is a fair way to evaluate an initiative of this kind. So the first thing we can learn from this is that the DfE should not be endorsing projects in this way. Particularly when the chances are some teachers, like Mrs S below, could have given a more accurate evaluation.
But another point, is the extent to which the people who run organisations like this become a vested interest, eagerly telling politicians and the public that schools are getting it wrong. There is a huge amount of expertise in the system in teachers and school leaders. Yet, it is staggering how often the AfA’s Professor Blandford was a voice in important debates. I have particularly noticed how such people, whose position means they don’t have to deal with the consequences of dangerous and out of control schools, seem to dominate the debate on exclusions. Professor Blandford was a particularly loud voice on this issue:
Calling for fewer exclusions in response to the Timpson Review.
In the TES claiming schools could do without exclusions
Talking on exclusions at Kinsgston University
Addressing LA conferences on reducing exclusions
Any one of these would have been far better used as an opportunity for a successful school leader to explain why exclusions are necessary. But our education system as a whole promotes the voices of “experts” whose ideas don’t work over the voices of practitioners with a proven track record.
And I won’t ever forget, as I reported here, back in the days when the Chartered College Of Teaching was still pretending it was going to be teacher led, Professor Blandford was one of the first non-teachers to be given a leadership role that, if promises had been kept, would have gone to a teacher.
I’ve always defended the right of non-teachers to help and advise schools, but we need a system where schools look first to a) practitioner expertise and b) what has been proven to work. Not a system where it’s only after 4000 schools and 9 years that we actually realise that we’ve been listening to the wrong people.
Share this:
Posted in Commentary | 3 Comments »