h1

Does this data show a primary school cheating in KS2 exams?

May 13, 2018

If you have worked in secondary schools, particularly in maths and English, you have probably heard gossip about that feeder school, the one whose kids come through with KS2 results that seem too high and this kid who went there told a TA who told a teacher, who told another teacher, who told you that there was some kind of cheating going on.

I’m inclined to be sceptical about these stories. I’ve heard the same story exactly about too many very different schools. More importantly, while there are definitely kids whose KS2 results massively overstate their abilities, there are also kids whose results are massively understated too, and we tend not to remember those kids quite so clearly as the kid we put in top set maths who does all their working out on their fingers. Inaccuracy alone is not evidence of cheating.

That said, high stakes tests do create incentives to bend or break the rules, and certainly an incentive to ignore mistakes in the wrong direction. And I have seen a couple of posts recently about this issue written by bloggers who work in primary schools:

A few weeks ago, Education Datalab wrote a post about Difficult questions about some schools’ Key Stage 2 results which you should probably read before continuing.

But just in case you don’t, it looks at results like this for students in one school with several different feeder primary schools:

Students who arrived from school H are making far less progress that their Key Stage 2 results would suggest they should. The post notes that children in school H achieved far higher results at Key Stage 2 than expected, compared with the other schools, and asks:

This is a real example. The secondary school is in an Ofsted category, criticised in its latest inspection for the progress of its most able pupils.

School H is rated outstanding.

We need to ask difficult questions about the Key Stage 2 results of School H.

Was it really the case that pupils performed at such a level at School H that they simply could not maintain their progress at secondary school?

Could they have received additional assistance when taking their KS2 tests?

Does test security need to be improved?

I think the blogposts I linked to earlier make a good, if anecdotal case, for improving test security, regardless of this set of data. However, the question that fascinates me here is whether the data does suggest children at school H have received additional assistance, or show any other kind of rule bending or cheating has taken place.

And I’m going to answer “no”, at least not without further data.

I think that if a school is very effective, doing all the right things and not cheating, children will achieve better than expected. However, I’m not sure we should expect them to continue to progress well. Academic achievement is a mix of what a child brings to a school (and I’m not going near the issue of the extent to which that comes from nature or nurture) and the teaching and support in the school; both “in school” and “out of school” factors. A highly effective school will get good results from kids whose “out of school” factors would normally lower their achievement. But the negative “out of school” factors will still exist, and yet their results will see them compared with students who will achieved those same results with only positive “out of school” factors. You would expect them to do less well than other students with the same Key Stage 2 results because you would expect those “out of school” factors to affect their achievement in their secondary school. There’s also room here for some kind of regression to the mean as well, although I’m not going to suggest how that would work.

It is well known in education research that the effect of educational interventions “wash out” (i.e. the relative educational gains will be far less the longer you wait before measuring them). I think the effects of great primary schools will also “wash out” over time. I think there is a general problem here with controlling by results from a previous school that needs to be addressed. We should expect some degree of “washing out” for the effects of highly effective education whenever we control for prior results, and we need a way to predict it.

And by the way, this is not the first time I’ve seen this problem. I’ve also seen people use A-level results as a control when considering how students from different types of school do at university and conclude that comprehensive students do better at university than students from grammars or independent schools. Again, a “wash out” effect for the advantages of selective or private schooling, could explain such data. Controlling for previous results is not straightforward if the same results can have different causes for different children.

And so to conclude, interpreting education data is complicated and we probably know less than we thought we did.

Damn it.

Advertisements

5 comments

  1. I certainly think that there are isolated cases of poorly administered tests providing questionable validity of some schools results , however more pervasive is a culture of cramming as Professor Willingham says

    “too many students were taking the wrong approach to committing information to memory, because “what they mostly do is read stuff over and over again”. This meant they were “not really thinking about the meaning at all – and, of course, it’s the meaning you’re hoping to remember [because] that’s what you’re going to be tested on”.”

    Both are the result of the high stakes, high accountability nature of ks2 exams

    https://www.tes.com/news/daniel-willingham-how-use-working-memory-your-pupils-advantage


  2. I think I agree with you on this. What fascinates me is the larger extent of this ‘washing-out’ process. How much of our education washes-out over the course of our lives…? How much do we gradually revert to the predictions of our genetic and social circumstances? How much of what we do in schools has genuine lasting value, and how much is a vanity project, or just a temporary means to an end…?


  3. Very interesting article! I do personally feel that too much energy is wasted in education on data collecting and providing evidence (particularly in the U.K.) you are so right about everyone knowing ‘that’ school with high grades though 😂


  4. It might be indicative of vast amounts of time spent on test prep ; I don’t know if Year 6 spend ages on practice papers nowadays ; they used to. It’s not absolutely indicative of cheating.


  5. Reblogged this on The Echo Chamber.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: