What’s Essential in the Education Debate Part 1: Truth

December 1, 2013

A few weeks ago, in a post about educationalists and their opinions, I made the following observation:

… there is a problem in education research that there is no agreement over epistemology; no identified methodology which we can expect educationalists to use. That does mean one can never assume beforehand that an educationalist has ever paid any attention to the logic of, or empirical evidence for, their position in a way we might expect from academics in some other fields.

At the time, there were those who chose to interpret this remark as being a claim that I had personal access to the one true methodology for determining truth in education and even tried to imagine what that method might be and attack it. However, I was making no such claim, and I would be the first to admit that determining the truth about matters relevant to education can be complicated. My view is not that I have the best possible methods, but I do think some methods are worthless yet, nevertheless, are accepted in education. While other disciplines may produce results or research that fall below the expected standard, education has no set of standards to be expected. Accordingly, any expertise in education is potentially expertise in using methods that are unreliable. In education, any argument claiming to be based on educational research or evidence collected by educationalists, or for that matter any argument phrased so as to sound like it involves specialist knowledge unavailable to the lay person, deserves special scrutiny.

Of course, if you raise the question of the quality of education research, you are likely to be faced with the following argument: all research is biased or flawed in some way, therefore you cannot dismiss research because it is biased or flawed. While there is some truth to this, it misses the obvious point that some biases, and some flaws, are far more serious than others. More importantly, some research will seek to avoid the flaws and biases and some will willingly incorporate them. This is what I wish to explore here, the point at which arguments which claim to be based on “research” can be safely ignored. I wish to ask what has only the outward appearance of proceeding from genuine scholarship but has none of the substance. This is not an attempt to explain how we might tell reliable and unreliable research apart (if that is what you want I would recommend this book) but an explanation of the principles that underlie the differences between arguments which claim to be informed by research. The two principles are: a respect for truth and an adherence to reason. I believe these are the absolute minimum values people should subscribe to if they hope to make a positive contribution to debate. Here I will discuss the importance of truth, in my next post I will discuss the importance of reason.

Now it may seem obvious that, when conducting research or arguing from it, we wish to establish something which is true but there are those who seem unconcerned about this. There are those who resort to obfuscation in debate even around this point. However, while it might be fun for sixth form philosophers to argue that there is no truth and there might be deeper issues around what it means for our statements to correspond to reality, one assumes that anybody claiming anything has some grasp of what it would mean for their claim to be true. What is more common, and more insidious, is indifference to truth. Claims and arguments which are made for effect without any regard to whether accepting them would lead to, or obstruct, a greater knowledge of the world. The philosopher Harry Frankfurt identified the word for this, and the word was “bullshit“.

…the fact about himself that the liar hides is that he is attempting to lead us away from a correct apprehension of reality; we are not to know that he wants us to believe something he supposes to be false. The fact about himself that the bullshitter hides, on the other hand, is that the truth-values of his statements are of no central interest to him; what we are not to understand is that his intention is neither to report the truth nor co conceal it. This does not mean that his speech is anarchically impulsive, but that the motive guiding and controlling it is unconcerned with how the things about which he speaks truly are.

It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction. A person who lies is thereby responding to the truth, and he is to that extent respectful of it. When an honest man speaks, he says only what he believes to be true; and for the liar, it is correspondingly indispensable that he considers his statements to be false. For the bullshitter, however, all these bets are off: he is neither on the side of the true nor on the side of the false. His eye is not on the facts at all, as the eyes of the honest man and of the liar are, except insofar as they may be pertinent to his interest in getting away with what he says. He does not care whether the things he says describe reality correctly. He just picks them out, or makes them up, to suit his purpose.

Any argument, or any research, that would enable one to reach conclusions independently of whether they happen to be true or not can be considered bullshit. We can all be wrong, but some methodologies lack any mechanism by which to avoid being wrong. While almost all errors of fact or reasoning can lead to the acceptance of an untrue claim,  I think there are three key questions that should be asked of all claims which are presented as reliable due to research or evidence.

Are we distinguishing between opinion and evidence? Too much education research is simply opinion dressed up as scholarship. This is a particular problem where research is classed as “qualitative” and based on personal observation. It’s not that an informed opinion is worthless, it’s that the informed opinion of a researcher who has visited a school writing up their views in an academic journal or dissertation is unlikely to be worth any more than the informed opinion of a teacher, or anyone else with prolonged access to a school. If anything, the need to write according to a pseudo-academic framework is likely to make somebody’s opinions less reliable and more prone to bias because of the need to fit observations to pre-existing theory. More generally, it is common in educational debates for somebody to claim to know what “the evidence” says, only to then quote opinions.

Was the evidence cherry-picked? I never cease to be amazed how selective people are in what they count as evidence. There is so much poor “evidence” that it is fine to ignore some of it, but you have to ignore it according to clear principles for rejection. The worst examples I have seen of cherry-picking are from among phonics denialists. There is no getting away from the fact that the evidence for the effectiveness of phonics is the strongest that exists for anything in education (perhaps in the whole of the social sciences), yet there are those who reject it and yet still complain that others ignore the evidence. Many people writing about education seem to have no criteria for what makes research reliable other than “do the conclusions agree with me?” (The website Local Schools Network seems particularly prone to this.) The recent trend towards calculating “headline” figures for effect sizes based on multiple meta-analyses has also revealed how selective researchers can be with evidence. How different studies are grouped into categories can make all the difference to the effect size.

Is this an argument from authority? It is well-recognised that a claim should not be accepted as valid or true simply because of who it is that makes the claim. This is why academic research is usually judged blind in peer review. However, in life, we don’t have time to become experts in every field and evaluate ever claim. We do rely on experts to tell us things we could not confirm for ourselves. In education this is a real problem. As a result of a lack of recognised methods,  “expertise” in education is always a highly questionable construction. There are some educationalists who are considered experts on educating people in an academic discipline who:

  • Have no degree in that academic discipline.
  • Have very fuzzy and idiosyncratic ideas about the nature of that discipline.
  • Write books containing inaccurate and poorly researched information.
  • Consistently propagandise for a particular point of view.
  • Conduct empirical research where the methodology is so unscientific as to make the results worthless.

None of this will stop somebody working in a university education department, or becoming a professor, or even an internationally prominent academic. One of the most dangerous, undemocratic suggestions in education is that policy be handed over to the experts and one of the practices most destructive to rational debate in education is to cite as reliable the opinions of those who have risen to prominence in education only by virtue of having those opinions. There are plenty of teachers out there who are more sincere, less ideological, more knowledgeable and more concerned about truth than many academics (not that power should be handed over to teachers either).

We cannot be given reason to accept a claim by any argument which is dependent on classing opinion as evidence, cherry-picking evidence or arguing from the authority of educationalists. Next time I will consider the importance of reason and how we might argue rationally against a claim.


  1. Reblogged this on The Echo Chamber.

  2. Reblogged this on Primary Blogging.

  3. Agree with all of this, and just one further thought. Where do you think disciplines come into this? I would argue (I half wrote a post on this but never quite got round to finishing it) that there is actually no such thing as ‘educational’ research in that education is an object of study, not a discipline. So we have psychology of education, sociology of education, history of education etc. which all address different kinds of questions about reality and, in some senses, have different conceptions about what constitutes evidence, proof etc. What a historian takes as evidence would not be considered as such by a cognitive psychologist, for example. Most of the poor quality research in education I have seen has been attempts to answer psychological and sociological questions about education while not adhering to disciplinary rules. This is not helped by the fact that these disciplines, especially sociology, are internally divided already. Of course there can be inter-disciplinary research, but this is even harder as one needs to have a grounding in more than one discipline! My view would be that, once one is clear about what discipline one wants to contribute to, then (and only then), one can determine a rigorous approach to research.

    If you have access to journals (not sure if you do) then I would thoroughly recommend David Bridges (philosopher of education) on this – his article on truth in educational research is a response to the same thing that you are criticising here.

    • I’ve deliberately stopped short of addressing the questions of which disciplines and which precise methods apply, precisely because I’m trying to explore the minimum standards. History and cognitive psychology are very different disciplines, but those in both fields can reasonably commit to seeking truth and avoiding falsehood. The question really is: which disciplines cannot help as they are indifferent to truth? Which disciplines are just bullshit?

      • I would suggest that those who write bs are not disciplined per se.

  4. Good to read your views on this. You might like to read these blogs by @GeoffJames42 on the same subject http://www.solution-support.co.uk/2013/11/30/ontology/ and http://www.solution-support.co.uk/2013/11/30/part-2-ontology-getting-relevant/

    • Is that on the same subject? The writer doesn’t seem to make it clear what he is addressing, just implies that he is ontologically more sophisticated than some unidentified positivists.

  5. An interesting post – thank you. It seems to me that the study of education has yet to decide what exactly it is. Having studied historiography (a long time ago!), I think that each academic discipline needs to have a firm philosophical underpinning, which its practitioners understand – and I’m not sure that this is the case in education.
    I think the problem is partly that education as a ‘thing’in itself is dominated by politics, which has a very different motivation to the search for ‘truth’. We very often come down to which slick salesman has the ear of the current minister (in my view).
    As far as ‘truth’ is concerned – is it necessary to find it in an educational context? I think of it as somewhat like those crystal things my grandma had in the 80s. They were multifaceted objects that sparkled as you twisted and turned them in your hand (not supposed to), each twist and turn giving a different perspective.
    Education, and teaching, is itself a multifacted animal, dealing as it does with human beings, and small underdeveloped human beings at that. The permutations of these small beings are endless – family background, religious belief, eyesight, hearing, personality; all these things they bring to the classroom, and we haven’t started looking at the person of the teacher yet, who certainly brings something indefinable to the mix.
    I’m not convinced, in this case, of a search for educational ‘truth’ – what matters to me is a constant reflection on classroom practice, a willingness to test and find out for yourself what works in YOUR classroom – a constant journeying towards the ever fixed mark, rather than the mark itself.
    I would bring a certain philosophical element into the training fro teachers – I don’t think enough people stop to think about what it is they think and why it is they think that.
    This is not to say that I don’t believe in the concept of ‘truth’ – just that I’m not sure, in an educational context, what it is supposed to be – or indeed how useful it would be.
    After all, what do you do when you come across the child who breaks the rules?
    I might write a blog post about it one day – if I’m not too exhausted by minutiae of family life.
    Thanks again.

    • Doesn’t really help to put “truth” in inverted commas like it was some difficult concept. We can say nothing at all without some notion of what it means for what we say to be true. Perhaps there are philosophical issues around truth, but we cannot deny that statements can be true without denying that there we can speak of reality. To be indifferent to truth is to be indifferent to reality, and I see little point to research of this sort.

      • So I see.
        My concern might be, in an educational context, that when one has come to a decsion about what is true (eg children read best when they experience a well-taught phnoics programme), what do we do when we come across a child who challenges that truth?
        Does the truth become less true, or do we see that truth in this sense is a faceted being?
        I think that truth is a difficult concept – it is, in human terms, endlessly displaced. We chase after it, but will we ever find it?
        As teachers, or educational researchers, we can’t ever escape our subjectivity. We can only look through our own eyes, and we can only really speak from our own experiences.
        Personally, I think we can only really tell our stories, and let others decide what from them they will take. All we can ever do is present our argument and back it up with the evidence we subjectively choose – however conscious that might be.
        However, I should point out that I am writing as a primary teacher of children with SEN (with a history degree), which I suspect colours my comments.
        I would be interested to know what you teach – or which academic discipline is your own – and how this affects your notion of truth.

        Note, I have not used inverted comments this time :)

        • You now appear to be ignoring the argument in the blogpost and just asserting without argument the very things I had challenged at length.

          You can put it in any language you like, but what you are arguing for is indifference to the truth. Of course we can tell stories. But we have a word for the stories people tell when they don’t care if they are true or not.

          • Not really.
            I feel that you may have misinterpreted my comment. I think your call for research based on truth is a laudable one – however, I don’t think that truth in that sense is possible.
            I think the best we can hope for is HONESTY (that’s an attempt at bold, rather than shoutiness) – in terms of intention, and transparency in terms of evidence.
            If we try to present an argument in this context, as educational researchers, as true then I think we are setting ourselves up for this very criticism.
            Sadly, we live in an age of moral relativism. I’m not at all indifferent to truth – but I do feel it is something that has to be seen as a matter of faith rather than evidence.
            I hope that’s clearer. I’m not looking for an argument. I think your post is interesting, and it is good to think about. All I attempted to do in my original comment, was add to it, not engage in an argument.

  6. It doesn’t matter what theory is Flavour Of The Month this month because there isn’t ever only one right way. There isn’t one standard clone teacher and one standard clone pupil who will “get” things in this way only. This effort to cram pegs of so many different shapes into arbitrary square holes causes so much unnecessary grief.
    When I was a child, the variety of approach of the teachers was one of the things that made school interesting. Now every teacher conducts a crocodile of children up the same level-ladder in every subject.

    • There might not be “one right way”, but there are a lot of very, very “wrong” ways and many of them have been forced onto teachers having been lapped up by SLT who should have known better.

      I mean. “Brain Gym!”. “VAK”. “Activities” for their own sake that _clearly_ divert attention (and cognitive effort) from what is supposed to be being taught, etc.

  7. […] Old gets himself embroiled in debates on the veracity or otherwise of theoretical claims, as his latest post shows. This has of course been going on for decades, well before the birth of blogs, but I think it has […]

  8. I am about to embark on a research project which begins in January at the start of the new term which is aimed at closing the attainment gap at the earliest possible stage before it becomes resistant to closure because it is known that it is this group which is responsible for the long tail of educational failure which determines our embarrassingly low position on international league tables. Significant studies have shown that the percentage of pupils failing to achieve a satisfactory standard of literacy at age 11 could theoretically be reduced from its present 20% to about 1.5%

    This authority has completed a pilot research project in a credible sample of their own primary schools to test the hypothesis that a non-ritual phonic teaching strategy could change that theory into practice and the outcomes of the project have convinced them that it is likely to do so! Their efforts in the coming months will therefore be research-led rather than rhetoric of theory-led which gives their project some considerable credence.

    From the beginning of January, the approach will be used by between 350 and 400 pupils with litskills deficits in each annual cohort for about 20 mins every day which means that about 2000 children will be using it daily across about 200+ schools.

    I suggest that the sheer size of this sample of pupils together with the existence of good historical data for comparisons means that the outcomes, whatever they may be, will easily stand up to comparison with academic studies in any other field. There will be no subjectivity in these outcomes – opinions will not be confused with ‘evidence’ – standards in reading and other related skills will improve significantly or they will not. The authority uses CEM AfE(InCAS) standardised tests!

    • In such research, it’s the sensible choice of control that matters. A positive effect size is almost guaranteed for any intervention and nobody wants another Reading Recovery scandal.

      • I couldn’t agree less. The standard research model has not served Education well – ‘controls’ are fine when testing identically produced medicines and placebos – the fact that for many decades, literacy standards are either not improving or getting worse is proof of that.

        In 1939 when every man in the UK was conscripted into the armed forces, it was discovered that 1 in 5 could not read the instructions on the deadly munitions they were expected to use No-one questioned the absence of ‘controls’ to underpin this revelation. The situation has not significantly improved in spite of probably thousands of academic research papers and a similar number of research projects with ‘controls’ and the expenditure of billions of tax-payers pounds.

        The main folly of Reading Recovery was its ridiculous costs. This project required no additional funding whatsoever.

        The ‘controls’ for my project are the significant proportion of children who enter Year 2 every year, unable to read and write confidently – then graduate to secondary school and ultimately leave school still unable to read and write confidently – they are the only ‘controls’ that matter. If that tragedy is brought to an end – no-one will give a damn about the ‘political correctness’ of the project which brought it about – instead, all the pundits and dogma zealots will be claiming that this was what they had been saying all along!

        The revolutionary feature of this project is the absence of any explicit phonics teaching in its methodology – something that directly challenges current national policy. This is also something that should be welcomed by those who implicitly believe that only rigorous phonics teaching will resolve literacy difficulties because if this project fails – it would support their views. I think however that only the very small proportion of them whose understanding is more than superficial will agree with that and welcome such a large and properly researched project. The others will be more inclined to cherry-pick the literature in search of anything, however remote, to rubbish it..

        The point of the project is to end that tragedy and raise us up from our consistent and embarrassingly low position on international literacy league tables. If that happens, the nit-pickers and nay-sayers can criticize the detail of the research till the cows come home and it will make not one iota of difference to anyone except them. The 100,000+ children every year who would otherwise have left school less than functionally literate will no longer do so – what difference will the absence of ‘controls’ make to them personally – to the economy or to the massive justice and social budgets which they consume?

        Another advantage of this project is that it is research-led and that the authority which is conducting the project carried out its own pilot project in its own primary schools so that all the variables which cannot in any event be accurately replicated for ‘control’ purposes have been taken into account because of the representative validity of the pilot project sample size. They did not rely on other people’s data or conclusions.

        In the field of UK literacy, Rome is still burning and Nero is still playing away at his fiddle.. .but thanks to the conviction and efforts of this authority, there is at least the possibility that that could be coming to an end thanks to the efforts and boldness of some educators who are more concerned with the outcome than the political correctness of how it was achieved.

  9. […] Wat is er nu echt belangrijk in onderwijsonderzoek? En aansluitend: vaak ligt de echte macht over onderwijs bij de onzichtbare persoon of personen achter de minister, check dit Brits verhaal. […]

  10. […] my previous post I considered the importance of valuing truth in making arguments in the education debate. Here, I […]

  11. […] I looked at the importance of a commitment to truth and a rejection of unreasonable methods of disagreement in the education debate. In this blogpost […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: