What’s Essential in the Education Debate Part 1: TruthDecember 1, 2013
A few weeks ago, in a post about educationalists and their opinions, I made the following observation:
… there is a problem in education research that there is no agreement over epistemology; no identified methodology which we can expect educationalists to use. That does mean one can never assume beforehand that an educationalist has ever paid any attention to the logic of, or empirical evidence for, their position in a way we might expect from academics in some other fields.
At the time, there were those who chose to interpret this remark as being a claim that I had personal access to the one true methodology for determining truth in education and even tried to imagine what that method might be and attack it. However, I was making no such claim, and I would be the first to admit that determining the truth about matters relevant to education can be complicated. My view is not that I have the best possible methods, but I do think some methods are worthless yet, nevertheless, are accepted in education. While other disciplines may produce results or research that fall below the expected standard, education has no set of standards to be expected. Accordingly, any expertise in education is potentially expertise in using methods that are unreliable. In education, any argument claiming to be based on educational research or evidence collected by educationalists, or for that matter any argument phrased so as to sound like it involves specialist knowledge unavailable to the lay person, deserves special scrutiny.
Of course, if you raise the question of the quality of education research, you are likely to be faced with the following argument: all research is biased or flawed in some way, therefore you cannot dismiss research because it is biased or flawed. While there is some truth to this, it misses the obvious point that some biases, and some flaws, are far more serious than others. More importantly, some research will seek to avoid the flaws and biases and some will willingly incorporate them. This is what I wish to explore here, the point at which arguments which claim to be based on “research” can be safely ignored. I wish to ask what has only the outward appearance of proceeding from genuine scholarship but has none of the substance. This is not an attempt to explain how we might tell reliable and unreliable research apart (if that is what you want I would recommend this book) but an explanation of the principles that underlie the differences between arguments which claim to be informed by research. The two principles are: a respect for truth and an adherence to reason. I believe these are the absolute minimum values people should subscribe to if they hope to make a positive contribution to debate. Here I will discuss the importance of truth, in my next post I will discuss the importance of reason.
Now it may seem obvious that, when conducting research or arguing from it, we wish to establish something which is true but there are those who seem unconcerned about this. There are those who resort to obfuscation in debate even around this point. However, while it might be fun for sixth form philosophers to argue that there is no truth and there might be deeper issues around what it means for our statements to correspond to reality, one assumes that anybody claiming anything has some grasp of what it would mean for their claim to be true. What is more common, and more insidious, is indifference to truth. Claims and arguments which are made for effect without any regard to whether accepting them would lead to, or obstruct, a greater knowledge of the world. The philosopher Harry Frankfurt identified the word for this, and the word was “bullshit“.
…the fact about himself that the liar hides is that he is attempting to lead us away from a correct apprehension of reality; we are not to know that he wants us to believe something he supposes to be false. The fact about himself that the bullshitter hides, on the other hand, is that the truth-values of his statements are of no central interest to him; what we are not to understand is that his intention is neither to report the truth nor co conceal it. This does not mean that his speech is anarchically impulsive, but that the motive guiding and controlling it is unconcerned with how the things about which he speaks truly are.
It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction. A person who lies is thereby responding to the truth, and he is to that extent respectful of it. When an honest man speaks, he says only what he believes to be true; and for the liar, it is correspondingly indispensable that he considers his statements to be false. For the bullshitter, however, all these bets are off: he is neither on the side of the true nor on the side of the false. His eye is not on the facts at all, as the eyes of the honest man and of the liar are, except insofar as they may be pertinent to his interest in getting away with what he says. He does not care whether the things he says describe reality correctly. He just picks them out, or makes them up, to suit his purpose.
Any argument, or any research, that would enable one to reach conclusions independently of whether they happen to be true or not can be considered bullshit. We can all be wrong, but some methodologies lack any mechanism by which to avoid being wrong. While almost all errors of fact or reasoning can lead to the acceptance of an untrue claim, I think there are three key questions that should be asked of all claims which are presented as reliable due to research or evidence.
Are we distinguishing between opinion and evidence? Too much education research is simply opinion dressed up as scholarship. This is a particular problem where research is classed as “qualitative” and based on personal observation. It’s not that an informed opinion is worthless, it’s that the informed opinion of a researcher who has visited a school writing up their views in an academic journal or dissertation is unlikely to be worth any more than the informed opinion of a teacher, or anyone else with prolonged access to a school. If anything, the need to write according to a pseudo-academic framework is likely to make somebody’s opinions less reliable and more prone to bias because of the need to fit observations to pre-existing theory. More generally, it is common in educational debates for somebody to claim to know what “the evidence” says, only to then quote opinions.
Was the evidence cherry-picked? I never cease to be amazed how selective people are in what they count as evidence. There is so much poor “evidence” that it is fine to ignore some of it, but you have to ignore it according to clear principles for rejection. The worst examples I have seen of cherry-picking are from among phonics denialists. There is no getting away from the fact that the evidence for the effectiveness of phonics is the strongest that exists for anything in education (perhaps in the whole of the social sciences), yet there are those who reject it and yet still complain that others ignore the evidence. Many people writing about education seem to have no criteria for what makes research reliable other than “do the conclusions agree with me?” (The website Local Schools Network seems particularly prone to this.) The recent trend towards calculating “headline” figures for effect sizes based on multiple meta-analyses has also revealed how selective researchers can be with evidence. How different studies are grouped into categories can make all the difference to the effect size.
Is this an argument from authority? It is well-recognised that a claim should not be accepted as valid or true simply because of who it is that makes the claim. This is why academic research is usually judged blind in peer review. However, in life, we don’t have time to become experts in every field and evaluate ever claim. We do rely on experts to tell us things we could not confirm for ourselves. In education this is a real problem. As a result of a lack of recognised methods, “expertise” in education is always a highly questionable construction. There are some educationalists who are considered experts on educating people in an academic discipline who:
- Have no degree in that academic discipline.
- Have very fuzzy and idiosyncratic ideas about the nature of that discipline.
- Write books containing inaccurate and poorly researched information.
- Consistently propagandise for a particular point of view.
- Conduct empirical research where the methodology is so unscientific as to make the results worthless.
None of this will stop somebody working in a university education department, or becoming a professor, or even an internationally prominent academic. One of the most dangerous, undemocratic suggestions in education is that policy be handed over to the experts and one of the practices most destructive to rational debate in education is to cite as reliable the opinions of those who have risen to prominence in education only by virtue of having those opinions. There are plenty of teachers out there who are more sincere, less ideological, more knowledgeable and more concerned about truth than many academics (not that power should be handed over to teachers either).
We cannot be given reason to accept a claim by any argument which is dependent on classing opinion as evidence, cherry-picking evidence or arguing from the authority of educationalists. Next time I will consider the importance of reason and how we might argue rationally against a claim.