h1

Performance Related Pay?

May 30, 2009

I think the following, the latest video from the great Dan Willingham, gives a pretty good summary of what’s wrong with trying to pay teachers by results:

About these ads

32 comments

  1. I agree with almost all the points the Daniel Willingham has made in this video, but not his conclusion.

    The education community will face ongoing unrelenting pressure from the broader community to prove that it is being “successful” – achieving its goals.

    Factors which negatively impact this success will be seen as excuse-making until they are resolved – and the wider community will not accept responsibility for these factors themselves.

    Worse, it is easy to “make political hay” out of this issue with the reactionary public. Saying that the proposed system is unworkable without proposing a solution is just setting the profession up for tarring & feathering as incompetents who can’t do the required job.

    Is there a solution? One must be found, or an external solution will be imposed by those who may/will have ulterior motives.

    Perhaps in this light, performance based pay is the least unpleasant option (the known evil)- at least teachers can influence the outcomes. If there are other options, I would love to evaluate them and discuss which is the best, but status quo is not an option, nor can we return to the antebellum conditions.

    What alternative to performance based pay would you propose, OldAndrew? Or do you accept that the profession will be forced to accept some form of performance based pay?


  2. I agree 100% and have said so on my blog in almost exactly the words you’ve used. I’ve encouraged teachers (through their unions) to seize control of this issue because if they don’t, someone else will impose a “solution” on them.
    I don’t have a solution in mind and I think that the solutions should come from the teachers, not from anyone else. *Teachers* know better than anyone else what makes a teacher effective; and sure, they should consult with measurement experts about how to do the measurement stuff, but the solution should come from teachers.


  3. Well argued, and clearly correct – but trying to persuade tabloid press journalists and their readers that would be about as difficult as proposing that our MPs need more allowances today!


  4. As Aus Andrew says, what alternative to Value Added (VA) for performance related pay?

    VA measures are certainly used as a tool to control teachers, but they are not the only one (although, you’d be hard pushed to see an alternative used in many schools.)

    The best I’ve seen is confident and principled managers taking an interest in the day to day work of their teachers, and not just when there seems to be a problem. I have spend any number of ‘annual appraisal’ meetings suggesting to the head teacher and dept. head that perhaps they would like to wander about his school, in and out of classrooms, getting to know the teachers and pupils, and getting a proper overview of what happens and how staff work.

    But, guess what? Not one has ever just wandered into my classroom. They are too busy number crunching in their offices to see anything with their own eyes.


  5. As Aus Andrew says, what alternative to Value Added (VA) for performance related pay?

    here’s an idea – pay successful schools a ‘whole school’ bonus for success (rather then individual teachers) – with the money to be shared equally between all teachers. That would allow a mutually supportive ethos to develop – a bit as it does (I find) in John Lewis stores where all staff are’members of the partnership’ and so tend to go a bit further with their service.

    I’m sure there are problems with this, but as a starting point?


  6. My alternative is to treat teaching as a profession.

    You don’t let people in unless they are willing and able to do the job, but once they are in you rely on their professionalism rather than on differential rewards for ensuring they do the job well and you only intervene if there is evidence that they aren’t doing their job.

    The fact is that we will never have an objective measure for distinguishing really good teachers from okay ones and performance-related pay will fall down as a result and become either “patronage-related pay” or just another piece of paperwork.


    • What is a profession, Andrew? I believe that, historically, the term arose to describe jobs with specialised education and examinations:

      “the development of formal qualifications based upon education and examinations”.

      The question, then, it whether teacher training is any good and whether we can examine teaching properly. If we can’t introduce performance related pay it is because we cannot measure teaching ability accurately. If we can’t measure teaching ability, how can we predict or assess or validate potential teaching ability in an examination?
      And can we teach teaching properly? We surely couldn’t test for or examine that objectively either. In fact, there are some studies that show that American teachers who underwent special short courses (2 weeks in some cases!) were just as ‘effective’ as more traditionally 1-year-course trained teachers.

      I quite seriously think that the inability to examine teaching ability is exactly why teaching is difficult to make into a profession.


      • What is a profession, Andrew?

        Sigh. For someone who seems to search for trifling points to pick holes in you sure are good at missing major issues that have already been discussed (even though you had commented on it under your old nom de plume).

        http://teachingbattleground.wordpress.com/2007/09/22/professionalism/


  7. You’re right there will never be an objective measure for teaching outcomes, but as with all limited stats they can be useful if interpreted carefully by someone who understands their limitation and does not rely heavily on them.

    I am intrigued by your emphasis on the word ‘patronage’, since that word seems to simply imply the ‘subjectivity’ that must be used if we abandon the idea of ‘objectivity’.

    I agree that there are no reliable single objective measures and that purely subjective patronage opens up a can of worms, but we must have some oversight. Relying on the nebulous idea of professionalism to protect pupils’ education standards has some value, but like VA and patronage, there must be checks and balances.

    How about a careful mixture of all three (along with compulsory statistics training for managers, given by proper statisticians perhaps, but not government functionaries), to encourage professionalism in managers?


  8. I think a mixture of all three just risks absorbing all three types of errors.

    But say we did (through extra training, new techniques and better tests) manage to measure teacher performance accurately. Would the benefits of rewarding that performance in any way cover the costs of identifying that performance? Would it actually improve teaching?

    I don’t know about you, but I have never become more committed to a class for the sake of financial rewards. If I was really seeking to maximise my earnings then the first thing I’d do is walk out of the classroom door and move to a better paid profession.


  9. In Victoria, Australia – where I teach, we are going through the phase of formalising teaching as a profession. We have a professional represenative body (the Victorian Institute of Teaching – VIT). We have requirements of a certain level of ongoing professional learning, and entry assessment to the profession to be granted professional status.

    This has yet to show benefits in the delivery of a successful educational program. I am very much in favour of continuing this process, as I am very hopeful that in the future it will yield some positive returns for teachers and the community as a whole, yet I think there is one missing aspect to the program.

    If we compare teaching to other recognised professions – such as engineers, lawyers, doctors, accountants etc, we find one core difference – pay scales. Education pays poorly on entry, and scales poorly as well. In the past, the trade off was constancy – the job would always be there. This is no longer the case – many teachers work on contract now, with all the uncertainty contracts bring (renewal, conditions). I am only anecdotally aware of the situation in other countries, but in Australia, the pay scale tops out for the highest qualified principals of the largest schools at sub 200K $AUS. This is no doubt a large amount of money, but in comparison to what? Can you imagine if this were true for Doctors or Accountants?

    This is not an argument asking for higher pay – this is a statement of fact. The lower entry conditions and remuneration mean that the more ambitious, more dedicated types will seek other alternative employment options with a better recognition of the value of its employees. This is not to cast aspersions on my colleagues, but a recognition of reality. While many who enter the career of teaching are dedicated individuals, the situation with respect to university training the low entry requirements for a teaching qualification tend to result in a significant fraction of individuals who perceive a career in education as a stop-gap until something better comes along.

    To restate my position clearly: professions require a certain amount of remuneration that is not on offer in education to retain the “best”.

    This situation would be (somewhat) ameliorated by performance pay (given a fair and workable mechanism*), as those individuals who are driven to succeed could be rewarded in a way that recognised their efforts in a tangible manner.

    *Note my first comment on this issue. Summarising: a fair and workable system must be established by teachers, or we will have one that is not of our design imposed upon us.


  10. This situation would be (somewhat) ameliorated by performance pay (given a fair and workable mechanism*), as those individuals who are driven to succeed could be rewarded in a way that recognised their efforts in a tangible manner.

    If you want more highly qualified people to work in teaching you have to reward qualifications not performance.


    • It seems to me that it is an assumption on your part that the institutions that award formal qualifications are better at recognising quality professionals than the profession itself.

      Furthermore, rewarding qaulifications does not ensure better outcomes for schools. Highly trained does not equate with highly competent.


      • It seems to me that it is an assumption on your part that the institutions that award formal qualifications are better at recognising quality professionals than the profession itself.

        I said nothing about “quality professionals”. I was talking about “highly qualified people”.

        I don’t think it’s a stretch to say that qualifications are an indicator of being highly qualified, nor that a system that rewards something other than qualifications is unlikely to be effective at attracting the highly qualified.


  11. My main problem with performance related pay is that there are strong indications that teacher performance is one of the smaller factors in student performance. It is a bigger factor (but not the biggest) earlier on in education, where learning is more led by the teacher. But as students are expected to learn independently, it shrinks even more. To quote some findings on the consistency of teacher effects – simple comparisons on test data year-to-year that could be repeated using school records:

    If teachers did differ in the effects which they have on their classes, then one would imagine that ‘good’ teachers would be able to get significant improvements with each class that they taught. This consistency should be apparent when comparing gain scores over successive years and should give rise to a positive correlation.
    If the gain scores were compared over two successive years for a number of different teachers, then consistent effectiveness (or consistent ineffectiveness) would be shown as scores clustering around a central line.
    Some limited early investigations found negligible consistency but a later review by of a number of studies by Shavelson and Russo (1977) found significant correlations which were in the region of about .4. Although this indicates some relationship, it should be noted that this only accounts for 16% of the variance. This means therefore that the majority of teachers vary quite a bit in the progress that they make with different classes.

    Anecdotally, I know a very effective Physics teacher who keeps such records, and is convinced that learning is mostly out of his control. Different classes respond differently to the same teaching, and he is not consistent. However, he is smart enough not to blame himself unduly. Prior attainment of the students is the biggest factor in future success. (In nursery kids, it is home background).


  12. My main problem with performance related pay is that there are strong indications that teacher performance is one of the smaller factors in student performance.

    The arguments in the video explaining why it is hard to measure a teacher’s impact on student performance are also good reason to doubt statements such as the above.


  13. When Alan Johnson, David Miliband, Ed Balls or any of the other pointless fly-by-nights we’ve endured as Education Minister (or whatever they’re called now) gets Gordon’s job, could we persuade Mr Willingham to run education in this country? Or perhaps he’d like to marry me.


    • It is possible that Willingham has just as much beneficial effect on UK education now as he would in government. At the moment he can attempt to broadcast rational argument in favour of a certain educational method.

      In government, he would have to take control of schools to an unliked extent to impose these ideas, and there could well be a backlash. How many teachers would want to give up their current way of doing things? Currently there is a happy illusion of freedom where many schools hire consultants to tell them how to deliver the set curriculum to meet various objectives. And this must please enough people, because the system is still functioning (even if barely).

      Also, I’m not sure to what extent Willingham would alter big issues, e.g. inclusion. Looking at the evidence – and unless he has seen different evidence to me – moderate inclusion is not harmful.


  14. The arguments in the video explaining why it is hard to measure a teacher’s impact on student performance are also good reason to doubt statements such as the above.

    You appear to be saying that the effect of teaching on performance is so problematic it simply cannot be measured.

    Have a look at my previous post. We have found evidence regarding teacher effects by examining year-on-year value-added. And it seems that teacher effects on performance is inconsistent, so inconsistent that teacher effects can be described as small.
    How would you use any of Willingham’s arguments to discredit such examinations of consistency?


    • I’m not sure what you haven’t understood here. If something is hard to measure (for reasons unrelated to its size) then it is hard to say that it is small (or indeed make any comment about its size). Understand?


  15. “It is possible that Willingham has just as much beneficial effect on UK education now as he would in government.”
    I had never heard of him before reading this blog. I’m pretty certain I’d hear a lot more about his ideas if he were Minister of Children, Parents, School Etc.


  16. “I’m not sure what you haven’t understood here. If something is hard to measure (for reasons unrelated to its size) then it is hard to say that it is small (or indeed make any comment about its size). Understand?”

    I’m not sure what you haven’t understood here. If something is hard to measure (for reasons unrelated to its size) then is it so impossible to yet conclude that whatever measurement we have of it shows that its effects are inconsistent?

    And if it has inconsistent effects, does that not most probably lead us to conclude that any effect it must have is small, relative to other causal variables?


    • If something is hard to measure (for reasons unrelated to its size) then is it so impossible to yet conclude that whatever measurement we have of it shows that its effects are inconsistent?

      Yes.

      The point is that the measurement we have of it is worthless. Yes? Therefore, we can’t draw any conclusions from it. Understand?


  17. Performance related pay would surely be criterion referenced rather than norm referenced, i.e. individual objectives would be either met or not. No need to compare individuals.

    Also teaching is a lot more than the academic results achieved by a teacher’s classes/pupils. Objectives may look at any aspect of a teacher’s practice, but obviously it would be good if some results element could be included. Once a VA target was agreed up front with a teacher and a method of calculating the measure was understood then as options go maybe it isn’t too bad.


    • “Performance related pay would surely be criterion referenced rather than norm referenced”

      I don’t think you quite get the concept of “performance-related”. I mean, if you are just pointing out that our ludicrous system of Performance Management is not really performance-related pay, then fair enough.

      But if you are suggesting the ability to meet arbitrarily drawn milestones measures performance then I cannot agree.


  18. ‘But if you are suggesting the ability to meet arbitrarily drawn milestones measures performance then I cannot agree.’

    I did not suggest this and I would not suggest this. Before I came into teaching I studied performance related pay, I implemented performance reated pay, was subjected to performance related pay. I had a management team, and as a team we used performance related pay very effectively. In a commercial environment, profitability and increasing sales are only one part of a ssystem intended to improve long term success. I believe the same is true in education.

    I am currently part of a performance related pay system which looks at various areas of performance, broadly in line with the teaching standards. The system has its faults as does the system of teaching standards but generally I think the way things are done at my place is a positive step. I am sure it will be improved over the next few years.

    ‘But if you are suggesting the ability to meet arbitrarily drawn milestones measures performance then I cannot agree.’

    So I am not suggesting this but I happy to disagree, after all it is disagreement that has brought about many advances in human kind.

    I also appreciate that your take on PRP may be different to mine, and your definition also.


  19. By the way, going back to Dan Willingham’s you tube video, which I don’t think I have referred to explicity.

    He makes the point reasonable well that it is difficult to to make objecive comparisons when it comes to “how much children have learned” to be used as a bsisi of PRP.

    After all, it is debatable how and whether we can actually measure how much children have learned in the first place (the assessment problem).

    He does not address any of the other factors that might be used and are being used in schools. So it is good as far as it goes, but it doesn’t go very far. (I received a copy of his book this morning so have little more light reading for the holidays.Thanks for the tip TB)


  20. In Dan’s defence I guess it is also worth pointing out that the video and blog entry of which it formed part was about using standardised test scores as a means of delivering PRP.

    ‘In 3 minutes and change it explains six problems (not a comprehensive list, but a start) in using standardized test scores as a basis for evaluating teachers.’


  21. i have used PM system on my staff.
    its a waste of time and energy
    its imposible to properly measure this stuff
    recently the goverment got rid of results targets anyway for PM…
    the money and time spent on this gumf should be just added equally to pay packets.

    on the other hand i would welcome increased powers to deal with unreasonable absences, staff who miss deadlines, or staff who fail to follow their contract or cause extra work for thier colleagues


  22. I’m not a statistician, but I think he might have got problem two wrong. Comparing correlated data is more reliable as you don’t have to account for the distribution of the individual students as it has the same shape in both samples (though it has hopefully been shifted upwards). If the data wasn’t correlated, then any test would have to take into account the natural variation across classes.

    This site seems to explain in nicely:
    http://vassarstats.net/textbook/ch12pt1.html

    Maybe I’m wrong? Everything else is spot on.


  23. [...] of reasons to be against performance related-pay for teachers. The video which I have shared here is a good starting point. Another might be the brief summary of its history and general [...]


  24. Reblogged this on Scenes From The Battleground and commented:

    I hear Tristram Hunt backed Gove on performance related-pay last night. Just in case anybody’s forgotten why it won’t work, here’s an old video from Gove’s (and my) favourite cognitive psychologist.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 8,037 other followers

%d bloggers like this: