Friday, 10 June 2011

Of Rats and Men

Performance measurement, behaviour and Big Society

I'm always intrigued by the way performance measurement influences the behaviour of people and organisations. Basically, people respond to what is actually measured, not to the intention behind that measurement, and this is evidenced by many examples of poor performance indicators generating bizarre consequences. The 'rats example' (hence the title - apologies to Burns and Steinbeck) is a particularly pertinent one.

Measuring Performance in the Public Sector (Hans de Bruin) quotes a US pest control company that rewarded its workers based on the number of rats they killed. The result was that the workers made sure there was a continuing and healthy rat population so that they had plenty to “harvest”! Hearing this story, a colleague (I believe it was Mike Chitty) told me of a UK Local Authority that sought to gauge their pest control efforts through the number of reports of rats received from the public (the fewer reports, the fewer rats there must be). They found their staff trying to persuade callers that they had seen large mice rather that rats – because mice didn’t count on the statistics!

Some consultants use this as an argument for abandoning any form of performance measurement. There is even 'Goodhart's Law' (after a former Bank of England advisor) that essentially states: “Anything that we choose to measure becomes unreliable, once we measure it for the purposes of control”.

I'm not a subscriber to this negative view. Instead, I believe we should think about performance measurement in terms of how people will respond to it, and deliberately design measures that influence behaviour in a positive way. So for example rather than waiting time in A&E, we could assess what patients think of their treatment, together with the medical consequences of any delay.

Another example: some service organisations measure the effectiveness of customer communication through the response rate they get to surveys. It doesn't apply everywhere, but in some situations it actually encourages desired behaviours amongst the organisation's staff – accessibility, customer focus, and above all responding to customers' views rather than just going through the motions of consultation.

Even the rat dilemma can be answered by understanding why we want to control the rat population. If community health is the reason, then a sensible measure might be the incidence of rat-associated diseases. If it’s simply that people don’t like having vermin around, then the number of reports could be a perfectly valid measure, and it doesn’t actually matter what species the callers have seen.

This has significant implications for Big Society. Measures such as number of volunteers or time spent volunteering are of limited use because they simply measure outputs; they don't address outcomes or what actually changes. The problem then becomes one of clarifying exactly what change is intended, because despite David Cameron promoting its principles, few people really grasp what Big Society should look or feel like.

I don't claim to have all the answers here, but just a suggestion: how about linking some element of local authority funding to the number of taxpayers in the area who are employed by third sector organisations? This might encourage local authorities to promote the growth of the sector in a much more positive way than they currently do, and hence create communities in which the third sector plays a much larger role. If that is what we want, that is.

There's another rat connection. Like it or not, we are all a bit like laboratory animals on which the latest government social initiatives are being tested. Let's hope we can respond more intelligently than the rodents.

Check my web site at www.real-improvement.com for more information and ideas.

1 comment:

  1. Thanks for this. Some good examples of how targets twist outcomes. I wrote briefly on this too, highlighting the contradiction between the impetus behind the Big Society on the one hand and trying to evaluate it on the other. http://www.involve.org.uk/evaluating-the-big-society-will-destroy-it/

    You might also really enjoy Matt Andrew's blog which is giving clear ways forwards for how to do these things. This week he's on fire! http://matthewandrews.typepad.com
    Simon

    ReplyDelete