Wednesday, 25 May 2016

How SMART Are You?

Allow me a mild rant. I don’t often get annoyed, but this point bugs me – particular when I see influential people repeating the error.

Do you know what SMART, as in SMART targets, stands for? Well, if you said yes, it’s:
S – Specific
M – Measureable
A – Achievable
R – Realistic
T – Time-bound
…then think again.

What’s the difference between Achievable and Realistic? Surely if a target is achievable it must be realistic and vice-versa? I suppose at a push you could argue that a realistic target is one that is measuring things in the right way, hence it’s rational and relevant.

But hang on, there’s several other R’s in there – Right, Rational, Relevant. And these are much more sensible interpretations of the R in SMART. Is the target, and how we’re measuring it, relevant to what we’re trying to achieve? Using ‘Relevance’ as the ‘R’ incorporates the vital principle of cause and effect.

To take an absurd example, would you measure children’s education through their shoe size? Why not? It meets all of the SMART criteria listed above, and can be expected to increase over time as children’s feet grow. Of course, it fails because the measure is nonsense; I’m aware of no research that links shoe size to academic performance. The measure is completely irrelevant.

A more pertinent example was recently published by the Centre for Health Economics, which looked at mortality (i.e. the % who die) of patients admitted to hospital at weekends rather than weekdays. Mortality for patients admitted at weekends is known to be higher, so does this mean we need to improve hospital care at weekends (“24-hour NHS” and all that)?

Well, the paper shows that fewer patients are admitted at weekends, and those that are tend to be more seriously ill, so are more likely to die anyway. It’s not poorer care that makes the difference, it’s sicker patients. Trying to reduce mortality for weekend admissions is therefore the wrong target, because it misses the point about cause and effect. (Jeremy Hunt please note.)

The moral is that handy acronyms (SMART or otherwise) are no substitute for real understanding. Understanding the outcomes that you’re trying to achieve and what works to achieve those outcomes. Only with that understanding can you formulate sound evaluation measures and targets.

PS: If you said SMART stands for Specific, Measurable, Attributable, Realistic, and Time-Bound, then I’ll allow that too. Shows you’re thinking!

No comments:

Post a Comment