Probably the single biggest challenge in setting performance objectives is making them measurable.
This is important because …
Human beings adjust behavior based on the metrics they’re held against. Anything you measure will impel a person to optimize his score on that metric. What you measure is what you’ll get
Dan Ariely article “You are what you measure” in Harvard Business Review
And if we get it wrong, it can be dangerous and lead to the measure having an ineffective, or damaging, impact …
It[‘]s really easy to decide to measure something … and screw up a team beyond belief. For example, if I measure how productive individual programmers are, then it[‘]s to the advantage of individuals to focus on their own work and spend less (or no!) time helping others. Completely kills teamwork
So it’s worth getting it right … but it’s not so simple …
The most important things cannot be measured
This is even more true as most things sit within complex systems and have an impact over the long-term and so can’t be easily isolated or measured within a single twelve-month performance appraisal period.
What makes a good measure
We tend to think of measures as being numeric, handy little objective numbers we can neatly slot into spreadsheets to form colourful graphics. I have worked in many places where data in colourful graphics (especially if simplified into traffic lights) is officially More Valid than any other form of data (1).
I read quite a few websites when preparing this post, and most would agree.
All mentioned the importance of “quantifying”, asking “how much” and “how many” as the key aspects of measureability (2). I don’t agree. I think they’re arguing that the most important thing about a measure is the measure itself, not the performance it tracks.
The numberyness of a measure is not the important factor.
I’d go further. The pursuit of a numerical measure means we often measure the wrong thing, just because it’s countable.
If a measure can be an objective number then that’s great, but much more important than that are these five factors:
- The individual can directly influence the measure through their actions
- Those actions have a positive impact on the organization
- They are the key actions that really make the difference
- The act of measurement is efficient
- The measure has an appropriate balance of quality and quantity
How to measure the right thing
Objectives need to state two things:
- A specific deliverable (it states “what” needs to be done)
- The reason to bother doing it, i.e. what is the desired outcome (it states “why” it needs to be done)
The thing to measure is in the second bullet: why bother? What is the reason behind this objective?
For example, if we want to produce a quarterly report for the board that summarizes the previous quarter’s performance and makes predictions for the next quarter, how could we measure this?
We’d need to understand why it is being produced.
We could just measure that the data is collected and the work done (the input), or that the report is produced and distributed on time, but all we’d be doing is checking that the output we requested was completed.
It’s a numerical measure: the number of reports produced per quarter, one or zero.
But we have to move away from that.
We need to build in quality and try to understand if the actions taken (the input) led to the right end product (the output) which in turn had the desired impact (the outcome).
The outcome is the most important factor – we are producing this report for a reason: so that the board feel that they have a good overview of the previous quarter, and have a solid prediction for the next, allowing them to make better decisions within their own areas.
That’s what we need to measure!
Do the board feel that they informed about previous performance, and have solid predictions for the near future?
This is the measure that counts!
It’s subjective, non-numbery, and beautifully ambiguous, but it’s really the only thing that matters!
People might not like this. It’s not a good measure because it’s subjective and woolly, but that’s not a fault of the measure. The measure reflects the objective. The objective is about producing a specific deliverable to meet a subjective woolly need.
But that’s fine.
In reality, the board will make their own subjective woolly judgements about the quality and usefulness of the report whether or not you capture that, and in reality, that subjective woolly judgement is what is going to count.
And if it’s what counts, you should count it – even if it’s subjective!
But sometimes you can’t.
Sometimes your desired outcome is just impossible to capture – or at least impossible to capture in an efficient and effective way. If you have to conduct in-depth market research to understand the impact of your deliverable, then this may not be viable.
So, unable to effectively measure the outcome, we go back a step and measure the output.
For example, if your objective is to run an event informing your suppliers of new arrangements for the order/payment/delivery process. The desired outcome may be to maintain relationships and remain on good terms so they continue to work with good will, and continue to provide quality products and services at good rates.
This is the real reason you’re doing a lovely event rather than just firing round an email.
The measurement of this outcome would be too great an overhead, far costlier than the event itself (and there’s probably no benchmark anyway), so let’s measure the output: do the suppliers attend the event and leave with the information about the new arrangements?
In this measure we are relying on a professional judgement that this output will lead to the desired outcome, and so it’s less good, but it’s still a fairly decent measure.
If we couldn’t even measure this, then we would measure the input.
This is least good, we’re now making even more assumptions.
We’re measuring that the event took place, not that anyone attended or that they left feeling informed, never mind the wider impact on good will and long-term relationships. We might measure that the venue was booked, that catering contracts were signed, that invitations were sent … we’re measuring essential actions (inputs), so it’s a third choice measure, but better than seeking irrelevant numerical measures just because they add up nicely.
So, in summary:
- Understand why an objective has been written: what is the desired outcome?
- If you can, measure the outcome, even if it’s subjective
- If you can’t, try to measure the output.
- If you can’t, as last resort, measure the input.
Measures are not everything
A final note.
A performance measure is not the same thing as a performance. It is a map of the terrain, not the terrain itself.
Understanding how people are performing in the workplace is hard work and yields ambiguous subjective results. This is why so many of us are tempted to make everything numerical, because that’s easier and harder to argue about – but measures are only the starting place to understanding performance, not the conclusion.
Brian Button again:
Metrics should never be used to judge a team or used to form conclusions about what is happening. … They should always serve as the starting point for a conversation, the beginnings of understanding an issue.
Endnote
This article is useful for action words relating to Bloom’s Taxonomy – I think these are a great tool for writing objectives and thinking about measures.
Notes
- Yes, I said “data is” and not “data are“. This is because this post is written in English not Latin, and in English data is widely used as an uncountable noun (like information), not as the plural to datum.
- Tim McMahon’s post (10 Characteristics of a Good Measure and 7 Pitfalls to Avoid) is good, I don’t agree with everything, but it’s a well-rounded list of things to bear in mind when considering how to measure an objective.
See here for my other posts on objectives:
Filed under: Management
