Metrics: A Cautionary Tale

“I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”

– William Thomson AKA Lord Kelvin, 1883

 

While it has been around for over 130 years, the concept of management through measurement is most often attributed to Peter Drucker, and his 1954 book The Practice of Management – perhaps the first book to identify management as a professional discipline. Without reliable metrics, we have no yardstick with which to measure the accomplishments and performance of our organizations. It’s an old idea, but an important one.

In April 2014 the updated Directive on Performance Management came into effect. This directive mandates annual written performance objectives for all employees and annual written performance assessments. In effect, this applies the concept of what gets measured, gets managed to individual performance agreements. Managers are to measure both work objectives (that for which each employee is responsible) and core competencies (how each employee goes about accomplishing their work).

The key question is this – for any given employee, what should be measured, and how? It can be easy to focus on metrics that are easily measured rather than the ones that really matter. And focusing on the wrong metrics can be dangerous.

Management of the National Health Service (NHS) in the United Kingdom implemented a campaign to measure and reduce patient wait times – a noble goal and one that should improve service. Indeed, this resulted in shorter waits. Unfortunately, however, it also resulted in an increase of heart-attack deaths, as reported by Simon Caulkin in the Guardian. He writes:

“Under competition, hospitals improved their patient waiting times. At the same time, the death-rate following emergency heart-attack admissions substantially increased. Why? As targets, waiting times were and are measured (and what gets measured gets managed, right?). Emergency heart-attack deaths were not tracked and therefore not managed. Even though no one would argue that the trade-off – shorter waiting times but more deaths – was anything but a travesty of NHS purpose, that’s what the choice of measure produced.”

Managers have great flexibility under the Directive on Performance Management to select metrics by which their employees will be evaluated. But with great power comes great responsibility. The true challenge will be deciding upon the right metrics. If an activity is measured and incentives are put in place to reward that activity, more of that activity will take place within the organization. Unfortunately, that means that less of other activities will occur – sometimes less of what managers would consider productive work. Worse, employees may find ways to meet the metrics at the expense of doing good work.

Everything an employee does consists of five categories of behaviour (I learned these from Mark Horstman of Manager Tools):

  • The words they say (or write)
  • How they say them
  • Their facial expressions
  • Their body language
  • Their work product (quality, quantity, accuracy, and timeliness).

So, these are the five types of behaviour that can be measured. Of these, which ones will you consider in measuring your employees’ performance? How will the metrics you choose vary based on the key deliverables for your team? Each employee’s metrics can and should vary based on their role and level of experience – frontline program officers will need different metrics from policy analysts, clerical staff, or IT specialists. Will you choose the right metrics?

Treasury Board Secretariat has thankfully created some resources to help managers choose wisely. These include a frequently asked questions document and a guide for managers and supervisors. The latter even includes examples of work objectives.

Guidelines are helpful, but they do not replace the hard work required of managers. Selecting the right metrics, measuring them on an ongoing basis, and providing ongoing feedback to employees on their performance relative to those metrics is core to effective management. It was fundamental when Drucker wrote The Practice of Management in 1954, and it remains key today.


George Wenzel George Wenzel was a journeyman public servant and is now working at a not-for-profit – pursuing his passions in what will be his fifth career. He recently completed a two-year secondment to the National Managers’ Community as the Alberta Regional Coordinator. You can find him online at http://about.me/georgewenzel, http://www.govlife.ca, and on Twitter @georgewenzel.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s