21 September 2015

Metrics time!

Our office has recently been on a metrics kick. It seems that we feel like we are not cool enough so we thought we would start reporting data in order to determine where we can improve.

I hate metrics. Well, most of them at least.

Generally they are numbers that are produced periodically in order to help management feel like they are managing (if only because they have summarized work into standardized outputs that can be tracked over time; this is data collection and is good but is often mistaken as metrics too). In the end though, the reports are often neglected, leaves the reporting teams with extra work and often causes the teams to drift into managing the metrics instead of managing their business.

Ultimately, it seems like metrics seem like a distraction from addressing the real, underlying question. This question, the one that is under all the metrics, is usually "How is the business doing?" The answer to this question changes depending on the person being asked (assuming each person manages a different facet of the business) and also changes based on the current context, something metrics are not good at measuring because part of the point of having metrics is to have something consistent to compare performance against over time.

Consider metrics about payroll. Metric ratios could be calculated based on pay versus hours logged in a system, communications handled or number of handshakes. All of these metrics would probably be trying to answer a question about whether the staff is worth their pay, but in a polite way. Before establishing any of these metrics, one should determine if any of these really matter, if they are actually reflective and indicative of the underlying question.

A good way to know if you are measuring the right things in your metrics is to ask, "Inherently, if this number goes up, does the business improve?" In asking this question it is important to explore the 'inherent' context to make sure there are few, if any, underlying assumptions. In order for a metric to be good, it needs to be a number that correlates well with the desired results. In other words, you need to make sure that the metrics cannot be boosted without the business being boosted as well. Otherwise you are probably wasting resources chasing pointless numbers.

Consider a pay versus handshakes ratio metric. The assumption is that more handshakes bring more value and thus makes an employee more profitable to the company and more deserving of their paycheck. Now we can ask our test question from above: "Inherently, if the [number of handshakes an employee performs] goes up, does the business improve?" The quick answer is 'no'. It is very easy to envision situations where a person could dramatically increase their daily handshakes by shaking hands with everyone they see. All the extra handshakes would boost the metric but are unlikely to result in a corollary boost in business.

All of this leads me to say, companies should gather and track data over time, that is how meaning can be found in trends and the impact of changes measured. However, collecting data should never be confused with metrics to hold employees accountable to, especially when that data is laden with false assumptions about correlation.

Improving metrics is not the same as improving business.

No comments:

Post a Comment