Picking the right metric(s) for your project

View All Blogs


If we are looking to manage a business, we would need a comprehensive range of balanced metrics that measure things like customer satisfaction, financial performance, process efficiency, employee engagement, learning & development etc. If we are trying to solve a problem where our actuals are not meeting the targets, then we would pick only the few key metrics that directly give us information about the problem at hand to not only solve the problem but also track the improvements to the process.

In this article, we are going to focus the second topic - how do we pick the right metrics to solve a specific business problem. We discuss 10 things to watch out for when we select or define our metrics.

1. Focus on the most important metric(s) first If there are multiple problems that the customers are complaining about, we need to prioritize on the most important metrics first. It is recommended that we address the priority in the following sequence:
   • Safety: We address all the safety related issues first
   • Quality: Make sure that the quality of our products & services is meeting customer expectations
   • Delivery: We next address the delivery issues to make sure that we are meeting our commitments to customers
   • Cost: Finally, once we address the safety, quality, and delivery, we focus on meeting the cost expectations.
   If there are multiple issues, we are not asking you to focus on one issue and ignore the rest. We are asking you to prioritize the issues and attack the problems in a sequence so that you can make the best use of limited resources.
2. Ensure metrics reflect what you are truly working on Once we pick a metric we are going to go after, we need to discuss scope of the metric. For example, if cost is your metric, the cost may have several elements to it such as labor cost, material cost, supplier costs, manufacturing costs, overhead costs etc. When we work on a project, whatever metric we pick, we should be able to show an impact in a reasonably short period of time. Hence, we need to scope the project – this is a delicate balance of making sure that the scope is large enough to be important to the customer and small enough that we can address it in a short period of time. Hence, picking overall cost as your metric may make the problem too big to handle in a short time period. So, you will need to get some data, perform a Pareto analysis to determine which costs are the most important for you to go after. Let’s say that the material costs are very high, and the team feels that this is within our span of control, then it makes sense to consider material costs as your primary metric instead of overall costs. If you pick the overall costs as your metric and only work on material costs, your overall costs may not show an improvement even if you do a great project on reducing material costs if some other costs that your team is not focusing on gets worse (say labor costs). Hence, make sure that your metrics reflect what you are truly working on.
3. Ensure metrics are within your team’s span of control & actionable Make sure that the metrics you pick and track on your improvement project are within your span of control. For example, if supplier costs are the biggest element of your cost structure but are outside your control, there is no point including supplier costs within your metric. Unless you can do something about it, the metric will continue to show the same performance without improvement. If you are unable to influence that metric, then we need to exclude it from our consideration. That does not mean, if supplier costs are the biggest component of our costs and not in our control, we ignore it. It just means our team will not work on it, but we need to ensure that we pass on the task of improvement to the team that is responsible for these costs and has it within their span of control and potentially work collaboratively with that team to make an improvement.
4. Ensure metrics are simple, clear and easily understood It is a good idea to use company standard definition if one exists to define your metric so that all stakeholders have the same interpretation. If no company standard exists, it is good idea to have a clear operational definition of the metric and share it with all stakeholders. Keep the metric as simple as possible. For example, if the metric is employee productivity, this may mean different things to different people. If you just define productivity as revenue ($) per employee per week that is not enough. It needs to be further clarified how revenue is defined (for example does it include returns data?), where is the data coming from? How are number of employees defined? Which employees are included and which excluded for this definition? What if there is a change in the number of employees within the week (someone left or joined the organization), how is week defined (is that a work-week or a calendar week) etc. So, make sure you clearly define the terms, the formula used to compute the metric, where the data is obtained from etc. and leave nothing in the metric to ambiguity.
5. Select continuous metrics whenever possible If you have a choice between continuous and discrete metrics, always choose continuous metrics. Firstly, continuous metrics have a lot more information compared to attribute metrics and enable us to perform a lot more statistical analysis. Secondly, continuous metrics can always be converted into a discrete metric at a later point in time if so desired. For example, let’s say we are interested in on-time delivery to our customers. One way to measure this would be number of times we missed delivering on-time. Here, we are counting the number of misses – say 14 misses out of 100 deliveries. This gives us some information, but it does not tell us how much we missed by? Instead, if we measure the number of days (or hours) to deliver to a customer and our target was 40 hours, then we can not only quantify the number of misses, but also how much we missed (by 2 hours or 100 hours)?
6. Use ratio metrics rather than absolute metrics whenever possible We can measure the metrics in absolute terms such as total number of defects, total units produced, total revenue etc. and in some cases, these make sense. However, you must be cautious in using them especially if extraneous factors can also influence these metrics. For example, if we are working on a project to improve quality and our metric is the number of defects. What happens when the production drops? Even if we don’t do anything on our project, the number of defects may be lower since the production quantity is lower. This does not truly reflect the improvements we have made on our project. Hence, a better metric may be number of defects per unit of production. Similarly, in some situations revenue per person may be a better metric compared to total revenue.
7. Use metrics that are cost effective Let’s say that you are trying to improve the quality of a manufactured part. You could use a metric that measures the exact location of the defects using x-rays. This would be a costly proposition and may slow down production. Another alternative may be to measure manufacturing defects on the left or right side of the part using an image capturing device. Hence, metrics must be easily measurable and cost-effective for us to measure.
8. Use metrics that offer enough data points For improvement projects, we need enough data to identify root causes and monitor success of improvement solutions. If you have metrics for which data is collected infrequently such as survey data from employees on engagement conducted once a year, this would be too infrequent to be of use for improvement projects. In such cases, look for metrics that are leading indicators (such as pulse surveys) and those that will enable more frequent data collection. You need to be able to measure at least several data points each week.
9. Sometimes multiple metrics may be required to prevent sub-optimization In order to ensure that we drive the right behavior in the organization, and we are not sub-optimizing the problem we may need to monitor multiple metrics. Let’s say that we are interested in reducing supplier costs. This may be one metric that you track to show an improvement in performance over time. But what if as a result of your project, the supplier costs go down but there are other problems like delivery time has gone up or quality defects have gone up? Hence, if you have a primary metric on supplier costs, you also need to track other metrics such as supplier lead-time and supplier quality. These metrics may not be the focus of your improvement project, but we just want to track them to be sure that they don’t get worse.
10. Ensure metrics cannot be manipulated We should also make sure that the metrics we select cannot be manipulated. Employees may manipulate the metrics for several reasons such as to achieve targets that trigger a bonus, avoid repercussions from managers, to overcome deficiencies in the process, if the risk of getting caught and punished is low etc. In order to reduce the risk of metric manipulation, managers should set the right targets and measures, train employees on code of ethics, ensure lack of conflict between metrics and payout, introduce automation and/or incorporate random audits.

---

Follow us on LinkedIn to get the latest posts & updates.