Dos and Do Nots for KPIs

A key part of managing your shop floor is understanding the current state of the shop floor. And KPIs are an important part of understanding the shop floor. But this is also not always easy. In my experience, the three most common pitfalls are to underestimate the effort (and hence the cost) of measuring a KPI, to overestimate the accuracy of a number, and (worst of all) to believe that the numbers tell you everything. Time for a reality check…

Introduction

KPI stands for key performance indicator, a quantitative measurement of some sort of performance. KPIs on the shop floor usually go in four directions:

• Safety: You should measure safety. This is usually a tracking of the frequency of accidents, sometimes also divided into smaller (no sick leave) and larger accidents (sick leave). Sometimes near misses are also measured, but it is harder to get good numbers here.
• Quality: How good are your products? Depending on your product complexity, this may be only a sample check or visual inspection of the finished good, or multiple detailed measurements along the production process.
• Cost: You want to know some sort of cost-related measure. While accounting gives you some more detailed numbers (but with many assumptions behind it), on the shop floor this is usually measured in good pieces produced per time.
• Time: How often are your deliveries on time? How often are you behind, and by how much? A popular measure is OTIF—on time in full, counting the percentage of orders that are delivered in its entirety before the deadline.

Safety is often only a single metric (counting accidents) and its derivatives (days since last accident), but the others are often multiple different measures. Quality is for different products, and often for different aspects at different stages of production. Cost is often not only pieces per time for different products and production stages, but also many other measures that influence cost like inventory, downtime, OEE, or many more. Delivery times are also often tracked for different products.

What Is Missing?

Do not fall for the mistake of believing that these numbers tell you everything. Management by numbers is a really dangerous way to run a factory. To truly understand a shop floor, spreadsheets and dashboards are not enough; you need to go there yourself and look at it. Visit the Shop Floor or Your People Will Fool You!

For example, one important aspect of production is employee morale… but that is really hard to measure. Some companies do an employee survey, but this merely scratches the surface of employee morale.

Another aspect that is hard to quantify are improvement potentials. Sure, you may see that your quality/cost/delivery time is lacking, but… how do you fix it? Overall, there are plenty of qualitative aspects that you need to understand for managing a shop floor. Even for quantitative aspects you cannot measure everything.

Is It Really a KEY?

You cannot measure everything, but it feels like some companies die trying to do that. Management often seems to believe that more numbers is better… but I disagree.

KPI stands for key performance indicator, where the “key” indicates that the number is important, and more so than a normal (not key) performance indicator. Any number wanted by management regularly is automatically a key performance indicator.

Unfortunately, it is often underestimated how much effort it is to get a measurement. Granted, some measurements are easier to get. Counting produced parts is often not too much of a hassle. But others are much more difficult. Measuring an OEE requires a lot of data, and hence a lot of effort. Even if you want put this into your ERP system to measure it automatically, it takes quite a bit of effort by programmers and users to set up the measurement and also maintain it.

In my experience there are plenty of departments (e.g., planning), where a large chunk of the time is used to collect data. In my own department one person spends 30+ minutes every day to measure a leveling accuracy (i.e., how close the actual sequence of production was to the planned sequence). Thirty minutes is a lot of time for a single number… which nobody ever looked at. How do I know nobody ever looked at it? I simply told my guy to stop measuring and reporting this number… and nobody ever noticed. The KPI was once established long ago since someone needed it, but such things have a life on its own, and if you don’t pay attention it will continue to be measured long after the need for the measurement is gone.

If possible, weed out your KPI every now and then to focus on the ones you truly need regularly.

Is It Accurate?

This brings me to my last point: Are the numbers accurate? Some numbers cannot be measured accurately and need a lot of assumptions (see for example my article on The Problems of Cost Accounting with Lean).

Many others can be measured accurately if you put in the right effort. But due to the overload of the people on the front lines or first line of management, they often simply don’t have the time. Hence (somewhat understandably) they cut corners, and the quality of the measurement goes down. And, for a measurement that is only wanted but not used or needed, I can even understand that.

This does not even include the intentional manipulation of numbers so they look good. I wrote a whole three-post series on Lies, Damned Lies, and KPI with examples of fudging, its effects, and possible countermeasures (simple measurements, verify occasionally, and punish liars).

I have seen plenty of examples where numbers were outrageously fudged to improve someone’s career chances or performance bonus. One example is inventory, where a company was measuring its inventory levels always at the end of the month. However, at one point someone looked not at the end of the month but at the daily levels of inventory, and the difference was staggering. Inventory always dropped magically on the last day of the month, just to pop back up again the day after. Outbound deliveries were accelerated, inbound deliveries were held back, and expensive parts were loaded on a truck and parked on a parking lot for a day, just to look good.

Overall, you need to know the strengths and limitations of your KPIs to understand the shop floor, as well as the effort that goes into measuring these KPIs in the first place. Now, go out, measure the right things on your shop floor, and organize your industry!