Today was a very frustrating day at work. I won’t go into a lot of it, but one theme had to do with the assumption that the things we can easily measure must also be the most important things. The problem is that sometimes the assumption is false. It’s quite common, actually, for people to make some improvement in terms of things they can measure – instructions, cycles, messages – and then find that the result is actually worse. One common example is tuning a piece of code for performance, only to find that – while that piece of code in and of itself does perform better – the way it interacts with the rest of the system results in worse overall performance. Even when overall performance is improved, it might be at the cost of introducing subtle bugs in less common cases, decreased generality or maintainability, or other hard-to-measure but nonetheless very real drawbacks.

That last point brings me to the more general problem with this false equivalence between ease of measurement and importance, because it’s not limited to programming. This same mistake is repeated throughout other areas of our lives as well. For example, a great deal of economic theory is in my opinion worthless because it’s based on an assumption that everything important can be accounted for in terms of easily measured dollars and cents. All other “externalities” are routinely ignored, even though effects of these transactions on health and environment and our “social fabric” and even (as we have so recently seen) the viability of the markets themselves can sometimes be very important indeed. Over here are sports fans who obsess over statistics which can never really predict scores. Over there are academics for whom “publish or perish” is but one form of emphasizing visibility over value. In example after example, people fetishize the measurements and ignore the reality those measurements are supposed to represent.

I’m not saying that there’s anything wrong with empiricism and measurement. I’m more committed to them than almost anyone else I know, but sometimes you have to step back and ask whether your measurements are serving you or you serving them. Always ask yourself whether there’s something else you should be measuring instead, or whether you should be measuring something a different way, or whether measurement is really the right thing to be doing at all. Measuring the size of the iceberg above the waterline is always the easy thing to do, but it’s rarely the most important.