Important things can be difficult to measure (but we should still try)
May 26, 2014
I keep track of lots of numbers when I go to the gym. For example, I can do x bicep curls at a time with an x-pound weight. Or I can run a mile in x minutes. Over time, I’ve seen the numbers generally go in the direction I want them to; I can run about 35% faster than I did three years ago, for example.
Measuring exercise progress is nice because I have time to do a broad range of exercises (weightlifting, running, swimming) and I know that my improvements across the board generally correspond with my goals in going to the gym: having better physical health and more energy.
But what if my goal is to become a more educated person? Well, I could track that by maybe tracking the number of books I read per week. Of course, I’d have to make sure I focus on the true classics. What determines this?
Well, culturally I suppose there’s an generally accepted canon, including ancient philosophy, well-known novels, and plays that are cultural touchstones. Maybe I should track how many of those I read. (What about what I get from them, though?)
Or I guess I could read books on how to code, or how to speak French.
Well, is it better to read more, shorter, books, or fewer, in-depth ones?
Is reading books even a good measurement of what it means to be “educated”? Maybe my goal is not defined well enough to be measured properly. But it’s probably a worthy one.
What if my goal is even more abstract, like becoming a better person?
There are cases where you can measure exactly what you are trying to improve. If you are trying to get a higher click-through rate on a page on your site, it’s really easy to measure that. But as the goals get more abstract, they’re much harder to measure. The more that happens, the more a measurement is simply a proxy, a simpler standin, or a model, for what you actually care about. It’s an easy question swapped out for a hard one.
In writing this article originally, the title was, “as goals get more important, they’re much harder to measure”. There are some trivial examples of this, like GDP, which is a very poor measure of economic success, especially for poorer countries, and yet one that people tend to focus on. But, on the other hand, there are some incredibly important goals that are actually easily measured by very simple numbers: things like infant mortality rates, malnutrition, and even deaths due to preventable diseases.
Given that it’s basically impossible to improve without measuring things, and given that there are at least some things we can say about what matters to us that is quantifiable, I think it’s still worth it to try to measure most things we care about.