You are not what you measure
On the day you die, what will your friends and family remember about who you were?
At work, they may try to measure you with numbers, but to your spouse, children and friends those measurements are meaningless. They know you from how you have helped them and hurt them. How you loved them or rejected them. What is measured about you in numbers has little bearing on the experience the people who matter most have.
There is value in careful measurement. Carefully chosen data can help us see. We are easily distracted, and good information can remind us to tend first to the important things we’d overlook in our daily chaos. I like the ideas in the film Moneyball, where wise use of information helped people see more clearly what the true value of people’s talent was.
But the trap is what is easiest to measure is nearly always the least important thing. You can measure kisses per day, but that won’t tell you how much you’ve loved someone. It won’t tell you what they’d prefer you do instead, or how they felt about you when you ignored them the rest of the day. Bad, easy data is always the most abundant kind.
And even good data can be used in dumb ways. Any measurement can be gamed. The person with the highest score may be the one who has the least integrity.
It’s a mistake to allow data to be a god. Data is dead. Numbers don’t know why they were created. Data, if granted the power, can lord over people mercilessly without any awareness that it’s out of date, behind the times, or having the opposite effect its creators intended. It is wise to be informed by data, but only a fool is data’s slave. You are more than what is measured about you.
I totally agree that bad data is abundant and dangerous. That doesn’t mean we shouldn’t strive for good data though. Love the phrase “it is wise to be informed by data”, i’ve long practiced “data informed design”, rather than “data driven design”.
I have to side with Richard here, but of course I’m biased in favor of data. Yes, data has its limits. But data is making our lives better. The self-quantification movement is nothing if not revolutionary–and may in fact result in better living.
Better living through data.
There’s an odd thing I’ve noticed in assuming the goodness or badness is *in* the data. That data is either or good or bad. I don’t think that’s true. All data is good or bad depending on how its being used, something the data itself doesn’t control.
In that sense all data is a weapon. All data can be used in good or bad ways and sorting out which is which is much more complicated than merely collecting the data.
Ok, I certainly agree that even “good data” can be used inappropriately. I’d posit that it’s a one-sided relationship, “good data” can be good or bad, but “bad data” can only ever be bad. By “bad data” in this context I mean data that is actually flawed because of measurement technique or bias, poor math/statistics, etc.
It’d be fun to think through all the different ways data can be bad.
And then all the ways ‘good data’ can be used badly.
I think you have some of the measurement thesis right. For sure you touch on materialism without using the word. You cannot eat empathy but you will starve without either money or access to food for free.
I think you might have said on twitter (paraphrasing), there is just objective truth, our feeling and context make it “good” or “bad”.
It’s a simple and powerful concept that is applicable to data as well. Data by definition is objective fact. Even if we collect it incorrectly, it’s still “good” data in the sense that it reflects a certain reality. May not be the reality we wanted, but history has shown plenty of good things come out of accidents or things we didn’t think we wanted at the time.
This is a moment where I wish I had the power to transport everyone who commented on this thread to a pub, with a drink of your choice in each of your hands, so we could talk about this post.
Scott, are you going to Adaptive Path’s MX conference or the IA Summit?
No – but I’m sure our paths will cross. And hopefully, at that moment, a pub will magically appear around us.
A nice corollary to “you are not what you do” … and an interesting counter-perspective to “you can’t manage what you don’t measure” … and the overall trend in “management by [measureable] objective”.
I’m reminded [in this and many other contexts] of the joke about the drunk searching for his lost keys under the lamppost, even though he lost them elsewhere, because the light is better there.
I’m also reminded of Robert Pirsig’s metaphysics of quality, and its primacy over any attempts to characterize or measure it, first articulated in Zen and the Art of Motorcycle Maintenance.
Everything I know about measurement I learned from South Vietnam, a land of measurable “pacified hamlets” and body counts and lots and lots of objective “cover your ass” paperwork. No one could measure honour, duty and country; no one could measure “winning hearts and minds;” … and they didn’t even bother to try.
Late last year I took a course on Knowledge Centered Support, and one of the take-aways I appreciated was when they got around to metrics. They distinguished between Activities as Leading Indicators and Business Outcomes as Lagging Indicators. The first class are easy to measure and easy to game. Activity measurements are diagnostic. Measurements of Business Outcomes take more effort to assemble, and more effort to game. The idea is to utilize the diagnostic activity metrics to help you figure out how to tune things to achieve the desired Business Outcomes.
With your loved ones, these metrics still exist. They are not numbers on a spreadsheet, but smiles and kind words are leading indicators charted in the dashboard of your heart. Being there for someone to hear their joys and concerns, or to share in their moments of triumph and need, these are the lagging indicators of highly desired outcomes.
A more accurate label for my opinion would have been “You are more than what you measure.”