Let's be Fwends #128:

What Everyone Should Know About Metrics

In a sense, there is no such thing as a random number; for example, is 2 a random number?

~Donald Knuth

Welcome to Let's be Fwends issue 128. Today, there's a link to a website you can use to check if a book might be relevant for you. Then we dive into a common issue with metrics and measurements, how capitalism feels right now in the tech sector, watch an expert recover from an initial, quite severe issue, and finally listen in to a conversation between a kid asking for help, and a professional who's job it is to help, but in a different way.

Get Five Bullet-Point Summaries of Books

I like this website which condenses all sorts of books (mostly of the productivity/coaching/entrepreneurial type) into five-bullet-points lists. I don't think that you can catch all interesting information from a book with five bullets (for some books, you can do with one or two, actually), but reading the summary allows me to judge if the books contents might be interesting.

What Everyone Should Know About Metrics

Measuring things is a powerful practice - for organisations and individuals alike. It removes biases and adherence to social desirability. Even more so, we tend to overreport activities that are "normative behaviour" that corresponds to our self-selected identity. A question about the frequency of a (desired) behaviour is typically re-interpreted as a question concerning the adherence to a norm.

In other words, the question is transformed from an inquiry about “what I do” to ask about “who I am.” Importantly, this self-view may not be rooted in the actual self. Rather, it may be strongly reflective of the ideal self—the person the respondent aspires to be. This transformation of the question allows identity prominence to exert its influence on the question answering process.

Tracking what is, instead of guessing how things might be (and probably coming to a conclusion that is more in line with how things should be), allows us to take a less biased look at the status quo.

The problem is that most things we can measure are actually proxies of what we would like to measure. For example, if you want to stay/get healthy/fit, you might want to exercise four times a week. That's a thing you can track. The problem is that the frequency of exercise might be linked to good health, but it doesn't have to be. Its easy to come up with many scenarios where there is no connection between the two at all. Measuring exercise bouts can be a good strategy, if your basic assumption that "getting more exercise will make me healthier" holds true.

The nice thing about those proxies is that they're much easier to measure than the thing they act as a stand-in for. "Did I exercise today?" is a question that is trivial to answer. "Am I healthy today?" is not only much harder to answer, but also the content of the question is totally unclear. What does "healthy" in this context even mean?

So, most of the time, we need to be content with measuring things that somehow correlate with the actual thing we want to accomplish.

What can possibly go wrong in this scenario?

Mistaking the map with the territory

Sometimes, we're substituting a difficult question with an easier one, and answer that. This is called an heuristic, and it is a really great trick of our lazy minds.

But if we're not careful, we can mistake the map for territory, and just think that because we exercise four times a week, we surely must be healthier, which might or might not be the case.

Gaming the system

Most measurements are easy to tilt in your favour. In our example, what constitutes a bout of exercise? Is walking ok? Should you count warm-ups, cool-downs, and rest intervals in between? If you're tracking a single datapoint, it's easy to game the system. Ask any Scrum team who steadily increase their velocity by simply estimating their work items higher.

Malicious compliance

Sometimes, you can even encounter something called Malicious Compliance, a behaviour when someone follows an instruction as stated, even if they know it is of no benefit or might even be aware that following the instruction actually sabotages its intent.

This can happen when for whatever reasons the thing we measure becomes a bad measurement of what we are trying to achieve. For example, if you sprain your ankle but still insist on tracking four runs a week will increase your health.

In all three situations, your initial set of metrics becomes useless, and might even harm your intent and purpose of measuring in the first place.

"When a measure becomes a target, it ceases to be a good measure."

which is a translation for people like you and me of something Charles Goodhart wrote:

"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."

Goodhart's Law

So, here's your conundrum: On one hand, you'd like to have a useful and easily tracked proxy metric for the outcome you're actually caring about. But on the other hand, you really want to avoid mistaking the proxy for the real target.

Or, you can of course do it like most people approach that problem:

What the Tech Sector Feels Like Right Now

You should take a moment to celebrate yourselves. You crushed it. You’ve helped us grow so much that the only possible way for you to continue to help us grow is by no longer working here, effective immediately.

"Our company is doing so well that you're all fired" by Andrew Singleton

Watch a Masterful Recovery

Maria João Pires is a very (very) accomplished pianist, but even the most experienced professionals aren't immune to mishaps. Watch her realising that she's been practicing for the wrong concerto, going through all the emotions from initial disbelieve, shock, despair and finally a focus on a solution, gracefully recovering and delivering a solid performance.

Bonus Reel: Here's Pires talking about the video just a couple of days ago.

When You Really, Really Need Help

What do you do when you're told you should call for help when you need help? You call for help, obviously.

That's it for this edition of Let's be Fwends. You can always call me if you need help. I probably won't be able to do much, but I'll listen. That's what fwends are for. 👂