The Art of Success Measurement: Insights from Griselda and Google
A Guide to Holistic Success Measurement
My husband and I were having dinner with good friends. In conversation, my friend told me I should watch Griselda. Since I’ve never heard of that show before, he explained it’s about a female drug lord (drug lady?) who mentored Pablo Escobar.
He recommended the show because he enjoyed it and felt it contributed to female empowerment — and he knows how much I care about that… It’s another story about a notoriously infamous man who was the successor of a trailblazing woman who mentored him.
To entice me to watch and drive his point of her superiority, he said:
“She was directly in charge of 250 killings.”
To my utter surprise, and probably his surprise as well, I replied with a series of questions:
“250 you say? What is the benchmark for killings in drug lord worlds? Can you normalize this for me? What percentile is that compared to her peers?”
In other words, what is the average number of killings among drug lords?
“What was the success rate?”
How many failed attempts preempted the desired outcome? one attempt per success is more impressive than 5, right?
“What was the collateral damage? How many unintended deaths happened?”
It turns out you can’t fool me with statistics, not even when discussing drug lords and pop culture over dinner.
One needs to look beyond surface-level metrics to understand the true impact of any action, whether in business, criminal activities, or any other field.
My time at Google, representing employees’ work in performance reviews and articulating project impact to gain funding or win awards, trained me to insist on holistic data to tell an unbiased and objective story.
This holistic approach also matches best practices in data analysis and ethics.
My reaction to my friend’s numeric argument reminded me of the first statistics class at business school, where our professor presented a book called “How to Lie with Statistics.” While I never read the book (yet?), I’m convinced I know what’s in there.
So, whether you are defining success metrics or assessing success, you should take a holistic approach. In my experience, rarely is a single metric enough to define success.
Example 1 — Resume
This one is taken from a great article about winning resumes by Lazlo Bock — Google’s famous ex-head of HR. Here’s one writeup (there are a few versions of this online.).
Compare this expression of success - taken from someone’s resume (this could also describe the employee’s performance):
‘Wrote 50 editorials for The New York Times.’
With this one:
‘Had 50 op-eds published compared to average of 6 by most op-ed [writers]’
The first one might sound impressive or might sound unimpressive. It’s hard to know if 50 is a lot or a little. It describes a throughput.
The second version provides a benchmark — point of comparison — to peers in the same role. Now we can conclude that this person performed above average compared with peers.
There is still an outstanding question: fine, they wrote a lot, but what was the quality of the writing?
There are several ways to assess the quality:
- Did anyone read it? How many unique readers or average readers per piece? — Is it value added or lost if you write a lot but no one wants to read it?
- Was it interesting/engaging? You can measure this as how much time readers spend on the op-eds (Medium does that). — Is that a good thing if you write a lot, but no one wants to read the whole piece?
- Have any readers shared it with others? This is a version of a famous business metric NPS — net promoter score — how likely the customer (the reader) is to recommend this product (the op-ed) to others.
There are increasing signals of quality there:
- Is anyone reading?
- How many readers get to the end of the op-ed?
- How many readers found so much value in the op-ed that they voluntarily shared it with someone else?
Note that engaging metrics can sometimes be misleading. Readers might share the content with friends to show how outrageous or of poor quality it is. Surveying the users for quality helps distinguish the good content from the bad one.
Example 2 — Promotion
A few years ago, I was mentally preparing to put myself up for promotion at Google. I was lucky that my best friend since my army days also worked for Google in a more senior position. She knew me, could give me tough love, and had relevant experience. In one of our previous conversations, I asked her to assist me in developing my promotion narrative. What would be my story?
My initial suggestion was:
“I made everything better and faster.”
Her reaction threw me off:
“Chibban, who cares?” (chibban is my childhood nickname)
Me:
“What do you mean who cares? I improved everything!”
I repeated. Her:
“Did we need it to be faster and better?”
Me:
“Isn’t better and faster always good?”
Her:
“Not if we didn’t have a problem with the pace or quality.”
In other words, her point was that not everything needs fixing, and not all problems are of the same worthiness. It’s on me to explain what was broken that required fixing or how making things better and faster helped the business achieve its goals.
Eventually, I realized that making things better and faster enabled my organization to deliver more impact (both in scale and quality) and enabled Googlers to be productive while working from home and returning to the office when needed.
Example 3 — Performance
I was sitting at calibration — this is when Google managers used to discuss and assess employee performance. One of my peers presented their employee’s work. It went something like this:
Him: “This person worked on this product and increased its usage by 10x.”
Hmm… that’s so impressive, I thought.
Me: “what was the user count before and after?”
Him: “10 before and 100 now”.
Me: “and how many users are in your target user group?” meaning, what is the total addressable market, or how many users _can_ use this product?
Him: “150,000.”
Hopefully, you see where I’m going with this. Sure, a 1000% increase in usage is fantastic. But going from 10 to 100 when the target is 150,000 isn’t nearly as impressive.
The shrewd among you might also ask: How long did it take them to get from 10 to 100? It matters if the answer is one month vs twelve months.
You can also ask: what actions did they take to drive the increase in usage? If users just came along, that’s one thing. But if they had to experiment with different techniques that go usage — that’s more meaningful.
Example 4 — Water park
In early January, during my kids’ winter vacation, we sought refuge from New York’s cold winter by booking a room at a Punta Cana hotel renowned for its water park.
One day, as my son fearlessly tackled new slides, he proposed a success metric for our water park adventure: Slides Per Hour (SPH). His critical thinking — at the age of 9 — filled me with pride!
Fast forward to the present, where I read this article to him, as is our habit. As I recounted the first example, he eagerly recalled SPH. Wanting to tailor this concept of holistic understanding of success to his age and knowledge - I introduced a set of qualitative water park metrics:
- Smiles per slide — measuring the frequency of joyous smiles.
- (Happy) screams per slide — quantifying moments of exhilarating joy.
- Cries per slide — assessing instances of fear-induced distress.
Achieving thirty slide runs per hour is seen in a different light when combined with ten cries throughout the event. It underscores the importance of considering emotional fulfillment alongside productivity and efficiency, enriching our understanding of success.
So, don’t let people use numbers to confuse you. And don’t let yourself declare victory too early.
Three principles to follow when describing metrics:
- Benchmark Success: Like the resume example, comparing Griselda’s actions to her peers without context lacks meaning. Understanding the scale of operations and their impact relative to the ecosystem they operated within provides a clearer picture.
- Quality AND Quantity: the questions about the success rate and collateral damage aim to assess quality over quantity. In performance reviews or assessing drug lord operations, the sheer number of achievements (or crimes) doesn’t tell the whole story without understanding the broader implications.
- Narrative and Context: As the promotion narrative and the calibration meeting example illustrate, the significance of achievements is deeply contextual. It’s not just the change that matters, but its relevance, impact, and the problem it solves.
Whether evaluating claims about historical figures, assessing employee performance, or measuring the impact of initiatives, this analytical mindset ensures a more nuanced, comprehensive understanding of success and impact.
Personal perspective
I’ve enjoyed writing on Medium for over a year. Looking at my stats page doesn’t tell me much. Is the 60% read ratio high, average or low? Are 200 views for a story good or bad?
A few months ago, I met a friend who also writes on Medium. I felt comfortable asking her to show me her stats page, and she did. That gave me one additional data point to evaluate my writing performance. However, it’s still hard to know what makes one article more successful than the other.
When someone gives me positive feedback, I always inquire further. “Thank you!” I say, “Can I ask what makes you say that? What did you like/appreciate about it?” non-specific praise is warm and fuzzy, but it doesn’t help understanding what works and what doesn’t. What should I do more, and what should I do less?
I’d love to learn how my writing helps you and what else would you like to know:
- Which topics or themes do you feel are currently underrepresented and need to be further explored in future articles?
- When thinking about data-driven decision-making, what challenges do you face? What do you wish you could measure (better)?
Need help thinking through choosing metrics? Reach out — it’s one of my favourite activities.
