post

Footer About

Footer Get Involved

Footer Links

Mind the Product

General

Product Unplugged Episode 3 with Pierce Healy

Product Unplugged Episode 2 with Lorilyn McCue

Product Strategy

Building Product Teams

Diversity and Inclusion

Digital Transformation

Artificial Intelligence (AI) & Machine Learning (ML)

Product Leadership

Product Management Career

Product Ethics

Minimum Viable Product

Enterprise Product Management

OKRs

Customer Research

Hypothesis and Testing

Business Strategy

User Research

👩‍🚀🚀   Live now for MTP members: <a href="https://mtp2017stg.wpengine.com/membership/#events" style="color:#FFFFFF;text-decoration:underline;">Driving Innovation AMA with Tendayi Viki</a>

Roadmaps: Break the rules without breaking the principles

Got questions about your product career?

Product roadmaps vs data – Jeremy Levy on The Product Experience

From engineering to product management at Google

Mind the Product joins Pendo

A long time data orientated product manager, now data leader. With a wild curiosity, he enjoys finding out how things work and problem solving. Best placed in a strange intersection of data/AI, customer experience and strategy.

Head of Data

Ash McCallum

<p>As a long time Product Manager, it was with a certain trepidation that I accepted a promotion to become Head of Data at tutti.ch. tutti.ch is a free-to-sell-on classifieds platform based in Zürich, Switzerland. The role came with the hopes and expectations to unlock value from the data we collect from the millions of customers a month that come to buy and sell on our platform.</p>
<p>One thing brought up by many before I started was to sort out our Key Performance Indicators (KPIs). We had changed these at least once a year and couldn’t settle on a real structure. There was a strong feeling that other companies were much better in this aspect. After much research, I pieced together bits of advice alongside my own thoughts and I hope this can be a help to you.</p>
<h2>Performance vs Technique</h2>
<p>Consider Usain Bolt. He runs fast and is <span style="text-decoration: underline;"><a href="https://olympics.nbcsports.com/2020/04/21/usain-bolt-beijing-olympics-2008-chicken-nuggets/">fuelled by chicken nuggets</a></span>.</p>
<figure id="attachment_21753" aria-describedby="caption-attachment-21753" style="width: 1000px" class="wp-caption aligncenter"><img loading="lazy" decoding="async" class="size-full wp-image-21753" src="https://mtp2017stg.wpengine.com/wp-content/uploads/2020/10/shutterstock_105290048.jpg" alt="" width="1000" height="690" srcset="/wp-content/uploads/2020/10/shutterstock_105290048.jpg 1000w, /wp-content/uploads/2020/10/shutterstock_105290048-300x207.jpg 300w, /wp-content/uploads/2020/10/shutterstock_105290048-768x530.jpg 768w" sizes="auto, (max-width: 1000px) 100vw, 1000px" /><figcaption id="caption-attachment-21753" class="wp-caption-text">Usain Bolt &#8211; 2009 IAAF World Athletics Finals. Photo by Ververidis Vasilis</figcaption></figure>
<p>He participated in many athletics events but excelled at 100m, 200m and 4x100m relay. This required a huge amount of training and focus but how could he have made sure what he was doing was correct?</p>
<h3>Performance KPIs</h3>
<p>Well, he had his performances, which are the outcomes from all of his training. But even these had a whole bunch of things he could have looked at:</p>
<ul>
<li>Personal Best (PB) time</li>
<li>Season Best (SB) time</li>
<li>Average time from past 3 races</li>
<li>Top 3 finishes</li>
<li>Gold Medals</li>
<li>World Ranking</li>
</ul>
<p>None of these is perfect on their own. PB time only changes if he beats it, it doesn’t show when he is in a slump. Top 3 finishes isn’t so great when you get so used to gold medals that silver seems like a loss. World Ranking is good but what if you want to be a legend of the sport with records that will stand for decades?</p>
<p>Picking the Performance KPIs that are right for you depend on what your goals are and you still most likely need a set of them to get around each other&#8217;s weaknesses. Even so, the smaller the better, 3-5 performance metrics is best for companies to get a clear enough picture in an easy to understand package.</p>
<p>OK, so Usain (and hopefully you) are now clear on what success is; improve those Performance KPIs. But how? What should Usain focus his training on to get better? He and his trainers get together and discuss. Usain thinks he needs more chicken nuggets to give him more fuel for his runs. His trainers aren’t so sure.</p>
<p>One of the trainers suggests they look to understand the biomechanics of running and proposes that the speed you run is governed by a simple formula:</p>
<p><span style="font-style: italic;">Step Length x Step Rate</span></p>
<p>Everyone agrees, but another trainer interrupts “How can we tell if we have hit the best balance between those?” <span style="font-weight: bold;">GOOD QUESTION!</span> They could look at videos from races to get those metrics for each of the other sprinters and see where they lay (aka benchmarking). They can run their own experiments to push Usain further than he would naturally go.</p>
<h3>Technique KPIs</h3>
<p>But these metrics, while informative, still aren’t to the correct level where they are actionable. You can try running with a longer step length than feels natural but chances are that your step rate goes down accordingly. So the team comes up with a plan, they <span style="font-style: italic;">believe</span> that Usain could improve his step length by improving his <span style="font-weight: bold;">technique</span> with great hip flexibility and calf strength and his step rate through better arm swinging and longer quality sleep for recovery. It must be said, they are not positive that these are the cause but they are using their experience to guide them as best as possible and as they increasingly learn they will adapt.</p>
<h2>What about product companies?</h2>
<h3>Performance KPIs</h3>
<p>Do you really think you are all that different from Usain Bolt and his trainers? Ok maybe a bit, but the same principles apply. First, find out what your Performance KPIs are. You need to look for where the key moments of value are for your customers and your business. This should come quite easily as it will be the main actions for your product, here are some ideas:</p>
<h4><span style="font-weight: bold;">Customer Performance Metrics</span></h4>
<ul>
<li>Items sold</li>
<li>Rides taken</li>
<li>Lessons completed</li>
<li>Emails sent</li>
<li>% of ratings given 5 stars</li>
<li>Gross Merchandising Value</li>
</ul>
<h4><span style="font-weight: bold;">Business Performance Metrics</span></h4>
<ul>
<li>Subscriptions started</li>
<li>Subscription churn rate</li>
<li>Advertising revenue</li>
</ul>
<p>For tutti.ch, I did this by mapping out our main customer journeys first and just picking the moments of most value as you can see below (value points in blue).</p>
<figure style="width: 947px" class="wp-caption aligncenter"><img loading="lazy" decoding="async" src="https://mtp2017stg.wpengine.com/wp-content/uploads/2020/10/6102da3e-25af-4546-9484-15a581bc8d8f_tutti.ch20flow20chart202.png" alt="Map of a customer journey" width="947" height="724" /><figcaption class="wp-caption-text">Map of a customer journey</figcaption></figure>
<h3>Technique KPIs</h3>
<p>Once you have these, you need to work out your Technique KPIs for improving them and this is where you need to bring your knowledge and expertise. Discuss with stakeholders of the performance metrics and identify what you believe is needed for success.</p>
<p>For instance, for our buyers customers segment, we believed that we needed:</p>
<ol>
<li>A steady stream of users coming that don’t bounce</li>
<li>Good quality search results</li>
<li>The impulse to contact the seller about the item</li>
<li>Them to think of tutti.ch again soon for another purchase</li>
</ol>
<p>We then paired each of these beliefs of what is important to corresponding Technique KPIs to get:</p>
<ol>
<li>Number of active buyers</li>
<li>% of searches that a top 3 result is clicked on</li>
<li>Contact rate</li>
<li>Buyer churn rate</li>
</ol>
<p>A word of warning: if you have mostly metrics that you will see in a standard web analytics tool (users, sessions, page views etc) then you should really consider if changing them will bring the impact that you desire. These are often called vanity metrics and rightly so as they make no direct difference to the value that you are creating. It’s ok to have one or two to contextualise other numbers but keep them few and far between. Focus on behavioural metrics that are core to your product and how it creates value.</p>
<h2>Recap</h2>
<p>Remember, first establish your moments of value and measure how often they occur or how much value they generate to get your Performance KPIs. Then use the expertise within your company to plan out the techniques that go into improving each of the Performance KPIs as it’s those you take action upon. Next, you need to think about how to present and report your KPIs and use them to generate validated learnings, but you’ll have to wait for the next post for advice on that!</p>
<p>Best of luck and let me know how you get on!</p>
<div class="g g-21"><div class="g-single a-73"><a class="gofollow" data-track="NzMsMjEsMSw2MA==" href="https://community.mindtheproduct.com/c/general/"><img decoding="async" src="https://mtp2017stg.wpengine.com/wp-content/uploads/2021/10/Membership-Circle-banner-ad-scaled.jpg" /></a></div></div>

core/freeform

Data Driven Product Management

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Testing Business Ideas, by David Bland

Developing a Data-Driven Culture at Dow Jones, By Helen Hewitt

When A/B Tests Aren’t Enough, By Scott Castle

Using Data to Build a Brand, by Abba Newberry

Socially Preferable by Nathan Kinch

They’re Just Fancy Averages by Ben Fields

Talking About Data Science by Jenessa Lancaster

Unlocking Data Science to Build the Future of Work by Mike Hyde

Is your Product Underperforming?

ProductTank Columbus

ProductTank Tauranga

ProductTank Edinburgh

Prioritised Member AMA - Perfecting product roadmaps

ProductTank San Diego

Navigating difficult relationships panel

ProductTank Philadelphia

ProductTank Bucharest

ProductTank NoVa

Remote Training Workshops

ProductTank London

ProductTank Birmingham

ProductTank Oslo

ProductTank Astana

ProductTank NoVA

ProductTank Zürich

ProductTank Perth

ProductTank Tokyo

ProductTank Chicago

ProductTank Auckland

ProductTank Richmond

ProductTank Charlotte

ProductTank Reston

ProductTank Washington

ProductTank Sofia

ProductTank Raleigh

ProductTank Ljubljana

ProductTank Kraków

ProductTank Athens

ProductTank Barcelona

ProductTank Dublin

ProductTank Montréal

ProductTank Seattle

ProductTank Madison

ProductTank Singapore

ProductTank Portland

ProductTank Budapest

ProductTank Valencia

ProductTank Porto

ProductTank Los Angeles

ProductTank Austin

ProductTank Norwich

ProductTank Des Moines

ProductTank San Francisco

ProductTank Helsinki

ProductTank Reading

ProductTank Brighton

ProductTank Bielefeld

ProductTank Charleston

ProductTank Bogotá

ProductTank Newcastle Upon Tyne

ProductTank Dallas

ProductTank Malmö

ProductTank Canberra

ProductTank Paris

ProductTank Manchester

#mtpcon SF+Americas 2022 Conference

#mtpEngage Hamburg 2022 Leadership Forum

#mtpEngage Hamburg 2022 Workshops

#mtpEngage Hamburg 2022 Conference

#mtpcon London+EMEA 2022 Conference

Is your Product Underperforming?

Ash McCallum

Read more

Comments