App review: Blinkist

marcabrahamLeave a comment

The main driver for this app review of Blinkist is simple: I heard a fellow product manager talking about it and was intrigued (mostly by the name, I must add).

My quick summary of Blinkist (before using it) – “Big ideas in small packages” is what I read when I Google for Blinkist. I expect an app which provides me with executive type summaries of book and talks, effectively reducing them to bitesize ideas and talking points.

How does Blinkist explain itself in the first minute? – When I go into Apple’s app store and search for Blinkist, I see a strapline which reads “Big ideas from 2,000+ nonfiction books” and “Listen or read in just 15 minutes”. There’s also a mention of “Always learning” which sounds good …

 

 

Getting started, what’s the process like? (1) – I like how Blinkist lets me swipe across a few screens before deciding whether to click on the “Get started” button. The screens use Cal Newport’s “Deep Work” book as an explain to demonstrate the summary Blinkist offers of the book, the 15 minute extract to read or listen to, and how one can highlight relevant bits of the extract. These sample screens give me a much better idea of what Blinkist is about, before I decide whether to sign up or not.

 

 

Getting started, what’s the process like? (2) – I use Facebook account to sign up. After I clicked on “Connect with Facebook” and providing authorisation, I land on this screen which mentions “£59.99 / year*”, followed by a whole lot of small print. Hold on a minute! I’m not sure I want to commit for a whole year, I haven’t used Blinkist’s service yet! Instead, I decide to go for the “Subscribe & try 7 days for free” option at the bottom of the screen.

 

Despite my not wanting to pay for the Blinkist service at this stage, I’m nevertheless being presented with an App Store screen which asks me to confirm payment. No way! I simply get rid  of this screen and land on a – much friendlier – “Discover” screen.

 

 

To start building up my own library I need to go into the “Discover” section and pick a title. However, when I select “Getting Things Done” which is suggested to me in the Discover section, I need to unlock this first by start a free 7-day trial. I don’t want to this at this stage! I just want to get a feel for the content and for what Blinkist has to offer, and how I can best get value out of its service. I decide to not sign up at this stage and leave things here … Instead of letting me build up my library, invest in Blinkist and its content and I only then making me ‘commit’, Blinkist has gone for a free trial and subscription model instead. This is absolutely fine, but doesn’t work for me unfortunately, as I just want to learn more before leaving my email address, committing to payment, etc.

 

 

Did Blinkist deliver on my expectations? – Disappointed.

 

 

 

My product management toolkit (26): PAUSE and LISTEN

marcabrahamLeave a comment

If there are two things I can definitely improve, I’d say it would be my ability to “pause” and “listen”. Too often, I’ve made the mistake of not listening to what the other person is saying. Instead, I’m thinking of what to say myself or making assumptions, completely ‘steamrolling’ the other person in the process …

 

Looking back, I guess my inability to listen was closely linked to my focus on products over people. This focus effectively meant that I cared more about products, and less about building relationships with people. For example, I felt at times that internal stakeholders were more of a necessary evil whose sole purpose was to hinder product development. Fortunately, I no longer adhere to this point of view and I’ve come to realise how critical collaboration is to building great products.

I had to think back to this transition when I recently read “Practical Empathy”, which Indi Young, an independent UX consultant, published in 2015. In this great book, Young explains that empathy is all about understanding what’s going on in other people’s minds. She describes “listening” as a vital tool to create a deeper understanding and talks about “a new way to listen”:

  • Listen for reasoning (inner thinking) – What is going through someone else’s mind?
  • Listen for reactions – Reactions often go hand in hand with reasoning. For instance, I could express an emotional response when I describe why I decided to make a career change.
  • Listen for guiding principles – A guiding principle is a philosophy or belief that someone uses to decide what action to take, what to choose, how to act, etc.

I found that for people like me – i.e. with lots of opinions, thoughts, ideas and assumptions – listening can be incredibly difficult. In the Netherlands where I was born, voicing your opinion and standing up for oneself are considered highly regarded attributes. I’ve had to learn – and am still learning – to “pause” a lot more. Instead of jumping to conclusions or simply getting my two cents in, I’ve learned to breathe and pause first before deciding to say something or to simply listen. I’m trying to remind myself continuously that each I forget to listen, I forfeit the opportunity to understand what’s driving the other person.

Let me share a real life example with you, to illustrate how listening can help to develop a better understanding of where the other person is coming from:

Example

A while ago, I was talking about team performance with an engineer and he mentioned that “we have to be careful, because most engineers are delicate beings”.

Me – before understanding a single thing about listening and empathy

With a comment like the engineer had made, I’d have jumped straight in there and would have said: “what are you talking about! Surely, not all developers are delicate human beings! I’ve worked with some developers who made me look like a wallflower!” and so on and so forth.

Me – with the beginnings of an understanding about listening and empathy

In this real life example, I didn’t respond at all. I paused and listened. By allowing the engineer speak, I started to understand that he cared deeply about the developers in the team and considered himself as their mentor, wanting to make sure they fully enjoyed their day job.

The moral of this story is that listening starts with taking that split second to stop that innate desire to respond immediately. I learned a lot about listening from reading and practising the insights from “Active Listening”, another valuable book. In this book by Josh Gibson and Fynn Walker, they describe the four components of active listening:

  1. Acceptance – Acceptance is about respecting the person that you’re talking to; irrespective of what the other person has to say but purely because you’re talking to another human being. Accepting means trying to avoid expressing agreement or disagreement with what the other person is saying, at least initially. I’ve often made this mistake; being too keen to express my views and thus encouraging the speaker to take a very defensive stance in the conversation.
  2. Honesty – Honesty comes down to being open about your reactions to what you’ve heard. Similar to the acceptance component, honest reactions given too soon can easily stifle further explanation on the part of the speaker.
  3. Empathy – Empathy is about your ability to understand the speaker’s situation on an emotional level, based on your own view. Basing your understanding on your own view instead of on a sense of what should be felt, creates empathy instead of sympathy. Empathy can also be defined as your desire to feel the speaker’s emotions, regardless of your own experience.
  4. Specifics – Specifics refers to the need to deal in details rather than generalities. The point here is that for communication to be worthwhile, you should ask the speaker to be more specific, encouraging the speaker to open up more or “own” the problem that they’re trying to raise.

The thing with empathy, as Young points out in her book, is that it isn’t about having to feel warmth for the other person or fully agree with him / her. In contrast, empathy means understanding and comprehending the other person. It takes time and skill to be able to drop in to a neutral frame of mind to:

  • Resist the urge to demonstrate how smart you are – Well known technology exec and investor Bill Krause used to write down “DNT” in his notepad during meetings. “DNT” stood for “Do Not Talk”, and he used it as a practical tool to stop himself from saying something or showing how smart he was. Young also provides some good tips on how to stop yourself from talking or saying too much (see Fig. 2 below).
  • Develop the knowledge to gain empathy – In “Practical Empathy”, Young suggests a few simple questions that can help you build a better understanding of the other person’s deeper reasoning and principles. Readers are encouraged to use the fewest number of words possible when asking question (see Fig. 1 below).
  • Apply empathy to both customers and colleagues – UX expert Erika Hall recently made a great point about the importance of applying empathy to co-workers (listen to the Aurelius podcast here for the interview with Erika). Again, I used to make the mistake of solely applying customer empathy but not paying enough attention or respect to colleagues. Showing empathy towards can be as simple as understanding about someone’s else workload or OKRs.

Main learning point: Empathy – both inward and outward – is SO SO important. Pausing and listening are your first tools on the path towards developing empathy. Yes, I look back on mistakes made and people that I’ve upset in the past due to a lack of empathy, but I feel that I’ve learned a lot since then (whilst I appreciate I still have got a long way to go). If you want to get started on developing and showing more empathy, I’d highly recommend reading “Practical Empathy” by Indi Young and “Active Listening” by Josh Gibson and Fynn Walker.

 

Fig. 1 – Use the fewest number of words possible – Taken from: Indi Young – Practical Empathy, p. 60

  • “Why’s that?”
  • “What were you thinking?”
  • “What’s your reasoning?”
  • Tell me more about <her phrase>.”
  • “Because?”

Fig. 2 – Summary of “a new way to listen” – Taken from: Indi Young – Practical Empathy, p. 77

What to listen for:

Reasoning: Thinking, decision-making, motivations, thought processes, rationalisation.

Reaction: Responses to something – mostly emotional, some behavioural.

Guiding Principle: Belief that guides decisions.

Follow the peaks and valleys:

  • Started with a broad topic
  • Let the speaker keep choosing the direction
  • Dig into the last few remarks
  • Use the fewest number of words possible
  • Reiterate a topic to show attention, verify your understanding and ask for more
  • Avoid introducing words the speaker hasn’t used
  • Try not to say “I”

Be supportive:

  • Don’t fake it – react, be present
  • Never switch abruptly
  • Adapt yourself to the mood
  • Don’t cause doubt or worry

Be respectful:

  • Be the undermind, not the overmind
  • Resist the urge to demonstrate how smart you are
  • Avoid implying or telling the speaker she is wrong

Neutralise your reactions:

  • Learn how to notice your emotional reactions
  • Dissipate your reactions and judgments

 

 

App review: Steemit

marcabrahamLeave a comment

Steemit.com is one of those products that feels super complex at first sight. I think it’s content platform but I need to give it a much closer look in order to understand how Steemit works:

My quick summary of Steemit (before using it): I reckon Steemit is a content creation and sharing platform, but I’m not sure what technology it’s built on or how it works.

How does the app explain itself in the first minute? “Your voice is worth something” is the first thing I see. When I continue reading above the fold, it says “Get paid for good content. Post and upvote articles on Steemit to get your share of the daily rewards pool.”

Getting started, what’s the process like (1)? The first thing I do is clicking on the “Learn more” button on the Steemit homepage. I then land on a useful FAQ page which covers the typical questions and answers you’d expect. Steemit enables “the crowd to reward the crowd for their content.” The platform is connected with the Steem blockchain, which is decentralised and open. Content contributors to Steemit are rewarded with STEEM, dependent on the attention their content is getting from other Steemit users.

Getting started, what’s the process like (2)? Signing up is very straightforward, nothing out of the ordinary. A nice progress bar, two-factor authentication and I now have to wait for Steemit to validate my sign-up request.

What can I do in the meantime? – I have a little nose around the Steemit platform, to learn about the content people publish. For example, I came across Dan Dicks, who has posted 71 posted on Steemed and has (sofar) received $123.86 for his latest post.

 

Main learning point: Steemit feels very similar to Quora and Reddit, but the main difference being the underlying blockchain and cryptocurrency element. Once my signup request has been approved, I’ll no doubt get a better sense of how the platform actually works. Currently, I’m not entirely clear on the dynamics in terms of being rewarded for your Steemit content.

Related links for further learning:

  1. https://steemit.com/steemit/@mindover/steemit-for-dummies-like-me-everything-you-need-know-to-get-started
  2. https://steemit.com/faq.html
  3. https://steemit.com/exploring/@kebin/what-is-steem-and-what-is-sbd
  4. https://steem.io/SteemWhitePaper.pdf

My product management toolkit (25): understanding the “unit economics” of your product

marcabrahamLeave a comment

As a product manager it’s important to understand the unit economics of your product, irrespective of whether you’re managing a physical or a digital product. Unit economics are the direct revenues and costs related to a specific business model expressed on a per unit basis. These revenues and costs are the levers that impact the overall financial success of a product. In my view there are a number of reasons why I feel it’s important for product managers to have a good grasp of the unit economics of your product:

  • Helps quantify the value of what we do – Ultimately, product success can be measured in hard metrics such as revenue and profit. Even in cases where our products don’t directly attribute to revenue, they will at least have an impact on operational cost.
  • Customer Value = Business Value – In an ideal world, there’s a perfect equilibrium between customer value and business value. If the customer is happy with your product, buys and uses it, this should result in tangible business value.
  • P&L accountability for product people (1) – Perhaps it’s to do with the fact that product management still is a relatively young discipline, but I’m nevertheless surprised by the limited number of pr0duct people I know who’ve got full P&L responsibility. I believe that having ownership over the profit & loss account helps product decision making and and accountability, not just for product managers but for the product teams that we’re part of.
  • P&L accountability for product people (2) – Understandably, this can be a scary prospect and might impact the ways in which we manage products. However, owning the P&L will (1) make product managers fully accountable for product performance (2) provide clarity and accountability for product decisions, (3) help investments in the product and product marketing and (4) steep product management in data, moving to a more data informed approach to product management.
  • Assessing opportunities based on economics – Let’s move away from assessing new business or product opportunities purely based on “gut feel”. I appreciate that at some point we have to take a leap, especially with new products or problems that haven’t been solved before. At the same time, I do believe it’s critical to use data to help inform your opportunity assessments. Tools like Ash Maurya’s Lean Canvas help to think through and communicate the economics of certain opportunities (see Fig. 1 below). In the “cost structure” part of the lean canvas, for example, you can outline the expected acquisition or distribution cost of a new product.
  • Speaking the same language – It definitely helps the collaboration with stakeholders, the board and investors if you can speak about the unit economics of your product. I know from experience that being able to talk sensibly about unit economics and gross profit, really helps the conversation.

Now that we’ve established the importance of understanding unit economics, let’s look at some of the key components of unit economics  in more detail:

Profit margin per unit = (sales price) – (cost of goods sold + manufacture cost + packaging cost + postage cost + sales cost)

Naturally the exact cost per unit will be dependent on things such as (1) product type (2) point of sale (3) delivery fees and (4) any other ‘cost inputs’.

In a digital context, the user is often the unit. For example, the Lifetime Value (‘LTV’) and Customer Acquisition Cost (‘CAC’) are core metrics for most direct to consumer (B2C) digital products and services. I learned from David Skok and Dave Kellogg about the importance of the ‘CAC to LTV’ ratio.

Granted, Skok and Kellogg apply this ratio to SaaS, but I believe customer acquisition cost (‘CAC’) and customer lifetime value (‘LTV’) are core metrics when you treat the user as a unit; you’ve got a sustainable business model if LTV (significantly) exceeds CAC. In an ideal world, for every £1 it costs to acquire a customer you want to get £3 back in terms of customer lifetime value. Consequently, the LTV:CAC ratio = 3:1.

I’ve seen companies start with high CAC in order to build scale and then lower the CAC as the business matures and relies more on word of mouth as well as higher LTV. Also, companies like Salesforce are well known for carefully designing additions (“editions”) to increase customer lifetime value (see Fig. 2 below). 

Netflix are another good example in this respect, with their long term LTV view of their customers. Netflix take into account the Netflix subscription model and a viable replacement for another subscription model in cable. The average LTV of Netflix customers is 25 months. As a result, Netflix are happy to initially ‘lose’ money on acquiring customers, through a 1-month free trial, as these costs costs will be recouped very soon after acquiring the customer.

Main learning point: You don’t need to be a financial expert to understand the unit economics of your products. Just knowing what the ‘levers’ are that impact your product, will put you in good stead when it comes to making product decisions and collaborating with stakeholders.

 

 Fig. 1 – Lean Canvas template by Ash Maurya – Taken from: https://blog.leanstack.com/

 

Fig. 2 – Pricing and functionality overview for Salesforce’s New Sales Cloud Lightning Editions:

 

Related links for further learning:

  1. https://soundcloud.com/saastr/saastr-142-why-cac-ltv-is-the
  2. https://inpdcenter.com/blog/understanding-product-economics-improve-product-development-success/
  3. https://people.kth.se/~msmith/ii2300_pdf/product_realization_7_2016.pdf
  4. https://www.quora.com/What-are-unit-economics
  5. https://youtu.be/RG_eyn0fRXs
  6. https://medium.com/@markroberge
  7. https://www.slideshare.net/RaviLakkundi/product-management-pricing-31102059
  8. https://www.inc.com/guides/price-your-products.html
  9. http://accountingexplained.com/managerial/cvp-analysis/cost-plus-pricing
  10. https://www.quora.com/What-are-unit-economics
  11. http://www.forentrepreneurs.com/saas-metrics-2-definitions-2/
  12. http://www.problemio.com/business/business_economics.php
  13. https://www.slideshare.net/austinneudecker/startup-unit-economics-and-financial-model
  14. https://www.linkedin.com/pulse/understanding-saas-business-model-unit-economics-ben-cotton/
  15. https://thepathforward.io/how-to-estimate-your-unit-economics-before-you-have-any-customers/
  16. https://thepathforward.io/unit-economics-by-sam-altman/
  17. http://launchingtechventures.blogspot.co.uk/2014/04/e-commerce-metrics.html
  18. https://medium.com/@parthgohil/understanding-unit-economics-of-e-commerce-9c77042a2874
  19. https://yourstory.com/2017/02/unit-economics-flipkart/
  20. https://www.entrepreneur.com/article/283878
  21. https://hbr.org/2016/08/a-quick-guide-to-value-based-pricing
  22. https://unicornomy.com/netflix-business-strategy-netflix-unit-economics/
  23. https://hbr.org/2017/04/what-most-companies-miss-about-customer-lifetime-value

Book review: “Zero to One”

marcabrahamLeave a comment

Whatever you think of Peter Thiel, he’s got a lot of ‘street cred’ in the world of technology and venture capital. We all know how he founded PayPal and turned it into a billion dollar company. As a tech investor, Thiel is widely known for being an early investor in the likes of Facebook and LinkedIn. Listening to a recent interview between Thiel and Reid Hoffman on the latter’s podcast inspired me to read Thiel’s “Zero to One: Notes on Startups or How To Build the Future”. Thiel published “Zero to One” in 2014, based on a course about startups that he taught at Stanford previously.

Truth be told, some of Thiel’s views and concepts in “Zero to One” didn’t resonate with me, mostly because I struggled to convert them into action points I can apply to my own situation (read: working at a successful but early stage startup, and being based in London and not in Silicon Valley). Perhaps that’s exactly the point of Thiel’s book; to provide readers with a wide range of views, some more visionary and though provoking than others, and leaving it with readers to ‘pick and mix’ as they see fit. Consequently, these are the main learning points that I took away from reading “Zero to One”:

  1. Forget about being the first mover, be the last mover instead (1) – In strategy terms, people often talk about the benefits of being a “first mover”; a company’s ability to have a head start over its competitors as a result of being first to market in a new product category. The Hoover vacuum cleaner or Apple’s iPad are good examples of products which opened up a whole new product category and therefore did enjoy (durable) first mover advantage. Thiel, however, flips this by arguing the benefits of being a last mover.
  2. Forget about being the first mover, be the last mover instead (2) – Thiel argues that “moving first is a tactic, not a goal.” He stresses that the point of any business is to generate future cash flows, so being the first mover doesn’t do you any good if someone else comes along and unseats you. Video streaming app Meerkat is a good example of a product which was first to market, but got quickly overtaken by late(r) entrants Periscope and Facebook Live. Thiel explains “It’s much better to be the last mover – that is, to make the last great development in  a specific market and enjoy years or even decades of monopoly profits.” He advises that in order to get to such a position, companies need to dominate a small niche and scale up from there, constantly moving toward their long-term vision.
  3. The value of long term planning – I really like Thiel’s point about “lean” being a methodology, not a goal in it’s own right. As much as I see the value and importance of learning early and often, I do agree with Thiel’s opinion s about the pointlessness of iterating just for the sake of it. What’s the point of a Minimum Viable Product if you aren’t going to learn from it and iterate accordingly? What’s the value of just releasing ‘stuff’ without reflecting on whether a release got you a step closer to achieving your overall vision and commercial success? Thiel describes how successful companies like Apple and Facebook used long-term planning and business planning to get a position of durable market success.
  4. What to do with the “Power Law”? (1) – Thiel gives readers a good insight into the workings of venture capital (‘VC’) companies when he discusses the “power law”. The power law is based on the Pareto Principle. You might have come across this principle in the form of the 80/20 rule; explaining the unequal relationship between inputs and outputs, with 20% of invested input being responsible for 80% of results obtained. Thiel explains that this uneven pattern exists just as much in the VC world: “The biggest secret in venture capital is that the best investment in a successful fund equals or outperforms the entire rest of of the fund combined.” To optimise for the power law, Thiel recommends focusing on one market, one distribution strategy and, as a consequence, to be cautious about diversification.
  5. What to do with the “Power Law”? (2) – For me, the most valuable bit of “Zero to One” is the part where Thiel covers how to best use the power law when making critical business and product decisions. Going over his questions, I learned the importance of being pretty single minded about your unique proposition and execution (see Fig. 1 below). Thiel’s thinking about these questions is pretty simple: “Whatever your industry, any great business plan must address each every one of them. If you don’t have good answers to these questions, you’ll run into lots of “bad luck” and your business will fail. If you nail all seven you’ll master fortune and succeed.”

Fig. 1 – “Seven questions that every business must answer” – Taken from: Peter Thiel, “Zero to One”, pp. 153-154

  1. The Engineering Question – Can you create breakthrough technology instead of incremental improvements?
  2. The Timing Question – Is now the right time to start your particular business?
  3. The Monopoly Question – Are you starting with a big share of a small market?
  4. The People Question – Do you have the right team?
  5. The Distribution Question – Do you have a way to not just create but deliver your product?
  6. The Durability Question – Will your market position be defensible 10 and 20 years into the future?
  7. The Secret Question – Have you identified a unique opportunity that others don’t see?

Main learning point: You can tell that “Zero to One” is written by someone who’s ‘been there and done that’. Thiel speaks with authority about the need to focus on a single market and busts commonplace myths about ‘lean’, first mover advantage and diversification.

Related links for further learning:

  1. http://www.telegraph.co.uk/technology/11098971/Peter-Thiel-the-billionaire-tech-entrepreneur-on-a-mission-to-cheat-death.html
  2. https://www.theguardian.com/technology/2016/jul/21/peter-thiel-republican-convention-speech
  3. https://art19.com/shows/masters-of-scale/episodes/09f191df-d089-49a3-876d-75c7730a3f94
  4. http://www.reidhoffman.org/
  5. http://zerotoonebook.com/
  6. https://hbr.org/2005/04/the-half-truth-of-first-mover-advantage
  7. https://marketing-insider.eu/categories-of-new-products/
  8. https://medium.com/@RevelX/first-mover-disadvantage-9-reasons-being-the-first-to-market-may-harm-your-business-9ec75a85b1d2
  9. https://www.forbes.com/sites/ralphbenko/2014/10/13/peter-thiel-we-dont-live-in-a-normal-world-we-live-under-a-power-law/#35b4d7fc7a7d
  10. https://en.wikipedia.org/wiki/Pareto_principle

 

Book review: “Designing with Data”

marcabrahamLeave a comment

I’d been looking forward to Rochelle King writing her book about using data to inform designs (I wrote about using data to inform product decisions a few years ago, which post followed a great conversation with Rochelle).

Earlier this year, Rochelle published “Designing with Data: Improving the User Experience with A/B Testing”, together with Elizabeth F. Churchill and Caitlin Tan. The main theme of “Designing with Data” the book is the authors’ belief that data capture, management, and analysis is the best way to bridge between design, user experience, and business relevance:

  1. Data aware — In the book, King, Churchill and Tan distinguish between three different ways to think about data: data driven; data informed and data aware (see Fig. 1 below). The third way listed, being ‘data aware’, is introduced by the authors: “In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions.” If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.
  2. How much data to collect? — The authors make an important distinction between “small sample research” and “large sample research”. Small sample research tends to be good for identifying usability problems, because “you don’t need to quantify exactly how many in the population will share that confusion to know it’s a problem with your design.” It reminded me of Jakob Nielsen’s point about how the best results come from testing with no more than 5 five people. In contrast, collecting data from a large group of participants, i.e. large sample research, can give you more precise quantity and frequency information: how many people people feel a certain way, what percentage of users will take this action, etc. A/B tests are one way of collecting data at scale, with the data being “statistically significant” and not just anecdotal. Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance.
  3. Running A/B tests: online experiments — The book does a great job of explaining what is required to successfully running A/B tests online, providing tips on how to sample users online and key metrics to measure (Fig. 2) .
  4. Minimum Detectable Effect — There’s an important distinction between statistical significance — which measure whether there’s a difference — and “effect”, which quantifies how big that difference is. The book explains about determining “Minimum Detectable Effect” when planning online A/B tests. The Minimum Detectable Effect is the minimum effect we want to observe between our test condition and control condition in order to call the A/B test a success. It can be positive or negative but you want to see a clear difference in order to be able to call the test a success or a failure.
  5. Know what you need to learn — The book covers hypotheses as an important way to figure out what it is that you want to learn through the A/B test, and to identify what success will look like. In addition, you can look at learnings beyond the outcomes of your A/B test (see Fig. 3 below).
  6. Experimentation framework — For me, the most useful section of the book was Chapter 3, in which the authors introduce an experimentation framework that helps planning your A/B test in a more structured fashion (see Fig. 4 below). They describe three main phases — Definition, Execution and Analysis — which feed into the experimentation framework. The ‘Definition’ phase covers the definition of a goal, articulation of a problem / opportunity and the drafting of a testable hypothesis. The ‘Execution’ phase is all about designing and building the A/B test, “designing to learn” in other words. In the final ‘Analysis’ phase you’re getting answers from your experiments. These results can be either “positive” and expected or “negative” and unexpected (see Fig. 5–6 below).

Main learning point: “Designing with Data” made me realise again how much thinking and designing needs to happen before running a successful online A/B test. “Successful” in this context means achieving clear learning outcomes. The book provides a comprehensive overview of the key considerations to take into account in order to optimise your learning.

Fig. 1 — Three ways to think about data — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 3–9

  • Data driven — With a purely data driven approach, it’s data that determine the fate of a product; based solely on data outcomes businesses can optimise continuously for the biggest impact on their key metric. You can be data driven if you’ve done the work of knowing exactly what your goal is, and you have a very precise and unambiguous question that you want to understand.
  • Data informed — With a data informed approach, you weigh up data alongside a variety of other variables such as strategic considerations, user experience, intuition, resources, regulation and competition. So adopting a data-informed perspective means that you may not be as targeted and directed in what you’re trying to understand. Instead, what you’re trying to do is inform the way you think about the problem and the problem space.
  • Data aware — In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions. If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.

Fig. 2 — Generating a representative sample — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 45–53

  • Cohorts and segments — A cohort is a group of users who have a shared experience. Alternatively, you can also segment your user base into different groups based on more stable characteristics such as demographic factors (e.g. gender, age, country of residence) or you may want them by their behaviour (e.g. new user, power user).
  • New users versus existing users — Data can help you learn more about both your existing understand prospective future users, and determining whether you want to sample from new or existing users is an important consideration in A/B testing. Existing users are people who have prior experience with your product or service. Because of this, they come into the experience with a preconceived notion of how your product or service works. Thus, it’s important to be careful about whether your test is with new or existing users, as these learned habits and behaviours about how your product used to be in the past can bias in your A/B test.

Fig. 3 — Know what you want to learn — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, p. 67

  • If you fail, what did you learn that you will apply to future designs?
  • If you succeed, what did you learn that you will apply to future designs?
  • How much work are you willing to put into your testing in order to get this learning?

Fig. 4 — Experimentation framework — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 83–85

  1. Goal — First you define the goal that you want to achieve; usually this is something that is directly tied to the success of your business. Note that you might also articulate this goal as an ideal user experience that you want to provide. This is often the case that you believe that delivering that ideal experience will ultimately lead to business success.
  2. Problem/opportunity area — You’ll then identify an area of focus for achieving that goal, either by addressing a problem that you want to solve for your users or by finding an opportunity area to offer your users something that didn’t exist before or is a new way of satisfying their needs.
  3. Hypothesis — After that, you’ll create a hypothesis statement which is a structured way of describing the belief about your users and product that you want to test. You may pursue one hypothesis or many concurrently.
  4. Test — Next, you’ll create your test by designing the actual experience that represents your idea. You’ll run your test by launching the experience to a subset of your users.
  5. Results — Finally, you’ll end by getting the reaction to your test from your users and doing analysis on the results that you get. You’ll take these results and make decisions about what to do next.

Fig. 5 — Expected (“positive”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 227–228

  • How large of an effect will your changes have on users? Will this new experience require any new training or support? Will the new experience slow down the workflow for anyone who has become accustomed to how your current experience is?
  • How much work will it take to maintain?
  • Did you take any “shortcuts” in the process of running the test that you need to go back and address before your roll it out to a larger audience (e.g. edge cases or fine-tuning details)?
  • Are you planning on doing additional testing and if so, what is the time frame you’ve established for that? If you have other large changes that are planned for the future, then you may not want to roll your first positive test out to users right away.

Fig. 6 — Unexpected and undesirable (“negative”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 228–231

  • Are they using the feature the way you think they do?
  • Do they care about different things than you think they do?
  • Are you focusing on something that only appeals to a small segment of the base but not the majority?

Related links for further learning:

  1. https://www.ted.com/watch/ted-institute/ted-bcg/rochelle-king-the-complex-relationship-between-data-and-design-in-ux
  2. http://andrewchen.co/know-the-difference-between-data-informed-and-versus-data-driven/
  3. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  4. https://vwo.com/ab-split-test-significance-calculator/
  5. https://www.kissmetrics.com/growth-tools/ab-significance-test/
  6. https://select-statistics.co.uk/blog/importance-effect-sample-size/
  7. https://www.optimizely.com/optimization-glossary/statistical-significance/
  8. https://medium.com/airbnb-engineering/experiment-reporting-framework-4e3fcd29e6c0
  9. https://medium.com/@Pinterest_Engineering/building-pinterests-a-b-testing-platform-ab4934ace9f4
  10. https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166

 

Design with Data.jpg

 

 

 

 

Book review: “Product Leadership”

marcabraham1 Comment

A few months ago, Richard Banfield, Martin Eriksson and Nate Walkingshaw published “Product Leadership”, a great book which shares stories and insights from top product managers who’ve built world class products and highly performant product teams. Banfield, Eriksson and Walkingshaw are all well established product management experts, and as part of writing “Product Leadership” they interviewed some of the best product managers in the field about their ways of working.

The authors distinguish between product management and product leadership, stressing the need to focus more on product leadership. This book isn’t about just managing a product team; as the book states, “A product leader is ultimately responsible for the success and failure of a product, and by extension, the company itself.” This raises the question about what makes a true product leader:

  1. “Product is people” – The book makes a great point about the importance of people within a product management context. More and more organisations are moving away from a traditional top-down approach, whereby senior management determines ‘upstream’ what people ‘downstream’ should be doing. Companies like Spotify have been very successful in creating autonomous and cross-functional teams. The success of this approach doesn’t only come from the cross-functional and collaborative nature of Spotify teams; people within these teams are also very close to the customer. When Banfield, Eriksson and Walkingshaw talk about ‘people’, they talk about people both within and outside of a product organisation.
  2. Focus on customer problems – I love how the book stresses the importance of focusing on customer problems. I liked the comment in the book from User Interface Engineering’s Jared Spool. He suggests having themes on the roadmap instead of features: “Themes are a promise to solve a problems not to build features.” Once you’ve truly understood customer problems, you can start prioritising. Banfield, Eriksson and Walkingshaw suggest that prioritisation is best done through the lens of the core product management principles: first, is it valuable; second, is it usable and third, is it feasible?
  3. Creating a successful product team – The book covers the ingredients of successful product teams (see Fig. 1 below). The authors point out that none of the people they interviewed mentioned hard skills, such as engineering, design or even specific product management expertise. In contrast, with product managers, the focus is much more on soft skills. One of my favourite product people, Drift CEO and cofounder David Cancel, believes that the traits of a successful product leader will always be weighted towards softer skills.”What we’re looking for are things that fit more on the qualitative side, which people hate because it’s so squishy”, Cancel explains.
  4. How to identify product leaders – The book rightly makes the point that hiring product leaders is hard. Experienced product people with strong soft skills are low in supply in most markets. One can derive the characteristics of good product leaders from the high level traits of a successful team (see previous point), and the book contains some helpful characteristics of good product leaders (see Fig. 2).
  5. Aligning team members – I’ve heard some product leaders talking about “treating your product team as a product”, and the book talks along similar lines about successful product teams. One key aspect in this respect is a product leader’s ability to align team members around a shared goal or vision.

Main learning point: “Product Leadership” is a great book for anyone interested in making the leap from being a product manager to a product leader. The book isn’t prescriptive in what makes a true product leader or how to become one; the authors have interviewed a wide range of global product leaders who share a wide range of experiences and perspectives. Because of its real life examples and lessons learned, “Product Leadership” isn’t your average ‘how to’ book. Instead, it helps readers reflect on what what product management is and what excellence in this field looks like.

 

Fig. 1 – Common characteristics of successful product teams – Taken from: Richard Banfield, Martin Eriksson and Nate Walkingshaw. Product Leadership, pp. 74-76

  • Lifelong learning – Actively seeking new information, insights and understanding is a cornerstone to successful product management. Connected to this is an open-mindedness and coachability in all respected leaders; they don’t assume they know all the answers and constantly refactor their thinking to take in new knowledge and feedback.
  • Strong communication – This is a big bucket that includes several soft skills like listening and presenting, but communication is fairly self-explanatory.
  • Empathy – Deeply connected to communication is empathy – not only empathy for team members inside and outside the organisation, but for customers as well.
  • Diversity – Strong leaders know that a diversity of backgrounds, experiences, and demographics provides the range of perspectives needed to build a complete product experience.
  • Business savvy – This characteristic is related to how product leaders understand their role in the value delivery process and their grasp of the broader business context.
  • Cross-functional representation – Teams without representation see the world with strong biases. The more a product leader can bring the functional areas of a product together, the more likely the team will act in a coherent manner.
  • Collocation – Working side by side with your team makes communication easier and faster.
  • Autonomy – The best teams have the ability to solve problems on their own without having to run everything up the ladder. Decision-making skills and the authority to implement those decisions are critical to a team’s speed and effectiveness.
  • Interdependence – This might seem almost contradictory to the autonomous point, but successful teams don’t wall themselves off from the rest of the company. Autonomy is a reference to the ability to make decisions, not a reflection on their avoidance of others.
  • Accountability – The best teams place guardrails in place that allow them to see clearly when either the qualitative research isn’t paying back or the product releases are not hitting the desired outcomes.

Fig. 2 – How to identify product leaders – Taken from: Richard Banfield, Martin Eriksson and Nate Walkingshaw. Product Leadership, pp. 76-82

  • Plays well with others – Product management and product leadership are people-first roles. Teams are made up of individuals who each bring their own personalities, perspective, and opinions. Understanding what makes people tick makes it easier to get the best out of them and identify when there is a problem. Being an empathic product leader doesn’t mean you have to be all things to all people but you’re able to engage emphatically with others, even if they don’t have a lot in common with them.
  • Always acts and thinks “team first” – The kind of leader that needs constant recognition and praise is not likely to be the best person for this job. The top product leaders hardly ever get recognition for their hard work and dedication. Furthermore, they tend to turn any limelight they receive back on their teams.
  • Is comfortable wearing lots of hats – Product leaders are typically comfortably wearing the hats of marketer, manager, technologist, customer advocate, and facilitator, to name a few. This doesn’t mean they are experts at each of these roles; rather, they are comfortable working across all areas as the role evolves or as the product demands.
  • Displays curiosity – This trait relates to several others mentioned here, but needs to be addressed specifically. Having a deep interest in learning, enthusiastically accepting new challenges, encouraging others to speak up, and listening to their perspectives are all actions driven by curiosity.
  • Communicates well – This might seem so obvious it’s not worth mentioning, but it’s surprising how many senior managers and leaders are poor communicators. Having the ability to share information in a clear, concise manner is a necessary and desirable skill for a product leader. Whether they’re talking, listening, writing, planing, sharing, or facilitating, the goal is always to be understood.
  • Possesses selling skills – This might be controversial, but when you consider what product leaders do each day, it’s no surprise that selling is one of the things to look out for in product leaders. To be clear, this is not the type of selling associated with business development; rather, it’s not the type that works to change minds and get buy-in from others.
  • Has exceptional time management skills – Shipping product is a race against time. Managing that race is a series of little decisions that collectively make up the product roadmap.
  • Shows equanimity / grace under fire – Being equanimous is tightly connected to the “plays well with others” trait mentioned above. “We practice equanimity – the idea of taking all this emotional anxiety and trying to structure it, understand it, and channel it into productive output,” says Jeff Veen, Design Partner at True Ventures. Who is on the team matters much less than how the team interacts, and the number one dynamic that defined successful teams was a sense of psychological safety – a sense that the team would not embarrass, reject, or punish someone for speaking up. Being able to instil this sense of psychological safety and trust is a key part of the product leader’s job.