My product management toolkit (33): launch and learn

“Build it and they will come!” I used to work once with a senior executive, who was of the opinion that a product or feature should just be launched, without any testing with customers beforehand. “I know that once it’s out there, people will want it” she’d explain to me, adding that “it’s what people want”.

 

 

Hearing this “build it and they will come” mantra time and time again did annoy me 🙂 At the same time, it did make me wonder whether it might be a good idea to (continuously) release product features without prior customer discovery … What if this executive is right and any new product, feature or service should just be launched, as a way of learning as quickly as possible!?

Being able to ‘launch and learn’ is a vital tool in any product person’s toolkit. I strongly encourage you to avoid ‘one-off product releases’ at any time; what are you going to learn from shipping a product only to then move on to the next thing!? One can debate about when to best learn – should you learn pre-release? – but the main point is that you’ll need to ship early and often to learn continuously.

Basecamp, a project management software compare, does take ‘launch and learn’ to the extreme, they don’t show customers anything until every customer can see it. In the book “It doesn’t have to be crazy at work”, Basecamp’s co-founders Jason Fried and David Heinemeier Hansson describe how at Basecamp:

  • “We don’t beta-test with customers.”
  • “We don’t ask people what they’d pay for something.”
  • “We do the best job we know how to do and then we launch it into the market.”
  • “The market will tell us the truth.”

Fried and Heinemeier Hansson argue that anything you ask or test with customers prior to launch is hypothetical: “Real answers are uncovered when someone’s motivated enough to buy your product and use it in their own environment – and of their own volition. Anything else is simulated answers. Shipping real products gives you real answers.” Whilst I do agree with this line of thinking, I don’t believe in simply launching some crappy product or feature and see if it sticks (just as much I don’t believe in “build it and they will come”).

 

 

My suggestion would to ‘launch confidently and learn’. This means that for each new product or feature you determine – based on your confidence level – whether it needs some form of customer research before launch:

  1. Deliver value in order to learn – You want to be smart about the things you want to learn. The best opportunity to learn comes when you’re confident about the value that you’re delivering to the customer. Naturally, people might not buy or use your product despite the value it intends to deliver, but that’s a learning in itself.
  2. Minimum Level of Confidence (1) – How confident are you? What exactly are you confident about (and why)? The main reason why I believe in product managers adhering to a confidence treshold is to avoid launching products that don’t work or provide an awful user experience. The Newton MessagePad which came out in 1993 is a good example of the launch of an incomplete product, which didn’t live up to its promise. Larry Tesler, senior exec at Apple at the time of of the Newton MessagePad, described Apple’s promise about the Newton’s handwriting capability as a large nail in the Newton coffin. The lesson learned here is that you shouldn’t launch when you’re not confident about the capability and value of your product or feature.
  3. Minimum Level of Confidence (2) – I’ve come up with a number of basic questions and criteria to apply when you’re thinking of launching a product (see Fig. 1-2 below). In my experience, identifying your Minimum Level of Confidence shouldn’t result in ‘analysis paralysis’. In contrast, it’s an important conversation to have throughout the product lifecycle to ensure that everyone fully understands what risks or unknowns are associated with the upcoming release. As an outcome of such a conversation you can decide whether to get customer feedback pre-release.
  4. Make sure you learn! – Whether you do or don’t engage with customers before launch, being clear about what you’re looking to learn from a release is paramount. Like I mentioned above, I view releasing something without learning from it  as a cardinal sin. It’s very important to continuously learn from real users and actual usage (or not) about your key hypotheses. These learnings – both quantitive and qualitative – will give you the data points to iterate or terminate a product.

Fig. 1 – Questions and criteria to check your confidence about launching a product or feature:

  • Internal quality assurance – Have you tested your product feature to ensure there are no obvious bugs or gaps in the user experience? Even if you don’t test with customers prior to launch, you should test some key acceptance scenarios internally before launch to make sure the product works as intended.
  • Does the feature or product touch on core user experience? – If “yes” is the answer to this, then I recommend you do test with customers prior to launch to identify any major usability issues worth solving before launch. You typically need to test with no more than five customers to unearth any critical usability issues.
  • How confident are you? – The combination of low confidence in something which your business has got a lot riding can be deadly. Yes, one can always try to do damage limitation, but it might already be too late at the time of you trying to repair things! The idea behind determining your confidence levels upfront isn’t a scientific one. Instead, it enables a conversation, making sure that people have got their eyes wide open and understand the level of risk and unknowns involved in an upcoming product launch (see Fig. 2 below).

Fig. 2 – Basic confidence levels to consider before launch:

  • High Confidence: Our confidence in the upcoming release is high because we tested it thoroughly internally, have launched a similar product or feature before or if there’s an issue the fallout will be small.
  • Low Confidence: Our confidence in the upcoming release is low because we haven’t fully tested it, it’s based on new technology or creates a totally new user experience.

 

 

 

Main learning point: Even if you decide not to generate customer learnings before a product launch, make sure you at learn after launch. Launch and learn. Don’t launch without learning!

 

Related links for further learning:

  1. https://www.mindtheproduct.com/2017/02/the-life-of-a-product-manager-learning-by-doing/
  2. https://www.intercom.com/blog/shipping-is-your-companys-heartbeat/
  3. https://medium.com/@joshelman/a-product-managers-job-63c09a43d0ec
  4. https://uxplanet.org/10-things-i-learned-from-jason-fried-about-building-products-5b6694ff02aa
  5. http://time.com/13549/the-10-worst-product-fails-of-all-time/
  6. https://twitter.com/jasonfried/status/935555293014036480
  7. https://247wallst.com/special-report/2014/03/03/worst-product-flops-of-all-time/2/
  8. https://www.macworld.com/article/2047342/remembering-the-newton-messagepad-20-years-later.html
  9. https://www.nytimes.com/1993/09/26/business/the-executive-computer-so-far-the-newton-experience-is-less-than-fulfilling.html

Book review: “Autonomy”

Lawrence Burns is a veteran of the automative industry. Having worked his entire professional career in the car industry – in Detroit, the birthplace of modern car manufacturing no less – you might expect Burns to be apprehensive about ‘change’ and modern technology. The opposite couldn’t be more true of Burns, since he’s been an advocate for driverless cars for the past 15+ years, starting his foray into this field whilst at General Motors.

In his latest book, “Autonomy: The Quest to Build the Driverless Car – and How It Will Reshape Our World”, Burns and cowriter Christopher Shulgan paint a picture of driverless cars dominating our streets and roads, and having a positive impact on the environment and transportation as a whole. For those sceptics out there who dismiss driverless cars as science fiction, I recommend they read “Autonomy” and take note of the technology and societal developments Burns describes:

Getting started, the DARPA Challenge and Google’s “Project Chauffeur”:

The book starts off with the story of the “DARPA Challenge” in 2004 and how this helped shaped learning and development with respect to driverless cars. Burns gives the reader a good close-up of the experiences and learnings from one of the teams that took part in this challenge. At this first DARPA challenge, every vehicle that took part crashed, failed or caught fire, highlighting the early stage of driverless technology at the time.

Image taken from: https://www.wired.com/story/darpa-grand-urban-challenge-self-driving-car/

Driverless cars are the (near) future:

Bob Lutz, former executive of Ford, Chrysler and General Motors, wrote an essay last year titled “Kiss the good times goodbye”, in which he makes a clear statement about the future of the automotive industry: “The era of the human-driven automobile, its repair facilities, its dealerships, the media surrounding it – all will be gone in 20 years.” There’s no discussion that driverless cars are coming, especially that both car and technology giants are busy developing and testing. When I attended a presentation by Burns a few months agogo, he showed the audience  examples of both self driving cars and trucks:

Image taken from: http://www.autonews.com/article/20170316/MOBILITY/170319877/bmw-says-self-driving-car-to-be-level-5-capable-by-2021

Image taken from: https://newatlas.com/volvo-vera-self-driving-truck/56312/

In “Autonomy”, Burns brings Lutz’ predictions to life through the fictitious example of little Tommy and his family. In this example, Tommy steps into a driverless which has been programmed to take him to school in the morning. Tommy’s grandma will be picked up by a driverless two-person mobility pod to take her to a bridge tournament. Burns describes a world where car ownership will be a thing of the past; people using publicly available fleets of self driving cars instead.

Image taken from: https://www.thenational.ae/business/technology/autonomous-pods-the-future-of-city-driving-1.730283

Together with Chris Borroni-Bird, Burns has done extensive research into the potential impact of an electronic self driven car, looking at metrics such as “total expense per mile”, “cost savings per mile” and “estimated number of parts”. Borroni-Bird and Burns provide some compelling stats, especially when contrasted against conventional cars. Reading these stats helps to make the impact of driverless technology a lot more tangible, turning it from science fiction or future music into a realistic prospect.

Main learning point: “Autonomy” by Lawrence is an insightful book about a driverless future, written by a true connoisseur of the car industry and the evolution of driverless technology.

Related links for further learning:

  1. https://spectrum.ieee.org/cars-that-think/transportation/self-driving/auto-consultant-lawrence-burns-dishes-the-dirt-on-waymo
  2. https://www.youtube.com/watch?v=-pLM-2bxNMc
  3. https://www.youtube.com/watch?v=SJVKY1DtZ84
  4. https://www.forbes.com/sites/greggardner/2018/08/23/an-interview-with-self-driving-visionary-larry-burns-co-author-of-autonomy/
  5. http://www.autonews.com/article/20171105/INDUSTRY_REDESIGNED/171109944/industry-redesigned-bob-lutz
  6. https://lucidmotors.com/
  7. https://electrek.co/2017/01/02/lucid-motors-autonomous-tech-all-electric-sedan-mobileye/
  8. http://www.thedrive.com/opinion/9024/who-is-really-1-in-self-driving-cars-you-wouldnt-know-it-from-navigants-controversial-report
  9. https://news.stanford.edu/2017/05/22/stanford-scholars-researchers-discuss-key-ethical-questions-self-driving-cars-present/
  10. https://www.thenational.ae/business/technology/autonomous-pods-the-future-of-city-driving-1.730283
  11. https://www.wired.com/story/darpa-grand-urban-challenge-self-driving-car/
  12. https://spectrum.ieee.org/cars-that-think/transportation/self-driving/google-has-spent-over-11-billion-on-selfdriving-tech

What product managers can learn about Design Systems

What makes a good product? What makes a well designed product? A few years ago, I learned about design principles and how principles such as “not getting in the way (of the user)” and “content first” can drive product design. Imagine my initial confusion and intrigue, as a non-designer, when I first heard about a “design system”. Chris Messina – former designer at Uber – has come up with a useful definition of what a design system is:

 

“Design systems provide a convenient, centralized, and evolving map of a brand’s known product territories with directional pointers to help you explore new regions.”

 

Later, Messina went on to add that “Design never was just how it looks, but now it’s also how it sounds, how it speaks, and where it can go.” Apart from capturing how brand and product communicate, look and feel, a design system is also a critical component when it comes to scale. Just take this statement by Vikram Babu – product designer at Gigster – for example:

 

“The problem facing design today isn’t a shortage of skills or talent but that design doesn’t scale when you move from a few screens of designed components to a platform of developed patterns where adding people only complicates the problem… hence design systems.” 

 

The key thing I learned about the value of design systems is that they intend to go beyond just a collection of design elements. Typically, companies will have a style guide. However, more often than not these style guides contain a bunch of design elements or patterns, but not create a fully comprehensive design language or tone of voice, as Nathan Curtis – owner of the EightShapes design firm – explains:

 

“A style guide is an artefact of the design process. A design system is a living, funded product with a roadmap & backlog, serving an ecosystem.” 

 

This raises the question how one goes about creating a design system. Some things that I’ve learned in this respect:

Before you get started

  1. What’s your company vision look like? And is mission?
  2. Which problem is your company looking to solve and why? For whom?
  3. What are the company values which underpin your company culture, product and service?
  4. What problem(s) are we trying to solve through the design system? Why?
  5. What’s the desired impact we expect the design system to have on the way we work?

Getting started

  1. What does the current design and design look like? What works and what doesn’t? Identify the gaps.
  2. Define some underlying design principles, which underpin a fluid and developing ‘design ecosystem’ (see Airbnb as a good example; Fig. 1 below).
  3. Create a visual design language, which comprises a number of distinct but ever evolving components (I loved Adobe’s Nate Baldwin breakdown of some of these components; see Fig. 2 below). Common components of a visual design language are: colour, typography, iconography, imagery, illustrations, sizing and spacing.
  4. Create a User Interface and pattern library.
  5. Document what each component is and how to use it.

 

Fig. 1 – Airbnb design principles – Taken from: https://airbnb.design/building-a-visual-language/

  • Unified: Each piece is part of a greater whole and should contribute positively to the system at scale. There should be no isolated features or outliers.
  • Universal: Airbnb is used around the world by a wide global community. Our products and visual language should be welcoming and accessible.
  • Iconic: We’re focused when it comes to both design and functionality. Our work should speak boldly and clearly to this focus.
  • Conversational: Our use of motion breathes life into our products, and allows us to communicate with users in easily understood ways.

 

Fig. 2 – The foundation of creating a Visual Design Language by Nate Baldwin – Taken from: https://medium.com/thinking-design/what-is-a-design-language-really-cd1ef87be793

  • Clearly defined semantics (and no, “error”, “warning”, “success”, and “info” aren’t nearly enough)
  • Thorough and mature mapping of core elements of design with clear purposes and meanings
  • A solid family of UI components and patterns that effectively support the semantics, and use design elements (based on theirmeanings) to support the meaning of the components
  • Thorough, comprehensive documentation about the visual communication system

 

To make this a bit more concrete, I’ll look at three good examples of design systems, by Google, Bulb and Salesforce.

 

Google Material Design

 

 

Bulb

 

 

Salesforce Lightning Design System

 

 

 

Main learning point: It’s important for product managers to understand what a Design System is and the purposes it serves. Even if you’re not directly involved in creating or applying a Design System, it’s key to understand your company’s design language and how it applies to your product.

 

 

Related links for further learning:

  1. https://bulb.co.uk/blog/introducing-bulbs-design-system
  2. http://design.bulb.co.uk/#/patterns/styles/colors/README.md
  3. https://www.fastcompany.com/90160960/the-design-theory-behind-amazons-5-6-billion-success
  4. https://www.invisionapp.com/blog/guide-to-design-systems/
  5. https://www.invisionapp.com/blog/scale-design-systems/
  6. https://medium.muz.li/how-to-create-a-style-guide-from-scratch-tips-and-tricks-e00f25b423bf
  7. https://www.invisionapp.com/blog/secrets-design-leadership/
  8. https://www.lightningdesignsystem.com/
  9. https://www.uxpin.com/create-design-system-guide
  10. https://medium.freecodecamp.org/how-to-build-a-design-system-with-a-small-team-53a3276d44ac
  11. https://www.uxpin.com/studio/ebooks/create-design-system-guide-checklist/
  12. https://blog.prototypr.io/design-system-ac88c6740f53
  13. https://medium.com/thinking-design/what-is-a-design-language-really-cd1ef87be793
  14. https://airbnb.design/building-a-visual-language/
  15. https://material.io/design/

App review: Forest

My quick summary of Forest before using the app – I think I first heard Nir Eyal, who specialised in consumer psychology, talk about Forest. Given that Nir mentioned the app, I can imagine Forest impacts people behaviour, helping them achieve specific outcomes.

How does Forest explain itself in the first minute? – “Stay focused, be presented” is Forest’s strap line which I see first. This strap line is followed swiftly followed by a screen that says “Plant a Tree” and explains that “Whenever you want to focus on your work, plant trees.” This suggest to me that Forest is an app which aims to help people focus on their work and eradicate all kinds of distractions.

How does Forest work? – The app first explains its purpose in a number of nicely designed explanatory screens.

After clicking “Go”, I land on a screen where I can adjust time; presumably the time during which I want to focus and avoid any interruptions.

I set the time at 10 minutes and click “Plant”. I love how, as the time progresses, the messages at the top of the screen keep alternating, from “Don’t look at me!” to “Don’t look at me!” to “Hang in there!” Nice messages to help keep me focus and not fall prey to checking my phone. At any stage, I can opt to “Give up” which presumably means that the tree that I’ve been planting – through staying focused – will be killed.

I’m motivated to see this through and plant my first tree. When I complete my 10 minutes of uninterrupted time, I expect to see a nice tree right at the end of it. Try and imagine my disappointment when I don’t see a tree but instead am encouraged to create a Forest acount

Did Forest deliver on my expectations? – I can see how Forest helps people to focus and avoid checking their phone constantly. Just want to explore the gamification element of the app a bit more.

Managing products of the future – Business as usual?

“Managing products of the future” came up when I was thinking of a suitable title for a piece about products that look and feel very different to most products that we see today. Products such as driverless cars and voice assistants popped into my head as examples of products that are likely to dominate our daily lives before we know it.

However, these products are here already and I’m keen to look at if and how this does affect the role and focus of product management.

Will we manage products differently when the user interface of these products changes? Do we need to think differently about our products when data becomes the main output? Will customer needs and expectations evolve? If so, how? These and other questions I will start thinking about; considering the nature of machine learning, different product scenarios and their impact on the role of the product manager.

Taken from: https://robertmerrill.wordpress.com/2009/04/15/the-future-is-already-here/

It’s easy to get swept up by the hype surrounding AI and products based on machine learning, and to start feeling pretty dystopian about the future. But how much will actually change from a product management point of view? People will continue to have specific needs and problems. As product managers, we’ll continue to look at best ways of solving these problems. Granted, the nature of people’s needs and problemx will evolve, as it has always done, but this won’t alter the problem solving and people centric nature of product management.

To illustrate this, let’s look at some AI-base products and the customer needs and problems that they’re aiming to solve: Google Photos, Sonos One and Eigen Technologies.

Google Photos

Google Photos’ strap-line is “One home for all your photos – organised and easy to find”. Over the coming months, Google Photos will roll out the following features:

  • Using facial recognition, Google Photos will know who’s in a picture and will offer a one-tap option to share it with the person in question – provided that this person is in your phone’s contact list, Google Photos will have learned this person’s face. If that person appears in multiple images, Google Photos will even suggest to share all of them in one go.
  • Automated image editing suggestions, Google Photos will suggest different corrections based on the look and quality of the image. For example, if there issues with the brightness of the image, Google Photos will automatically display a “Fix brightness” suggestion.

Taken from: https://www.digitaltrends.com/photography/google-photos-suggested-edits/

With these new features, Google Photos aim to address customer needs with regard to sharing pictures and improving image quality respectively. These needs aren’t new per se, but the ‘intelligent’ aspect of Google Photos’ approach is.

Sonos One

The Sonos One is entirely controlled by voice. The speaker works fully with Amazon Alexa, which means that if you’ve got an Amazon Alexa compatible device, you can control your Sonos sound system through Amazon Alexa. Because Alex is a native app within the Sonos platform, you don’t even need to have an external Amazon device – i.e. Echo or the Dot – installed to control your Sonos One speaker. The installation of the Alexa mobile app will be enough.

Taken from: https://uniquehunters.com/sonos-one-marries-amazons-alexa-high-end-audio-hardware-exquisite-musical-enjoyment/

The integration with the Amazon’s Alexa voice assistant is a logical next step within Sonos’ mission to “empower everyone to listen better” and makes it easier for people to control the music they listen to. Granted, the user interface of Sonos One is different to other product; it doesn’t have buttons, for example. However, it still is a product like any other in a sense that it delivers tangible value to customers by solving their music listening needs.

Eigen Technologies

“Turn your documents into data” is London and New York based Eigen Technologies’ mission statement. The company enables the mining of documents for specific data. For example, if you work for a mortgage lender and are looking to make a decision about the credit worthiness of a home, Eigen’s data extraction technology helps to quickly pull out key ‘decision inputs’ from a number of – often very lengthy – property documents.

Taken from: https://www.artificiallawyer.com/2017/11/03/legal-ais-dark-horse-eigen-technologies-comes-into-the-light/

The way in which Eigen Technologies use machine learning algorithms, is ultimately to improve the speed and quality of decision making. Even though the underlying technology is based on machine learning, the outcome is very much like that of any other product: a clear user interface which shows the relevant document data that a user is interested in and needs to make decisions.

Main learning point: AI and machine learning based products will no doubt change the ways in which we interact with products and what we expect of them. However, existing examples such as Google Photos and Sonos One already show that the core of the product manager’s role will remain unchanged: building the right product for the right people and building it right!

 

Related links for further learning:

  1. https://productsthatcount.com/blog/66-google-vp-product-ai/
  2. https://www.wired.com/2015/05/bradley-horowitz-says-that-google-photos-is-gmail-for-your-images/
  3. https://blog.sonos.com/en-gb/making-sonos-one/
  4. https://www.engadget.com/2018/05/08/google-photos-will-add-ai-powered-suggestions-to-fix-your-images/
  5. https://techcrunch.com/2017/10/04/sonos-announces-alexa-controlled-wireless-speakers/
  6. https://www.digitaltrends.com/photography/google-photos-suggested-edits/
  7. http://www.wired.co.uk/article/sonos-one-alexa-review-uk-price
  8. https://techcrunch.com/2018/02/20/sonos-one-is-the-speaker-to-beat-for-those-that-want-great-sound-and-smarts/
  9. http://uk.businessinsider.com/connected-speakers-explainer-sonos-libratone-echo-google-home-2018-4
  10. https://assistant.google.co.uk/
  11. https://www.sonos.com/en-gb/social-impact
  12. https://www.artificiallawyer.com/2017/11/03/legal-ais-dark-horse-eigen-technologies-comes-into-the-light/
  13. https://www.eigentech.com/
  14. https://blog.bolt.io/what-cracking-open-a-sonos-one-tells-us-about-the-sonos-ipo-dcab49155643

Michael Margolis: “user research, quick and dirty” (2)

I wrote earlier about Michael Margolis’ Startup Lab workshop, in which he teaches attendees about “User research, quick and dirty”.  Michael Margolis, UX Research Partner at Google Ventures covers user research topics such as user interview types and getting to the right learnings. He also offer a number of practical tips with respect to recruiting users and how to best conduct user interviews:

Recruiting users

Margolis mentions that recruiting 5 people to get feedback from is often sufficient, especially when you’re doing usability testing. He does stress that it’s worth the effort recruiting these people selectively and carefully, as this will help generate better results and avoid wasting time. Creating a simple participant screener document or survey is a good way to recruit the ‘right’ users (see an example in Fig. 1 below).

Fig. 1 – Ethnio.io screen survey example – Taken from: https://www.nngroup.com/articles/live-intercept-remote-test/

Margolis lists a number of very helpful questions to feed into your screener document, in order to engage with the right users (and exclude those that aren’t right):

Users to include

  • Who do you want to want to talk to?
  • What exact criteria will identify the people you want to talk to?
  • What screening questions will you ask? (questions shouldn’t reveal “right” answers)

Users to exclude

  • Who do you want to want to exclude?
  • What exact criteria will identify the people you want to excliude?
  • What screening questions will you ask? (questions shouldn’t reveal “right” answers)

Conducting a user interview

Fig. 2 – Arc of a typical user interview, by Michael Margolis – Taken from: https://library.gv.com/the-gv-research-sprint-finalize-schedule-and-complete-interview-guide-day-3-b8cddb8f931d

The representation of the user interview in the form of an arc, I probably found the most helpful aspect that Margolis (see Fig. 2 above). This arc really helps in structuring an interview, identifying the appropriate sequence of activities during the interview.

Main learning point: User research doesn’t have to be complicated, super time consuming or overly expensive. A huge thanks to Michael Margolis for sharing such a wealth of very useful and practical user research insights!

 

Related links for further learning:

  1. https://www.usertesting.com/blog/2015/01/29/screener-questions/
  2. https://www.nngroup.com/articles/live-intercept-remote-test/
  3. https://library.gv.com/the-gv-research-sprint-finalize-schedule-and-complete-interview-guide-day-3-b8cddb8f931d
  4. https://www.nngroup.com/articles/interviewing-users/

Michael Margolis: “user research, quick and dirty” (1)

Why do I keep coming across businesses that struggle to engage with their (prospective) customers, to learn about their needs and behaviours? Too often for my liking, I hear comments like:

“Marc, we’re a startup, we don’t have the time and budget to do customer research!” 

“I’m not allowed to talk to customers.” 

“In my old place, we used to have a dedicated user research team and they’d just give me their research report on a platter, after them having spoken to users.”

It therefore felt quite timely when a colleague pointed me in the direction of Michael Margolis, UX Research Partner at Google Ventures.  Back in 2013, Margolis delivered a great Startup Lab workshop in which he covered the ins and outs of “User research, quick and dirty”. The recording of the 90 minute workshop is available on YouTube and you can find Margolis’ slides here (see also Fig. 1 below).

Fig. 1 – Michael Margolis’ Startup Lab workshop: “User Research, Quick ‘n’ Dirty” – Published on 26 February 2013 on https://youtu.be/WpzmOH0hrEM

I watched Margolis’ workshop in full and these are my main takeaways:

Seeing through users’ eyes

Margolis started off his session by talking about the importance of continuously learning about users, seeing things through their eyes. In a subsequent Medium post, Margolis writes that in his experience, startups will typically use UX research to achieve one of these objectives:

  1. Improve a process or worklflow
  2. Better understand customer shopping habits
  3. Evaluate concepts
  4. Test usability
  5. Refine a value proposition

Two types of user interviews

It’s great to hear Margolis making a distinction between two types of interviews:

  • Usability: A usability interview is all about learning whether users can actually use your product and achieve their goals with it. Can users do it? Can they understand it? Can they discover features?
  • Discovery: Discovery type user interviews tend to be more contextual, and delve more into the actual user. Who? Where? When? Why? How? All key questions to explore as part of discovery, as well as the user’s existing behaviours, goals, needs and problems.

Margolis then talks about combining the two interview types and highlight two sample questions to illustrate this combination:

“How do you do things now?”

“How do you think about these things?”

The distinction between “usability” and “discovery” isn’t just an artificial one. I love Margolis’ focus on objectives, acknowledging that objectives are likely to vary depending on the type of product, its position within the product lifecycle and the learnings that you’re looking to achieve. I’ve found – at my own peril – that it’s easy to jump straight into defining user tasks or an interview script, without thinking about your research objective and what Margolis calls “North Star questions” (see Fig. 2 below).

Fig. 2 – Michael Margolis’ 5 studies startups needs most- Taken from:  https://library.gv.com/field-guide-to-ux-research-for-startups-8569114c27fb – Published on 5 May 2018 

Margolis provides some very useful pointers about discovery and usability questions, which you can use to create a research plan and an interview guide:

Sample discovery questions – as suggested by Michael Margolis:

  • What are users’ behaviours, attitudes and expectations towards the product?
  • Who are the key user groups? What are their needs and behaviours?
  • What are the pros/cons of different designs? Why?
  • What are the pros/cons of competitor products?
  • How are people using existing/competitor products? What features are mots important and why?
  • What barriers hinder users from adopting <product>?

Sample usability questions – as suggested by Michael Margolis:

  • Can users discover feature X?
  • Are users able to successfully complete primary tasks? Why (not)?
  • Do users understand feature X? Why (not)?

In a similar vein, I believe it’s important to distinguish between problem and solution interviews. There’s a risk of your customer insights becoming muddled when you mix problem and solution interviews, especially if you alternate problem questions with solution questions.

In a problem interview, you want to find out 3 things:

  • Problem – What problem are you solving? For example, what are the common frustrations felt by your customers and why? How do their problems rank? Ask your customers to create a top 3 of their problems (see the problem interview script in Fig. 1 below).
  • Existing alternatives – What existing alternatives are out there and how does your customer perceive your competition and their differentiators? How do your customers solve their problems today?
  • Customer segments – Who has these problems and why? Is this a viable customer segment?

Fig. 3 –  Outline of a problem interview script – Taken from: Ash Maurya – “Running Lean”

In a solution interview, you want to find out 3 things:

  • Early adopters – Who has this problem and why? How do we identify and engage with early adopters? (see Fig. 3 below)
  • Solution – How will you solve their problems? What features do you need to build as part of your solution, why?
  • Pricing/Revenue – What is the pricing model for your product or service? Will customers pay for it, why?

Fig. 4 – Outline of a solution interview script – Taken from: Ash Maurya – “Running Lean”

Main learning point: In his Startup Lab workshop, Michal Margolis, drops a lot of very valuable tips on how to best keep customer research quick and simple, whilst still learning the things about your customer and/or product that you’re keen to learn. So much so that Michael Margolis’ tips warrant another blog post, which I’ll share soon!

Related links for further learning:

  1. https://library.gv.com/field-guide-to-ux-research-for-startups-8569114c27fb
  2. https://library.gv.com/user-research-quick-and-dirty-1fcfa54c91c4
  3. https://www.slideshare.net/LauraKlein1/shut-the-hell-up-other-tips-for-learning-from-users
  4. https://www.youtube.com/watch?v=WpzmOH0hrEM
  5. https://library.gv.com/tagged/design
  6. https://medium.com/@maa1/book-review-just-enough-research-2d714d447eda
  7. https://medium.com/@maa1/my-product-management-toolkit-23-customer-empathy-a1e66ff15012