Book review: “Designing with Data”

I’d been looking forward to Rochelle King writing her book about using data to inform designs (I wrote about using data to inform product decisions a few years ago, which post followed a great conversation with Rochelle).

Earlier this year, Rochelle published “Designing with Data: Improving the User Experience with A/B Testing”, together with Elizabeth F. Churchill and Caitlin Tan. The main theme of “Designing with Data” the book is the authors’ belief that data capture, management, and analysis is the best way to bridge between design, user experience, and business relevance:

  1. Data aware — In the book, King, Churchill and Tan distinguish between three different ways to think about data: data driven; data informed and data aware (see Fig. 1 below). The third way listed, being ‘data aware’, is introduced by the authors: “In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions.” If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.
  2. How much data to collect? — The authors make an important distinction between “small sample research” and “large sample research”. Small sample research tends to be good for identifying usability problems, because “you don’t need to quantify exactly how many in the population will share that confusion to know it’s a problem with your design.” It reminded me of Jakob Nielsen’s point about how the best results come from testing with no more than 5 five people. In contrast, collecting data from a large group of participants, i.e. large sample research, can give you more precise quantity and frequency information: how many people people feel a certain way, what percentage of users will take this action, etc. A/B tests are one way of collecting data at scale, with the data being “statistically significant” and not just anecdotal. Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance.
  3. Running A/B tests: online experiments — The book does a great job of explaining what is required to successfully running A/B tests online, providing tips on how to sample users online and key metrics to measure (Fig. 2) .
  4. Minimum Detectable Effect — There’s an important distinction between statistical significance — which measure whether there’s a difference — and “effect”, which quantifies how big that difference is. The book explains about determining “Minimum Detectable Effect” when planning online A/B tests. The Minimum Detectable Effect is the minimum effect we want to observe between our test condition and control condition in order to call the A/B test a success. It can be positive or negative but you want to see a clear difference in order to be able to call the test a success or a failure.
  5. Know what you need to learn — The book covers hypotheses as an important way to figure out what it is that you want to learn through the A/B test, and to identify what success will look like. In addition, you can look at learnings beyond the outcomes of your A/B test (see Fig. 3 below).
  6. Experimentation framework — For me, the most useful section of the book was Chapter 3, in which the authors introduce an experimentation framework that helps planning your A/B test in a more structured fashion (see Fig. 4 below). They describe three main phases — Definition, Execution and Analysis — which feed into the experimentation framework. The ‘Definition’ phase covers the definition of a goal, articulation of a problem / opportunity and the drafting of a testable hypothesis. The ‘Execution’ phase is all about designing and building the A/B test, “designing to learn” in other words. In the final ‘Analysis’ phase you’re getting answers from your experiments. These results can be either “positive” and expected or “negative” and unexpected (see Fig. 5–6 below).

Main learning point: “Designing with Data” made me realise again how much thinking and designing needs to happen before running a successful online A/B test. “Successful” in this context means achieving clear learning outcomes. The book provides a comprehensive overview of the key considerations to take into account in order to optimise your learning.

Fig. 1 — Three ways to think about data — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 3–9

  • Data driven — With a purely data driven approach, it’s data that determine the fate of a product; based solely on data outcomes businesses can optimise continuously for the biggest impact on their key metric. You can be data driven if you’ve done the work of knowing exactly what your goal is, and you have a very precise and unambiguous question that you want to understand.
  • Data informed — With a data informed approach, you weigh up data alongside a variety of other variables such as strategic considerations, user experience, intuition, resources, regulation and competition. So adopting a data-informed perspective means that you may not be as targeted and directed in what you’re trying to understand. Instead, what you’re trying to do is inform the way you think about the problem and the problem space.
  • Data aware — In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions. If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.

Fig. 2 — Generating a representative sample — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 45–53

  • Cohorts and segments — A cohort is a group of users who have a shared experience. Alternatively, you can also segment your user base into different groups based on more stable characteristics such as demographic factors (e.g. gender, age, country of residence) or you may want them by their behaviour (e.g. new user, power user).
  • New users versus existing users — Data can help you learn more about both your existing understand prospective future users, and determining whether you want to sample from new or existing users is an important consideration in A/B testing. Existing users are people who have prior experience with your product or service. Because of this, they come into the experience with a preconceived notion of how your product or service works. Thus, it’s important to be careful about whether your test is with new or existing users, as these learned habits and behaviours about how your product used to be in the past can bias in your A/B test.

Fig. 3 — Know what you want to learn — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, p. 67

  • If you fail, what did you learn that you will apply to future designs?
  • If you succeed, what did you learn that you will apply to future designs?
  • How much work are you willing to put into your testing in order to get this learning?

Fig. 4 — Experimentation framework — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 83–85

  1. Goal — First you define the goal that you want to achieve; usually this is something that is directly tied to the success of your business. Note that you might also articulate this goal as an ideal user experience that you want to provide. This is often the case that you believe that delivering that ideal experience will ultimately lead to business success.
  2. Problem/opportunity area — You’ll then identify an area of focus for achieving that goal, either by addressing a problem that you want to solve for your users or by finding an opportunity area to offer your users something that didn’t exist before or is a new way of satisfying their needs.
  3. Hypothesis — After that, you’ll create a hypothesis statement which is a structured way of describing the belief about your users and product that you want to test. You may pursue one hypothesis or many concurrently.
  4. Test — Next, you’ll create your test by designing the actual experience that represents your idea. You’ll run your test by launching the experience to a subset of your users.
  5. Results — Finally, you’ll end by getting the reaction to your test from your users and doing analysis on the results that you get. You’ll take these results and make decisions about what to do next.

Fig. 5 — Expected (“positive”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 227–228

  • How large of an effect will your changes have on users? Will this new experience require any new training or support? Will the new experience slow down the workflow for anyone who has become accustomed to how your current experience is?
  • How much work will it take to maintain?
  • Did you take any “shortcuts” in the process of running the test that you need to go back and address before your roll it out to a larger audience (e.g. edge cases or fine-tuning details)?
  • Are you planning on doing additional testing and if so, what is the time frame you’ve established for that? If you have other large changes that are planned for the future, then you may not want to roll your first positive test out to users right away.

Fig. 6 — Unexpected and undesirable (“negative”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 228–231

  • Are they using the feature the way you think they do?
  • Do they care about different things than you think they do?
  • Are you focusing on something that only appeals to a small segment of the base but not the majority?

Related links for further learning:

  1. https://www.ted.com/watch/ted-institute/ted-bcg/rochelle-king-the-complex-relationship-between-data-and-design-in-ux
  2. http://andrewchen.co/know-the-difference-between-data-informed-and-versus-data-driven/
  3. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  4. https://vwo.com/ab-split-test-significance-calculator/
  5. https://www.kissmetrics.com/growth-tools/ab-significance-test/
  6. https://select-statistics.co.uk/blog/importance-effect-sample-size/
  7. https://www.optimizely.com/optimization-glossary/statistical-significance/
  8. https://medium.com/airbnb-engineering/experiment-reporting-framework-4e3fcd29e6c0
  9. https://medium.com/@Pinterest_Engineering/building-pinterests-a-b-testing-platform-ab4934ace9f4
  10. https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166

 

Design with Data.jpg

 

 

 

 

Book review: “Just Enough Research”

Back in 2013, Erika Hall, co-founder of Mule Design, wrote “Just Enough Research”. In this book, Hall explains why good customer research is so important. She outlines what makes research effective and provides practical tips on how to best conduct research. Reading “Just Enough Research” reminded me of reading “Rocket surgery made easy” by Steve Krug and “Undercover UX” by Cennydd Bowles, since all three books do a good job at both explaining and demystifying what it takes to do customer research.

These are the main things that I learned from reading “Just Enough Research”:

  1. What is research? – Right off the bat, Hall makes the point that in order to innovate, it’s important for you to know about the current state of things and why they’re like that. Research is systematic inquiry; you want to know more about a particular topic, so you go through a process to increase your knowledge. The specific type of process depends on who you are and what you need to know. This is illustrated through a nice definition of design research by Jane Fulton Suri, partner at design consultancy IDEO (see Fig. 1).
  2. Research is not asking people what they like! – I’m fully aware of how obvious this statement probably sounds. However, customer researcher is NOT about asking about what people do or don’t like. You might sometimes hear people ask users whether they like a particular product or feature; that isn’t what customer research is about. Instead, the focus is on exploring problem areas or new ideas, or simply testing how usable your product is.
  3. Generative or exploratory research – This is the research you do to identify the problem to solve and explore ideas. As Hall explains “this is the research you do before you even know what you’re doing.” Once you’ve gathered information, you then analyse your learnings and identify the most commonly voiced (or observed) unmet customer needs. This will in turn result in a problem statement or hypothesis to concentrate on.
  4. Descriptive and explanatory research – Descriptive research is about understanding the context of the problem that you’re looking to solve and how to best solve it. By this stage, you’ll have moved from “What’s a good problem to solve” to “What’s the best way to solve the problem I’ve identified?”
  5. Evaluative research – Usability testing is the most common form of evaluative research. With this research you test that your solution is working as expected and is solving the problem you’ve identified.
  6. Casual research – This type of research is about establishing a cause-and-effect relationship, understanding the ‘why’ behind an observation or pattern. Casual research often involves looking at analytics and carrying out A/B tests.
  7. Heuristic analysis – In the early stages of product design and development, evaluative research can be done in the form of usability testing (see point 5. above) or heuristic analysis. You can test an existing site or application before redesigning. “Heuristic” means “based on experience”. A heuristic is not a hard measure; it’s more of a qualitative guideline of best usability practice. Jakob Nielsen, arguably the founding father of usability, came up with the idea of heuristic analysis in 1990 and introduced ten heuristic principles (see Fig. 2).
  8. Usability testing – Testing the usability of a product with people is the second form of evaluative testing. Nielsen, the aforementioned usability guru, outlined five components that define usability (see Fig. 3). Hall stresses the importance of “cheap tests first, expensive tests later”; start simple – paper prototypes or sketches – and gradually up the ante.

Main learning point: “Just Enough Research” is a great, easy to read book which underlines the importance of customer research. The book does a great job in demonstrating that research doesn’t have to very expensive or onerous; it provides plenty of simple and practical to conduct ‘just enough research’.

 

Fig. 1 – Definition of “design research” by Jane Fulton Suri – Taken from: https://www.ideo.com/news/informing-our-intuition-design-research-for-radical-innovation

“Design research both inspires imagination and informs intuition through a variety of methods with related intents: to expose patterns underlying the rich reality of people’s behaviours and experiences, to explore reactions to probes and prototypes, and to shed light on the unknown through iterative hypothesis and experiment.”

Fig. 2 – Jakob Nielsen’s 10 Heuristics for User Interface Design (taken from: http://www.nngroup.com/articles/ten-usability-heuristics/)

  1. Visibility of system status – The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
  2. Match between system and the real world – The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
  3. User control and freedom – Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
  4. Consistency and standards – Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
  5. Error prevention – Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
  6. Recognition rather than recall – Minimise the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
  7. Flexibility and efficiency of use – Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
  8. Aesthetic and minimalist design – Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
  9. Help users recognise, diagnose, and recover from errors – Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
  10. Help and documentation – Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

Fig. 3 – Jakob Nielsen’s 5 components of usability – Taken from: Erika Hall. Just Enough Research, pp. 105-106

  • Learnability – How easy is it for users to accomplish basic tasks the first time they come across the design?
  • Efficiency – Once users have learned the design, how quickly can they perform tasks?
  • Memorability – When users return to the design after a period of not using it, how easily can they reestablish proficiency?
  • Errors – How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
  • Satisfaction – How pleasant is it to use the design?

 

My product management toolkit (23): customer empathy

A few weeks ago I attended the annual Mind the Product conference in San Francisco, where David Wascha delivered a great talk about some of his key lessons learned in his 20 years of product management experience. He impressed on the audience that as product managers we should “protect our customer”; as product managers we need to shield our teams, but ultimately we need to protect our customers and their needs.

Dave’s point really resonated with me and prompted me to think more about how product managers can best protect customers and their needs. I believe this begins with the need to fully understand your customers;  “customer empathy” is something that comes to mind here:

  1. What’s customer empathy (1)? – In the dictionary, empathy is typically defined as “the ability to understand and share the feelings of another.” In contrast, sympathy is about feeling bad for someone else because of something that has happened to him or her. When I think about empathising with customers, I think about truly understanding their needs or problems. To me, the ultimate example of customer empathy can be found in Change By Design, a great book by IDEO‘s Tim Brown. In this book, Brown describes an IDEO employee who wanted to improve the experience of ER patients. The employee subsequently became an emergency room patient himself in order to experience first hand what it was like to be in an ER.
  2. What’s customer empathy (2)? – I love how UX designer Irene Au describes design as “empathy made tangible”. Irene distinguishes between between analytical thinking and empathic thinking. Irene refers to a piece  by Anthony Jack of Case Western University in this regard. Anthony found that when people think analytically, they tend to not use those areas of the brain that allow us to understand other people’s experience. It’s great to use data to inform the design and build of your product, and any decisions you make in the process. The risk with both quantitative data (e.g. analytics and surveys) and qualitative data (e.g. user interviews and observations) is that you end up still being quite removed from what the customer actually feels or thinks. We want to make sure that we really understand customer pain points and the impact of these pain points on the customers’ day-to-day lives.
  3. What’s customer empathy (3)? – I recently came across a video by the Cleveland Clinic – a non-profit academic medical centre that specialises in clinical and hospital care – which embodies customer empathy in a very inspiring and effective way (see Fig. 1 below). The underlying premise of the video is all about looking through another person’s eyes, truly trying to understand what someone else is thinking or feeling.

Fig. 1 – Cleveland Clinic Empathy: The Human Connection to Patient Care – Wvj_q-o8&feature=youtu.be

I see customer empathy as a skill that can be learned. In previous pieces, I’ve looked at some of the tools and techniques you can use to develop customer empathy. This is a quick recap of three simple ways to get started:

Listen. Listen. Listen  I often find myself dying to say something, getting my two cents in. I’ve learned that this desire is the first thing that needs to go if you want to develop customer empathy. Earlier this year, I learned about the four components of active listening, from reading “The Art of Active Listening” . Empathy is one of the four components of active listening:

Empathy is about your ability to understand the speaker’s situation on an emotional level, based on your own view. Basing your understanding on your own view instead of on a sense of what should be felt, creates empathy instead of sympathy. Empathy can also be defined as your desire to feel the speaker’s emotions, regardless of your own experience.

Empathy Map – I’ve found empathy mapping to be a great way of capturing your insights into another person’s thoughts, feelings, perceptions, pain, gains and behaviours (see Fig. 2 below). In my experience, empathy maps tend to be most effective when they’ve been created collectively and validated with actual customers.

Fig. 2 – Example empathy map, by Harry Brignull – Taken from: “How To Run an Empathy Mapping & User Journey Workshop” https://medium.com/@harrybr/how-to-run-an-empathy-user-journey-mapping-workshop-813f3737067

Problem Statements – To me, product management is all about – to quote Ash Maurya – “falling in love with the problem, not your solution.” Problem statements are an easy but very effective way to both capture and communicate your understanding of customer problems to solve. Here’s a quick snippet from an earlier ‘toolkit post’, dedicated to writing effective problem statements:

Standard formula:

Stakeholder (describe person using empathetic language) NEEDS A WAY TO Need (needs are verbs) BECAUSE Insight (describe what you’ve learned about the stakeholder and his need)

Some simple examples:

Richard,who loves to eat biscuits wants to find a way to eat at 5 biscuits a day without gaining weight as he’s currently struggling to keep his weight under control.

Sandra from The Frying Pan Co. who likes using our data platform wants to be able to see the sales figures of her business for the previous three years, so that she can do accurate stock planning for the coming year.

As you can see from the simple sample problem statements above, the idea is that you put yourself in the shoes of your (target) users and ask yourself “so what …!?” What’s the impact that we’re looking to make on a user’s life? Why?

Main learning point: Don’t despair if you feel that you haven’t got a sense of customer empathy yet. There are numerous ways to start developing customer empathy, and listening to customers is probably the best place to start!

Related links for further learning:

  1. https://www.ideo.com/post/change-by-design
  2. https://designthinking.ideo.com/
  3. http://www.sciencedirect.com/science/article/pii/S1053811912010646
  4. http://www.insightsquared.com/2015/02/empathy-the-must-have-skill-for-all-customer-service-reps/
  5. https://www.youtube.com/watch?v=cDDWvj_q-o8&feature=youtu.be
  6. https://www.linkedin.com/pulse/20131002191226-10842349-the-secret-to-redesigning-health-care-think-big-and-small?trk=mp-reader-card
  7. https://medium.com/@harrybr/how-to-run-an-empathy-user-journey-mapping-workshop-813f3737067
  8. https://blog.leanstack.com/love-the-problem-not-your-solution-65cfbfb1916b
  9. https://www.interaction-design.org/literature/article/stage-2-in-the-design-thinking-process-define-the-problem-and-interpret-the-results
  10. https://robots.thoughtbot.com/writing-effective-problem-statements
  11. https://www.slideshare.net/felipevlima/empathy-map-and-problem-statement-for-design-thinking-action-lab

 

App review: Toutiao

Fig. 1 – Screenshot of www.toutiao.com/ homepage 

When I first heard about Toutiaou I thought it might be just another news app, this coming one from China. I learned, however, very quickly that Toutiaou is much more than just a news app; at the time of writing, Toutiao has more than 700 million users in total, with ore than 78 million users reading over 1.3 billion articles on a daily basis.

Toutiao, known officially as Jinri Toutiao, which means “Today’s Headlines”, has a large part of its rapid rise to its ability to provide its users with a highly personalised news feed. Toutiao is a mobile platform that use machine learning algorithms to recommend content to its users, based on previous content accessed by users and their interaction with the content (see Fig. 2).

Fig. 2 – Screenshot of Toutiao iOS app

I identified a number of elements that contribute to Toutiao’s success:

  1. AI and machine learning – Toutiao’s flagship value proposition to its users, having its own dedicated AI Lab in order to constantly further the development of the AI technology that underpins its platform. Toutiao’s algorithms learn from the types of content its users interact with and the way(s) in which they interact with this content. Given that Toutiao users spend on average 76 minutes per day on the app, there’s a wealth of data for Toutiao’s algorithms to learn form and to base personalisations on.
  2. Variety of content types to choose from – Toutiao enables its users to upload short videos, and Toutiao’s algorithms of will recommend selected videos to appropriate users (see Fig. 3). Last year, Ivideos on Toutiao were played 1.5 billion times per day, making Toutiao China’s largest short video platform. Users can also upload pictures, similar to Instagram or Facebook, users can share their pictures, with other users being abel to like or comment on this content (see Fig. 4).
  3. Third party integrations – Toutiao has got strategic partnerships in place with the likes of WeChat, a highly popular messaging app (see Fig. 5), and jd.com, a local online marketplace. It’s easy to see how Toutiao is following an approach whereby they’re inserting their news feed into a user’s broader ecosystem.

Main learning point: I was amazed by the scale at which Toutiao operate and the levels at which its users interact with the app. We often talk about the likes of Netflix and Spotify when it comes to personalised recommendations, but with the amount of data that Toutiao gathers, I can they can create a highly tailored content experience for their users.

Fig. 3 – Screenshot of video section on Toutiao iOS app 

Fig. 4 – Screenshot of user generated content feed on Toutiao iOS app

IMG_4954

Fig. 5 – Screenshot of Toutiao and WeChat integration on Toutiao iOS app

Related links for further learning:

  1. https://www.toutiao.com/
  2. https://www.crunchbase.com/organization/toutiao#/entity
  3. http://technode.com/2017/06/05/podcast-analyse-asia-187-toutiao-with-matthew-brennan/
  4. https://www.technologyreview.com/s/603351/the-insanely-popular-chinese-news-app-that-youve-never-heard-of/
  5. https://www.forbes.com/sites/ywang/2017/05/26/jinri-toutiao-how-chinas-11-billion-news-aggregator-is-no-fake/#24d401d64d8a
  6. https://en.wikipedia.org/wiki/Toutiao
  7. http://lab.toutiao.com/
  8. https://www.liftigniter.com/toutiao-making-headlines-machine-learning/
  9. https://techcrunch.com/2017/02/01/chinese-news-reading-app-toutiao-acquires-flipagram/
  10. https://lotusruan.wordpress.com/2016/03/20/cant-beat-giant-companies-on-wechatweibo-try-toutiao/
  11. https://www.chinainternetwatch.com/tag/toutiao/

 

 

 

Some good conversational UI examples to learn from

It was Dennis Mortensen – CEO/Founder of x.ai – who made me aware a few years ago of the concept of ‘invisible interfaces’. He talked about applications no longer needing a graphical user interface (GUI), taking “Amy” – x.ai’s virtual personal assistant as a good example (see Fig. 1 below).

hi-im-amy-xai

Fig. 1 – Amy, x.ai’s virtual assistant – Taken from: http://www.agilenetnyc.com/business/x-ai/

Since then, I’ve been keeping more of an eye out for bots and virtual assistants, which can run on Slack, WeChat, Facebook Messenger or Amazon Echo. Like “Amy” these applications can be driven entirely by complex machine learning algorithms, or can be more ‘smoke and mirrors’ and operated entirely by humans. Let’s just have a look at some relevant examples to illustrate where I think some of these virtual assistants and chatbots are heading.

Example 1 – Nordstrom Chatbot and Operator offering personalised discovery:

US based Nordstrom recently launched its first chatbot for the 2016 holiday season. If you’re already on Facebook Messenger or Kik, Nordstrom’s virtual assistant is only a click away. Users who engage with Nordstrom’s bot will be asked a number of questions about who they’re shopping for. The bot will then respond with bespoke gift suggestions based on the user’s responses.

nordstrom-v1

Fig. 2 – Nordstrom Chatbot – Taken from: https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca#.l5e2i887r

You can get a similar experience using Operator, which is driven entirely by human experts who’ll provide you with personalised advice on what to buy (see Fig. 3 below).

screen-shot-2016-12-19-at-20-46-37

 

Fig. 3 – Operator’s experts providing tailored advice to its users – Taken from: https://www.operator.com/

Example 2 – KLM sharing flight information via Facebook Messenger:

KLM, the well known international airline, now enables customer to receive their flight documentation via Facebook Messenger. After booking a flight on KLM’s website, customers can choose to receive their booking confirmation, check-in details, boarding pass and flight status updates via Messenger. It’s built on a Messenger plug-in which customers only have to enable in order to receive ‘personalised’ messages from KLM (see Fig. 4 below).

screen-shot-2016-12-19-at-20-17-33

Fig. 4 – Screenshot of KLM’s Messenger app – Taken from: https://messenger.klm.com/

Example 3 – Telegram using buttons for discovery and shortcuts:

As much as it’s great to have a very simple ‘single purpose’ conversational user interface, there are messenger apps and virtual assistants out there that do offer user functionality that works better with buttons to click. A good example is the Telegram app, which has buttons for specific actions and shortcuts (see Fig. 5 below).

telegram-v1

Fig. 5   – Screenshot of the buttons in Telegram’s messenger app – Taken from: http://alistapart.com/article/all-talk-and-no-buttons-the-conversational-ui

Main learning point: I’ll no doubt learn more about conversational user interfaces over the coming months and years, but looking at simple examples like x.ai, Nordstrom’s Chatbot, Operator, Telegram and KLM’s Messenger feels like a very good starting point!

Related links for further learning:

  1. http://alistapart.com/article/all-talk-and-no-buttons-the-conversational-ui
  2. https://uxdesign.cc/10-links-to-get-started-with-conversational-ui-and-chatbots-3c0920ef4723#.yqpfdz5re
  3. https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca#.l5e2i887r
  4. http://www.geekwire.com/2016/new-nordstrom-mobile-chat-bot-ready-help-shoppers-find-perfect-holiday-gift/
  5. https://www.techinasia.com/talk/complete-beginners-guide-chatbots
  6. https://www.smashingmagazine.com/2016/07/conversational-interfaces-where-are-we-today-where-are-we-heading/
  7. http://www.theverge.com/2016/3/30/11331168/klm-facebook-messenger-boarding-pass-chat-integration
  8. https://messenger.klm.com/
  9. https://www.operator.com/

Varo Money and its focus on the banking – customer relationship

Varo Money is a US based Fintech startup that provides mobile banking and personal financial management services. We’ve seen mobile banks launching in various forms left right and centre over the last two years; think N26 in Germany, Simple in the US and Monzo in the UK, just to name a few. I’m keen to explore Varo more and learn more about its focus on personal financial management and building an ongoing relationship with its customers.

I listened to a podcast interview with Colin Walsh – CEO and Co-Founder of Varo – recently, in which he outlined as Varo’s core proposition and its main points of differentiation:

  • Next generation of consumers – In the interview, Colin explains how Varo sees the so-called generation of ‘millennials’ as a white space, currently not addressed well by existing banks. Varo aims to provide these target customers with an easy way to manage their accounts, but also focuses on providing them with financial guidance on how to manage their money.
  • Mobile first – Given that Varo targets ‘millennials’, Colin made a point of explaining that Varo’s customer experience needs to be intuitive and mobile first, since this has become the standard for millennial users. He describes this mobile first approach as a key differentiator for Varo, along with “delivering meaningful insights to customers.”
  • Relationship focus – Varo is all about “earning the relationship with the customer.” This means gathering customer data so that Varo can advise customers better and deepening the relationship with the customer by addressing their needs. This doesn’t make Varo any different to any other banks in my opinion, but it will be interesting to see how Varo will design an experience tailored to the needs of its customers. I liked Colin’s point about using data to enhance customer relationships, and I wonder how Varo will build this ‘customer understanding’ into its experience.
  • Goal-based – Similar to Qapital, Varo is all about helping its customers reach certain financial goals and outcomes. For example, if you want to save money for a big expenditure, Varo is looking to create an experience which will make it easier to set related goals and manage your money accordingly (see Fig. 1 below). I like how Varo enables users to have a single view of their money across a number of accounts (Fig. 2 below).
  • Underpinned by partnerships – Like many Fintech startups, Varo partners with a number of established third parties to provide the components of their platform. Varo is partnering with companies like Galileo (payment processing) and Socure (identity verification) who, as Colin explains, “things they do very well at scale” and will help with Varo’s speed to market. Varo configures these existing technologies in order to not have the reinvent the wheel. Instead, Varo wants to focus its efforts more on a human-centered approach to design and experience, providing customers with insights to help deepening relationships with them (see Fig. 3 and 4 below).

Main learning point: For a company that hasn’t even yet released its product into Beta, Varo has done a good job in creating a buzz around its proposition and its services. With so many new banking platforms popping up, it will be interesting to see how Varo will differentiate itself and establishes a critical mass of US customers and, as Colin says will become “a credible alternative to a traditional bank account.”

Fig. 1 – Screenshot of Varo’s goal-setting functionality – Taken from: http://www.varomoney.com/whatisvaro.php#2

varo-3

Fig. 2 – Screenshot of Varo’s ability to provide a single view of all their accounts – Taken from: http://www.varomoney.com/whatisvaro.php#2

varo-1

Fig. 2 – Screenshot of Varo’s card functionality – Taken from: http://www.varomoney.com/whatisvaro.php#2 

 

varo-2

Fig. 4 – Screenshot of “V”, providing insights to customers – Taken from: http://www.varomoney.com/whatisvaro.php#3

varo-4

Related links for further learning:

  1. https://soundcloud.com/wft
  2. https://blog.varomoney.com/2016/10/05/from-toasters-to-mobile-banking-moving-to-better-experiences-and-better-outcomes/#more-775
  3. http://www.wsj.com/articles/silicon-valley-looks-at-something-new-starting-a-bank-1462146047
  4. http://www.varomoney.com/data/Varo-Bancorp-Partnership-Announcement-2016.pdf
  5. https://www.sofi.com/
  6. https://www.prosper.com/
  7. https://www.crunchbase.com/organization/varo-money#/entity

Book review: Sprint (Part 6 – Day 5)

The fifth and final day of the sprint is all about interviewing your (target) customers and learning from how they interact with your prototype.

Interview

“Five is the magic number”, is the point that Knapp, Zeratsky and Kowitz are making in Sprint with regard to the number of people to interview. The value of this number of interviewees was proven by usability expert Norman Nielsen who found that typically 85 percent of problems were observed after just five people (see Fig. 1). “The number of findings quickly reaches the point diminishing returns,” Nielsen concluded. “There’s little additional benefit to running more than five people through the same study; ROI drops like a stone.”

When it comes to conducting the actual interview, having a structured and consistent way of running these conversations is critical. Knapp, Zeratsky and Kowitz write about the “Five-Act Interview”, which consists of the following stages (see Sprint, p. 202):

  1. A friendly welcome to start the interview
  2. A series of general, open-ended context questions about the interview
  3. Introduction to the prototype(s)
  4. Detailed tasks to get the customer reacting to the prototype
  5. A quick debrief to capture the customer’s overarching thoughts and impressions

The book also provides some useful tips for the interviewer, asking open-ended and ‘broken’ questions (pp. 212 – 215):

  • DON’T ask multiple choice or “yes/no” questions – “Would you …?””Do you …?””Is it…?”
  • DO ask “Five Ws and One H” questions – “Who …?””What …?””Where …?””When …?””Why …?””How …?”
  • Ask broken questions – The idea behind a broken question is to start asking a question – but let your speech trail off before you say anything that could bias or influence the answer. For example: “So, what … is …” (trail off into silence)

Fig. 1 – Why You Only Need To Test With Five Users – Taken from: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

 

20000319-user-testing-diminshing-returns-curve

Learn

Ultimately, this is what the fifth and final day of your sprint is all about: finding the end to your sprint story. Once you’ve had a chance to see how your customers react to your prototype, you’ll be able to answer your sprint questions and decide on next steps. For example, if you and your team take interview notes as a group during the five interviews, you should be able to do a good recap of all your learnings, answer the original sprint questions and decide on what to next. For example, a common next step would be to make a go/no go decision about a particular product idea.

Main learning point: In “Sprint”, Knapp, Zeratsky and Kowitz offer a very cost-efficient way to explore product questions and solutions before committing to an idea (and a large investment of time, money and effort). The reality is that as a product manager you’ll almost always will have to take a punt, but being disciplined about doing sprints and continuous discovery will help you make better informed decisions, based on real customer feedback.

Related links for further learning:

  1. https://marcabraham.wordpress.com/2015/06/24/interviewing-customers-to-explore-problems-and-solutions/
  2. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  3. https://marcabraham.wordpress.com/2014/04/20/how-to-do-effective-user-interviews/
  4. https://marcabraham.wordpress.com/2014/11/21/julia-shalet-explains-about-user-research-at-the-mobile-academy/
  5. https://marcabraham.wordpress.com/2015/10/19/collaborative-user-research-learning-from-erika-hall/