I recently learned more about the distinction between ‘problem interviews’ and ‘solution interviews’, when interacting with your customers. The problem interview is Ash Maurya’s term for the interview you conduct to validate whether or not there’s a real problem that our target audience has. In contrast, the goal of a solution interview is to find out what features you need to build in order to solve the customer problem(s).
In the problem interview, you want to find out 3 things:
- Problem – What problem are you solving? For example, what are the common frustrations felt by your customers and why? How do their problems rank? Ask your customers to create a top 3 of their problems (see the problem interview script in Fig. 1 below).
- Existing alternatives – What existing alternatives are out there and how does your customer perceive your competition and their differentiators? How do your customers solve their problems today?
- Customer segments – Who has these problems and why? Is this a viable customer segment?
Reading Javier Sandoval’s post titled ‘You’ve Been Doing Your Customer Problem Interviews Wrong’ was a great a reminder of some common mistakes that can easily be made when conducting problem interviews (see Fig. 2 below). For example, I know from experience how easy it can be to loose sight of learning and focusing purely on validating assumptions instead.
In the solution interview, you want to find out 3 things:
- Early adopters – Who has this problem and why? How do we identify and engage with early adopters? (see Fig. 3 below)
- Solution – How will you solve their problems? What features do you need to build as part of your solution, why?
- Pricing/Revenue – What is the pricing model for your product or service? Will customers pay for it, why?
With solution interviews, I’ve learned over the years to listen a lot when it comes to exploring solutions, as you don’t want to guide your customers towards a solution that you’ve got in your mind or have a prototype for. You might want to have a quick look at this great article titled ‘3 common mistakes during user interviews’ or my earlier blog post about effective user interviews to find out more common mistakes.
Main learning point: The distinction between problem and solution interviews is a very helpful one. Whether you do one or the other (or combine the two) thinking upfront about the nature and outcomes of your interviews will help ensure you ask the right questions, listen and avoid common interview ‘pitfalls.’
Fig. 1 – Outline of a problem interview script – Taken from: Ash Maurya – Running Lean
Fig. 2 – Common mistakes when conducting problem interviews – Taken from: http://blog.javierasandoval.com/youve-been-doing-your-customer-problem-interviews-wrong
- Only interviewing. Ask to observe them working. ASK TO DO THEIR JOB. Then do it. Use their products. Go to their events. Read what they read. BECOME YOUR CUSTOMER.
- Going in with the mentality of validating, not learning. “But isn’t the point to ‘validate our assumptions,’ Javier?” Yes, but all too many times I’ve seen entrepreneurs act like lawyers, and not reporters. Instead of only looking for evidence that validates their hypotheses, founders must also try to disprove their assumptions. An investigative reporter builds up both sides, and focuses on learning. A lawyer always thinks they’re right and only appreciates what supports that dogma.
- Interviewing multiple people at once, which produces group think and creates an environment where customers can’t be honest nor personal. If you’re building a product for couples, interview each lover separately. If the whole company wants to talk to you in a meeting atmosphere, say no. No to focus groups!
- Focusing too narrowly in the beginning. Cast a wide net with customer segments. Interview each potential segment at least 5 times, then get rid of the least responsive, and do another set of five interviews with each segment and lose the least responsive, then repeat until you’ve identified the most viable target segment.
- Recording the feedback and leaving out the customer’s habits. Don’t ask customers what they want. Measure what they do. If they say, “Oh, yeah, I do that a lot.” Ask, “Can you be more specific? Around how many times? When most often?” Remember, you have to make what they’re ALREADY DOING, easier. If they’re not doing it often, they must not care.
- Not asking How, When, Why enough. You have to investigate problems to their elements. If they say that they don’t like their current solutions, why? “What about it? Can you describe the last time it bothered you specifically? How often do you use the solution? What do you like about it? Why?” Then Why? again. Strip the problem to its job-to-be-done.
- Leaving out the product ecosystem. A BVL team seeks to solve the problem of people overhearing your phone conversation. They need to learn about what other phone products people buy, not just direct competitors. What attachments do they own? If they own none, why? Where did they acquire their phone case? Why that one? How do they carry their phones? They need to learn all of this in order to build a product that’s native to its ecosystem.
- Only interviewing direct users. Your customers customers are your customers. Let that sink in. Yes, Draft, a product that helps writers incorporate edits easily, makes money off writers, but the founder needs to learn the perspective of the people with whom those writers share their work. What problems do they encounter when sharing edits? What does the problem look like from their perspectives? If they hate editing articles with Draft, writers won’t use it!
- Asking customers what they want. Customers are THE EXPERTS about the problem, but not the solution. That’s the entrepreneur’s job. You go to them to learn about the problem, but it’s up to you to come up with the solution. Abstain from growing the feature list; learn which features mean the most then shorten the feature list, don’t build it.
- Shying from strong calls-to-action. If customers say they want this product asap, ask for an advanced payment and provide a money-back guarantee.
- Asking, “What solutions do you use to solve this problem?” because clear solutions may not exist. Customers may be working around the problem without realizing it: people emailed themselves files before Dropbox. In place, ask, “How do you go about solving the problem.” What they reply may not be a traditional competitor, but if that’s how they’re solving the problem, you better damn well pay attention. Think of email and Evernote. Although email doesn’t seem like a direct competitor to Evernote, a lot of people create drafts to quickly jot something down.
- Asking specific questions. You don’t know what the right questions are to ask, so let them talk! You just listen and ask Why and When and How and How again and Why again. WHY and HOW should be 80% of questions. Ask about their workflow and listen. If it’s important, they’ll mention it.This is the same reason why I hate surveys.
- Skipping the big hypothetical questions: “If you could have a magic wand and change anything, what would it be?” Sounds silly, but customers’ large scale answers tell you what they prioritize “What’s the biggest pain in this situation?” “How much does this problem cost you (in terms of lost revenue, customers, time, frustration)?” “What should I read to learn about this industry? Whom should I know?”
- Learning only about the problem. You need to validate segments and channels, too. Learn about your customers. Not only demographic data, but personal information. Do they believe people similar to them have the same problem? Why? Who do they believe has the problem most? Then the channels: how do they hear about similar products? What industry blogs or sites do they learn from? Where do they go to try to solve similar problems?
- Failing to discover true customers. In businesses and organizations, the person using the product may not be the one who pays for it. Interview the decision maker. How do they decide? How do they learn about products? How do they learn if something’s working or not working if they’re not the one’s using the product? How do the actual users convince them to get new products? Also, influencers may exist in addition to users and purchasers. How can they influence the decision makers? What do they think of the problem?
- Forgetting to ask for a follow-up or a referral. Just stupid.
- Emailing and asking for interviews incorrectly.
- Not building personal relationships. Build a feedback loop so you create evangelists. You’ll need the feedback of these customers later with demos, prototypes, and MVPs, but you also want them rallying behind you. That’s why you meet in person, that’s why you pay for coffee. Email them out of the blue to see how they’re doing. Personally send them interesting articles or helpful links. Follow them on Twitter or Facebook and chat them up.
Fig. 3 – Outline of a solution interview script – Taken from: Ash Maurya – Running Lean
Related links for further learning:
I do User Story Mapping. I do Impact Mapping. And I’ve recently added another forming of ‘mapping’ to my arsenal: “Experience Mapping.” I learned about Experience Mapping through this very helpful guide by Adaptive Path. An Experience Map is useful tool for capturing and presenting key insights that occur across user experiences with a particular product or service. Similar to user story mapping, an experience map helps to gain a better understanding of ‘how’ and ‘why’ people interact with your product or service.
Adaptive Path’s Guide to Experience Mapping contains some useful definitions (see Fig. 1 below) as well as the four key steps involved in creating an experience map (see also Fig. 2 below):
- Step 1: Uncover the truth – This first step in creating an Experience Map is to study customer behaviour and interaction across channels and touchpoints. Typically this comes down to making sense of a whole heap of data – both quantitative and qualitative (see Fig. 3 below). For instance, if you’re looking to understand how customers engage with your mobile app to purchase an item, it’s worthwhile to look at all the existing data you can gather, varying from analytics data to user interviews or complaints. Don’t expect your initial data to be very organised, but you should be able to start spotting themes and patterns quickly. Keep in mind the following perspectives: what’s the customer doing, feeling and thinking.
- Step 2: Chart the course – When outlining the Experience Map, the key thing is to make it a collaborative experience by getting all the relevant ‘stakeholders’ to go over the experience map and the research that you’ve done. The main components of the experience map framework are: the lens, the customer journey model, and the takeaways (see Fig. 4 below)and I’ve included some great examples of effective Experience Maps (see Fig. 5 -7 below). Running an Experience Mapping Workshop is a great way to get people involved in charting the course and the guide provides some great recommendations on how to best facilitate an Experience Mapping Workshop (see Fig. 8 below).
- Step 3: Tell the story – Adaptive Path stresses the importance of deciding early on what is and what isn’t going to be involved in the story that you’re looking to tell.This means separating important insights from nice-to-have details, while identifying the relative priorities among your building blocks. Take a moment to evaluate your work and identify the key components of the story your map will tell. Adaptive Path provides some useful suggestions for how to best reach the end of the user journey successfully: (1) have a point of view, (2) consider your audience and (3) design for impact (see Fig. 9 below). Sketching your story is good way of visualising and fleshing out your story.
- Step 4: Use your map – Having a nice Experience Map is no good if you don’t share it widely across your organisation and use it as an ongoing reference point for your product development process and product decisions. Adaptive Path provides some great pointers for ‘next steps’ once you’ve a got an Experience Map in place. For example, you can use your Experience Map as a tool to assess product opportunities or making product decisions (see Fig. 10 below).
Main learning point: I learned an awful lot from Adaptive Path’s approach to Experience Mapping and creating such maps is a great way of capturing insights and interactions across the user journey.
Fig. 1 – Key definitions related to experience maps – Taken from: http://mappingexperiences.com/
- Channel – A medium of interaction with customers or users. Print, the web, mobile, voice calls, and brick and mortar locations are all common channels for reaching out to and interacting with customers. A channel defines the opportunities or constraints of a touchpoint.
- Touchpoint – A point of interaction between a person and any agent or artefact of an organisation.These interactions take place at a certain point in time, in a certain context, and with the intention of meeting a specific customer need. Create a shared frame of reference around the customer experience. Build organisational knowledge of customer behaviours and needs across channels.
Fig. 2 – A graphic overview of the key four steps involved in experience mapping – Taken from: http://mappingexperiences.com/
Fig. 3 – The building blocks of experience mapping – Taken from: http://mappingexperiences.com/
Fig. 4 -The main components of the experience map framework – Taken from: http://mappingexperiences.com/
- The lens – The lens is an overriding filter through which you view the journey, such as a persona, more general experience principles, or a value proposition.
- The customer journey model – The customer journey model depicts the range of interactions customers have across channels, touchpoints, time, and space in pursuit of satisfying one or more needs.
- The takeaways – The takeaways summarise key findings from the experience mapping process. The moment you conceive of a plan to map the customer journey, you need to chart a course to actionable results. The takeaways signal which way you are recommending the organisation head next. Your takeaways could include: → Strategic insights → Recommendations → Design principles. Takeaways are typically added to the map late in the process, once you have begun to pivot from understanding the current state of your customer experience to envisioning the future state. There are different takeaways you could include, but they should answer the questions “So what?” and “What now?”
Fig. 5 – Example of an Experience Map – Taken from: http://www.adaptivepath.com/ideas/the-anatomy-of-an-experience-map/
Fig. 6 – Lego Experience Map – Taken from: https://uk.pinterest.com/uxjam/journeys/
Fig. 7 – PeopleSmart Experience Map – Taken from: http://www.lorettaneal.com/peoplesmart.html
Fig. 8 – Recommended steps for facilitating an Experience Mapping Workshop – Taken from: http://mappingexperiences.com/
- Set the context – Prepare a short presentation to catch everyone up on your discovery and research work.
- Organise yourselves – Divide participants into teams of four to six. Make sure each team has a balance of different roles and functions.
- Deconstruct – Each team will need to go through the research notes and pull out the building blocks.
- Stage – As the sticky notes build up, have one person from each team move them to the butcher paper, starting with Doing.
- Construct – From this point on, the team should start to group duplicate stickies and begin finding relationships among them.
- Shape – By the end of the session, each group should be moving from figuring out the customer journey to arranging the key insights into a story.
- Workshop Supply List → Butcher paper → Painter’s tape → Black sharpies → Sticky notes (5 or more colours) → A camera
- Facilitation Tips → Keep groups to six people or fewer → Create handouts with clear instructions → Provide copies of research notes → Remember to take breaks → Share out across groups → Take lots of pictures.
Fig. 9 – Suggestions to hep turn an Experience Map into a compelling story – Taken from: http://mappingexperiences.com/
- Have a point of view – Can you summarizse the key points you want someone to walk away with after viewing the map? What story do you want them to tell to other people?
- Consider your audience – What kind of details will help them best understand the story? Which insights are essential for them to make good strategic and design decisions?
- Design for impact – What immediate next steps do you want your map to initiate? What other uses of the map are you hoping to encourage in the short-, mid-, and long-term? Your goal is to craft a communication piece that can stand on its own, inspire new ideas, and have longevity.
Fig. 10 – Pointers for ‘next steps’ once you’ve a got an Experience Map in place – Taken from: http://mappingexperiences.com/
- Issue / Opportunity Identification and Prioritisation – Using the structure provided by your map, chronicle issues or opportunities for addressing customer pain points at each stage of the customer journey. Prioritise according to business and customer value. This method helps you quickly work with stakeholders to identify high-value areas of opportunity.
- Experience Storyboards – Using your map and simple storyboard templates, along with additional tools like personas or experience principles, use rapid ideation to generate stories of future experiences. This approach provides stakeholders with a forum for ideas grounded in the insights of your customer journey.
- Future Experience Mapping – Using the map as a reference, define the ideal customer journey through mapping out what customers would ideally do, think, and feel as they interact with touchpoints on the way to satisfying their needs. This method encourages cross-functional collaboration to define cross-channel experience principles.
Related links for further learning:
Any self respecting developer or product person will be doing team retrospectives on a regular basis. These retrospectives provide a great opportunity to reflect on past releases, build team dynamics and identify any areas which could be improved.
However, I’ve found product retrospectives to be a lot less common. Let me first explain what I mean by “product retrospectives”: a regular opportunity for a team to reflect on the product or service. This is explicitly not about the team or process improvements, it’s all about the product:
- How is our product performing (or not)? – I appreciate that many of us will monitor product performance on a daily or even hourly basis. However, there might people in your business who don’t keep track of recent product releases or performance. They might also not be au fait with the assumptions – business and user – which underpin a particular product feature or improvement.
- How are we doing against our roadmap and business objectives? – This is an opportunity to look at the bigger picture and review progress against key roadmap milestones and success factors. What have we been doing (and why)? What’s coming up and what’s the impact that we’re expecting?
- What are our users saying? – If you’re getting user feedback on a regular basis, I find product retrospectives a great time to share user learnings and insights. This can be as simple as sharing excerpts or quotes from user feedback sessions or a video of users testing your product remotely .
- What’s happening in the market? – To avoid looking at a product or proposition in isolation, I always try to look at competitors as part of a product retrospective. What are our competitors doing? Why? Similarly, I also like broadening perspectives by learning from businesses in different sectors or markets.
- What product ideas do you have? – Facilitating a product retrospective isn’t a one way street. It’s also an opportunity for the people in the session to provide their suggestions based on the data and learnings that you discuss with them.
As with regular retrospectives, this is how I typically structure my product retrospectives:
- Set the stage – I’ll outline the goal for the retrospective, identifying both the desired outcomes of the session as well as what I expect to cover. For example, I will explain how we will use the product retrospective to further shape the product roadmap and that some concrete product objectives are a desired retrospective outcome.
- Gather data – To create a shared picture of what’s happened or is happening, I’ll start of by presenting hard facts around releases, roadmap progress, business objectives and key results. I’ll use these facts as a starting point for discussion, facilitating a conversation around data, underlying assumptions and problems to address.
- Generate insights – This part of the product retrospective is about asking “why” and thinking about what could be done differently. At this stage, I’ll talk about qualitative user feedback and we’ll discuss the “why” behind some of our insights.
- Decide what to do – At this point in the product retrospective, the audience is likely to have some product ideas or suggestions worthy of further exploration. From experience I’ve learned that the format of a product retrospective doesn’t lend itself particularly well to a lengthy discussion about the merits of a product idea. However, it’s a great opportunity to gather thoughts and to prioritise ideas for further discovery.
- Close the retrospective – This is where I tend to do a quick recap of the main insights and actions discussed during the retrospective. I’ll also ask if there are any specific items that the audience would like to see covered in the next product retrospective.
Main learning point: Hopefully you can see how easy it is to to do a product retrospective and the benefits you’d get from doing this on a regular basis. I’ve done product retrospectives with single teams (e.g. development teams or marketing teams) but have also run them for ‘multi-disciplinary audiences’ (i.e. members from different teams in one room). In both cases, you’ll be able to create a shared understanding of product performance and get valuable feedback to help move your product forward.