I recently learned more about the distinction between ‘problem interviews’ and ‘solution interviews’, when interacting with your customers. The problem interview is Ash Maurya’s term for the interview you conduct to validate whether or not there’s a real problem that our target audience has. In contrast, the goal of a solution interview is to find out what features you need to build in order to solve the customer problem(s).

In the problem interview, you want to find out 3 things:

• Problem – What problem are you solving? For example, what are the common frustrations felt by your customers and why? How do their problems rank? Ask your customers to create a top 3 of their problems (see the problem interview script in Fig. 1 below).
• Existing alternatives – What existing alternatives are out there and how does your customer perceive your competition and their differentiators? How do your customers solve their problems today?
• Customer segments – Who has these problems and why? Is this a viable customer segment?

Reading Javier Sandoval’s post titled ‘You’ve Been Doing Your Customer Problem Interviews Wrong’ was a great a reminder of some common mistakes that can easily be made when conducting problem interviews (see Fig. 2 below). For example, I know from experience how easy it can be to loose sight of learning and focusing purely on validating assumptions instead.

In the solution interview, you want to find out 3 things:

• Early adopters – Who has this problem and why? How do we identify and engage with early adopters? (see Fig. 3 below)
• Solution – How will you solve their problems? What features do you need to build as part of your solution, why?
• Pricing/Revenue – What is the pricing model for your product or service? Will customers pay for it, why?

With solution interviews, I’ve learned over the years to listen a lot when it comes to exploring solutions, as you don’t want to guide your customers towards a solution that you’ve got in your mind or have a prototype for. You might want to have a quick look at this great article titled ‘3 common mistakes during user interviews’ or my earlier blog post about effective user interviews to find out more common mistakes.

Main learning point: The distinction between problem and solution interviews is a very helpful one. Whether you do one or the other (or combine the two) thinking upfront about the nature and outcomes of your interviews will help ensure you ask the right questions, listen and avoid common interview ‘pitfalls.’

Fig. 1 – Outline of a problem interview script

Fig. 2 – Common mistakes when conducting problem interviews – Taken from: http://blog.javierasandoval.com/youve-been-doing-your-customer-problem-interviews-wrong

1. Only interviewing. Ask to observe them working. ASK TO DO THEIR JOB. Then do it. Use their products. Go to their events. Read what they read. BECOME YOUR CUSTOMER.
2. Going in with the mentality of validating, not learning. “But isn’t the point to ‘validate our assumptions,’ Javier?” Yes, but all too many times I’ve seen entrepreneurs act like lawyers, and not reporters. Instead of only looking for evidence that validates their hypotheses, founders must also try to disprove their assumptions. An investigative reporter builds up both sides, and focuses on learning. A lawyer always thinks they’re right and only appreciates what supports that dogma.
3. Interviewing multiple people at once, which produces group think and creates an environment where customers can’t be honest nor personal. If you’re building a product for couples, interview each lover separately. If the whole company wants to talk to you in a meeting atmosphere, say no. No to focus groups!
4. Focusing too narrowly in the beginning. Cast a wide net with customer segments. Interview each potential segment at least 5 times, then get rid of the least responsive, and do another set of five interviews with each segment and loose the least responsive, then repeat until you’ve identified the most viable target segment.
5. Recording the feedback and leaving out the customer’s habits. Don’t ask customers what they want. Measure what they do. If they say, “Oh, yeah, I do that a lot.” Ask, “Can you be more specific? Around how many times? When most often?” Remember, you have to make what they’re ALREADY DOING, easier. If they’re not doing it often, they must not care.
6. Not asking How, When, Why enough. You have to investigate problems to their elements. If they say that they don’t like their current solutions, why? “What about it? Can you describe the last time it bothered you specifically? How often do you use the solution? What do you like about it? Why?” Then Why? again. Strip the problem to its job-to-be-done.
7. Leaving out the product ecosystem. A BVL team seeks to solve the problem of people overhearing your phone conversation. They need to learn about what other phone products people buy, not just direct competitors. What attachments do they own? If they own none, why? Where did they acquire their phone case? Why that one? How do they carry their phones? They need to learn all of this in order to build a product that’s native to its ecosystem.
8. Only interviewing direct users. Your customers customers are your customers. Let that sink in. Yes, Draft, a product that helps writers incorporate edits easily, makes money off writers, but the founder needs to learn the perspective of the people with whom those writers share their work. What problems do they encounter when sharing edits? What does the problem look like from their perspectives? If they hate editing articles with Draft, writers won’t use it!
9. Asking customers what they want. Customers are THE EXPERTS about the problem, but not the solution. That’s the entrepreneur’s job. You go to them to learn about the problem, but it’s up to you to come up with the solution. Abstain from growing the feature list; learn which features mean the most then shorten the feature list, don’t build it.
10. Shying from strong calls-to-action. If customers say they want this product asap, ask for an advanced payment and provide a money-back guarantee.
11. Asking, “What solutions do you use to solve this problem?” because clear solutions may not exist. Customers may be working around the problem without realising it: people emailed themselves files before Dropbox. In place, ask, “How do you go about solving the problem.” What they reply may not be a traditional competitor, but if that’s how they’re solving the problem, you better damn well pay attention. Think of email and Evernote. Although email doesn’t seem like a direct competitor to Evernote, a lot of people create drafts to quickly jot something down.
12. Asking specific questions. You don’t know what the right questions are to ask, so let them talk! You just listen and ask Why and When and How and How again and Why again. WHY and HOW should be 80% of questions. Ask about their workflow and listen. If it’s important, they’ll mention it.This is the same reason why I hate surveys.
13. Skipping the big hypothetical questions: “If you could have a magic wand and change anything, what would it be?” Sounds silly, but customers’ large scale answers tell you what they prioritise “What’s the biggest pain in this situation?” “How much does this problem cost you (in terms of lost revenue, customers, time, frustration)?” “What should I read to learn about this industry? Whom should I know?”
14. Learning only about the problem. You need to validate segments and channels, too. Learn about your customers. Not only demographic data, but personal information. Do they believe people similar to them have the same problem? Why? Who do they believe has the problem most? Then the channels: how do they hear about similar products? What industry blogs or sites do they learn from? Where do they go to try to solve similar problems?
15. Failing to discover true customers. In businesses and organisations, the person using the product may not be the one who pays for it. Interview the decision maker. How do they decide? How do they learn about products? How do they learn if something’s working or not working if they’re not the one’s using the product? How do the actual users convince them to get new products? Also, influencers may exist in addition to users and purchasers. How can they influence the decision makers? What do they think of the problem?
16. Forgetting to ask for a follow-up or a referral. Just stupid.
17. Emailing and asking for interviews incorrectly.
18. Not building personal relationships. Build a feedback loop so you create evangelists. You’ll need the feedback of these customers later with demos, prototypes, and MVPs, but you also want them rallying behind you. That’s why you meet in person, that’s why you pay for coffee. Email them out of the blue to see how they’re doing. Personally send them interesting articles or helpful links. Follow them on Twitter or Facebook and chat them up.

Fig. 3 – Outline of a solution interview script – Taken from: Ash Maurya – Running Lean

Related links for further learning:

1. http://leanstack.com/customer-development-getting-started/
2. http://mikezornek.com/2013/03/22/running-lean-problem-interviews/
3. http://startitup.co/guides/285/problem-interview-script
4. http://startitup.co/guides/978/solution-interview-script
5. http://blog.javierasandoval.com/youve-been-doing-your-customer-problem-interviews-wrong
6. https://medium.com/@DaisyHappy2014/3-common-mistakes-to-avoid-during-user-interviews-4f3aa97347f