“Build it and they will come!” I used to work once with a senior executive, who was of the opinion that a product or feature should just be launched, without any testing with customers beforehand. “I know that once it’s out there, people will want it” she’d explain to me, adding that “it’s what people want”.

Hearing this “build it and they will come” mantra time and time again did annoy me 🙂 At the same time, it did make me wonder whether it might be a good idea to (continuously) release product features without prior customer discovery … What if this executive is right and any new product, feature or service should just be launched, as a way of learning as quickly as possible!?

Being able to ‘launch and learn’ is a vital tool in any product person’s toolkit. I strongly encourage you to avoid ‘one-off product releases’ at any time; what are you going to learn from shipping a product only to then move on to the next thing!? One can debate about when to best learn – should you learn pre-release? – but the main point is that you’ll need to ship early and often to learn continuously.

Basecamp, a project management software compare, does take ‘launch and learn’ to the extreme, they don’t show customers anything until every customer can see it. In the book “It doesn’t have to be crazy at work”, Basecamp’s co-founders Jason Fried and David Heinemeier Hansson describe how at Basecamp:

• “We don’t beta-test with customers.”
• “We don’t ask people what they’d pay for something.”
• “We do the best job we know how to do and then we launch it into the market.”
• “The market will tell us the truth.”

Fried and Heinemeier Hansson argue that anything you ask or test with customers prior to launch is hypothetical: “Real answers are uncovered when someone’s motivated enough to buy your product and use it in their own environment – and of their own volition. Anything else is simulated answers. Shipping real products gives you real answers.” Whilst I do agree with this line of thinking, I don’t believe in simply launching some crappy product or feature and see if it sticks (just as much I don’t believe in “build it and they will come”).

My suggestion would to ‘launch confidently and learn’. This means that for each new product or feature you determine – based on your confidence level – whether it needs some form of customer research before launch:

1. Deliver value in order to learn – You want to be smart about the things you want to learn. The best opportunity to learn comes when you’re confident about the value that you’re delivering to the customer. Naturally, people might not buy or use your product despite the value it intends to deliver, but that’s a learning in itself.
2. Minimum Level of Confidence (1) – How confident are you? What exactly are you confident about (and why)? The main reason why I believe in product managers adhering to a confidence treshold is to avoid launching products that don’t work or provide an awful user experience. The Newton MessagePad which came out in 1993 is a good example of the launch of an incomplete product, which didn’t live up to its promise. Larry Tesler, senior exec at Apple at the time of the Newton MessagePad, described Apple’s promise about the Newton’s handwriting capability as a large nail in the Newton coffin. The lesson learned here is that you shouldn’t launch when you’re not confident about the capability and value of your product or feature.
3. Minimum Level of Confidence (2) – I’ve come up with a number of basic questions and criteria to apply when you’re thinking of launching a product (see Fig. 1-2 below). In my experience, identifying your Minimum Level of Confidence shouldn’t result in ‘analysis paralysis’. In contrast, it’s an important conversation to have throughout the product lifecycle to ensure that everyone fully understands what risks or unknowns are associated with the upcoming release. As an outcome of such a conversation you can decide whether to get customer feedback pre-release.
4. Make sure you learn! – Whether you do or don’t engage with customers before launch, being clear about what you’re looking to learn from a release is paramount. Like I mentioned above, I view releasing something without learning from it  as a cardinal sin. It’s very important to continuously learn from real users and actual usage (or not) about your key hypotheses. These learnings – both quantitive and qualitative – will give you the data points to iterate or terminate a product.

Fig. 1 – Questions and criteria to check your confidence about launching a product or feature:

• Internal quality assurance – Have you tested your product feature to ensure there are no obvious bugs or gaps in the user experience? Even if you don’t test with customers prior to launch, you should test some key acceptance scenarios internally before launch to make sure the product works as intended.
• Does the feature or product touch on core user experience? – If “yes” is the answer to this, then I recommend you do test with customers prior to launch to identify any major usability issues worth solving before launch. You typically need to test with no more than five customers to unearth any critical usability issues.
• How confident are you? – The combination of low confidence in something which your business has got a lot riding can be deadly. Yes, one can always try to do damage limitation, but it might already be too late at the time of you trying to repair things! The idea behind determining your confidence levels upfront isn’t a scientific one. Instead, it enables a conversation, making sure that people have got their eyes wide open and understand the level of risk and unknowns involved in an upcoming product launch (see Fig. 2 below).

Fig. 2 – Basic confidence levels to consider before launch:

High Confidence: Our confidence in the upcoming release is high because we tested it thoroughly internally, have launched a similar product or feature before or if there’s an issue the fallout will be small.

Low Confidence: Our confidence in the upcoming release is low because we haven’t fully tested it, it’s based on new technology or creates a totally new user experience.

Main learning point: Even if you decide not to generate customer learnings before a product launch, make sure you at learn after launch. Launch and learn. Don’t launch without learning!

Related links for further learning:

1. https://www.mindtheproduct.com/2017/02/the-life-of-a-product-manager-learning-by-doing/
2. https://www.intercom.com/blog/shipping-is-your-companys-heartbeat/
3. https://medium.com/@joshelman/a-product-managers-job-63c09a43d0ec
4. https://uxplanet.org/10-things-i-learned-from-jason-fried-about-building-products-5b6694ff02aa
5. http://time.com/13549/the-10-worst-product-fails-of-all-time/
6. https://twitter.com/jasonfried/status/935555293014036480
7. https://247wallst.com/special-report/2014/03/03/worst-product-flops-of-all-time/2/
8. https://www.macworld.com/article/2047342/remembering-the-newton-messagepad-20-years-later.html
9. https://www.nytimes.com/1993/09/26/business/the-executive-computer-so-far-the-newton-experience-is-less-than-fulfilling.html