You might have read my one of my previous blog posts about the so-called goal oriented roadmap. I prefer goal-oriented roadmaps over their more traditional counterparts. My problem is that ‘classic’ roadmaps contain a mix of features and timings, but don’t provide any context whatsoever (I’ve included an example in Fig. 1 below). I typically don’t include features on a roadmap and focus on user or business problems instead. As such, the roadmap becomes a tool for ‘working backwards’ – enabling product teams to explore solutions for given problems or desired results.
I recently learnt from Jared Spool and Bruce McCarthy about adding “themes” to product roadmaps. McCarthy told Spool about the concept of adding themes, and Spool became an instant fan. Like me, McCarthy isn’t a fan of having features on a roadmap. Instead, he suggests adding “themes” to the roadmap and this is why:
- Open to ‘options’ – McCarthy believes that by having specific features on a roadmap, you run the risk of sending a product team down a certain avenue (and closing off any potential side streets). For example, if you were to just put “data dashboard” on a roadmap, chances are you’ll end up with a data dashboard. However, what would happen if you were to put “provide our customers with data to make key decisions” or “no data access” on the roadmap instead? By focusing solely on solution you run the risk of falling victim to what Josh Wexler calls solution sickness. Solution sickness is all about fixating on a solution and ignore any alternative ways of solving a problem.
- Customer focus – Jared Spool makes a great point when he says “When companies talk about features, they are saying, “Look at us. Look at what we can do.” He goes on to explain that “When companies talk about the problems of customers, they are saying, “Look at what you’re dealing with. Look at how we want to help.” It’s very easy to get fixated on features and forget about the underlying problems you’re looking to solve.
- Helping trade-off decisions – One of the things I love about both the goal-oriented and ‘theme’ approach to roadmapping is that it forces you to consider ‘why’ you want to develop certain products or services. What problem are we looking to solve and why? Why do we feel this problem is worth solving? Why should we prioritise this problem over others? Both the goal oriented and the theme roadmap approaches help make customers and their problems the core of everything you do as a product person. When I use the goal oriented roadmap, I always take out the “feature” layer purely because I don’t want us to be pinned down to one solution.
- “Marketing the story of solutions” – Which do you think would be an easier story to sell to customers? Option 1: “You can now use this real-time data dashboard which you can access via your content management system, enabling you to filter the data and extract reports” or Option 2: “You can use our data to decide on whether now is good time to sell your house?” … Easy, isn’t it!? As Jared Spool points out, the ‘theme’ approach makes it much easier for marketing teams to tell a story about a product or service: “Here are the problems we set out to solve and here’s how we solved them.”
- A more cohesive design – Design is another area which is positively impacted by a shift in focus from solutions to problems. From experience, I know how much easier it is to prioritise against a problem that you’re looking to solve, instead of trying to cram in a lot of features which you think might have an impact. As Spool puts it: ” Without a commitment to specific solutions, the team has flexibility.” As a result, product teams stand a better chance of creating simpler, well designed products.
Fig. 1 – Example of a traditional roadmap – Taken from: https://roadmunk.com/
Fig. 2 – Goal oriented roadmap example – Taken from: http://www.romanpichler.com/blog/agile-product-roadmap/
Related links for further learning:
“What Product Managers Need To Know About Rapid Prototyping” by Mike Fishbein and Josh Wexler is a great ebook for any product manager keen on prototyping before building the actual product. Mike Fishbein is a content manager with Alpha UX and Josh Wexler is a solutions director at Originate. In this book, they’ve collated their learnings and tips with regard to rapid prototyping.
I’m currently learning how to create prototypes, using tools like Balsamiq, Illustrator, InVision and Sketch. Reading “What Product Managers Need To Know About Rapid Prototyping” felt very timely. These are the main things I learnt from it:
- Why do we need a prototype!? – Fishbein and Wexler point out very early on in the book that experiments are critical to alleviate risks inherent in new product or feature development. Prototyping is at the core of experimentation; it provides a quick way to validate your assumptions with real users and tackling key product risks early on in the process. It can be as simple as using a piece of paper with a sketch of a new feature for initial product validation. The main thing is that you use a prototype to mitigate risk by getting feedback early and often in the product (development) lifecycle.
- Why do product managers need to know about prototyping? – Building on the importance of experimentation, Fishbein and Wexler argue that “the product manager’s primary role becomes enabling experimentation as a seamless capability. At the core of experimentation is rapid prototyping.”
- Generative vs evaluative experiments – The distinction between generative and evaluative experiments is an important one. Typically, new products or features will start with “generative” experiments, where it’s all about identifying and assessing customer pain points or problems. In contrast, “evaluative” experiments are used to evaluate different options for a product direction. You’re evaluating a number of ways to resolve a customer pain point that you’ve established during a generative experiment. Rapid prototyping is critical during evaluative experiments.
- Consider constraints when prototyping – Using the words “prototyping” and “constraints” in one sentence might sound like a contradiction in terms, as you might think that prototype means having a free rein. In contrast, it’s important to pay attention to constraints when creating prototypes. The book suggests a number of important questions to ask before creating a prototype (see Fig. 1 below).
- Use a Learning Model, plotting ‘perception of value’ vs ‘simulation of value’ – Fishbein and Wexler describe the perception of value (‘PV’) as “a conglomeration of of metrics that together provide insight into comparative user feedback.” In other words, the higher the PV the better. Simulation of value (‘SV’) shows how close the user is to actually experiencing value. Going back to my earlier example of a simple sketch on a piece of paper; this might generate a high PV, because the user gets the value, but the SV is likely to be very low in comparison. It’s fair to say that higher SV values will generate more reliable user learnings and insights, as the prototype delivers more concrete value to the user.
- Plotting goals and constraints – It was really interesting to learn about plotting goals and constraints when thinking about creating a prototype. A good example of a constraint is “risk-threshold”, which represents the minimum amount of data you need to collect before your business gives the green light for a product. The ‘goal’ element covers the ‘why’ and perceived value of a product.
- Don’t forget about the ‘story’ – In addition to perception and simulation, Fishbein and Wexler also suggest looking at the ‘story’. The story dimension represents a continuum of the user’s narrative and how they might interact with the value proposition of a product. The book talks about about a so-called ‘double dip effect’, This effect typically occurs when product managers succeed in unlocking the ‘story’ element; when iteration cycles are so short that, following positive experiment results, product teams can run another experiment to understand the “why” behind the initial results. The key point here is that when prototyping becomes a rapid process, the product manager will be able to fully learn the narrative of her (target) users.
- Balancing simulation of value with iteration time and resources – Fishbein and Wexler also address some of the tradeoffs inherent in rapid prototyping. They point out that “prototyping lower simulations of value costs less and has shorter iteration cycles than prototyping higher simulations of value.” However, the insights you typically get from lower simulations of value are likely to be less reliable and detailed compared to high simulations of value. I’ve included Fishbein and Wexler’s pointers on how to manage these tradeoffs in Fig. 2 below.
Main learning point: I recommend “What PMs Need To Know About Rapid Prototyping” to any product person who wants to use prototyping as a way to learn quickly and often. Fishbein and Wexler have written a very comprehensive book which provides product managers with a great overview of dos and don’ts with regard to rapid prototyping. This book has definitely helped me in thinking about prototypes in a much more structured way!
Fig. 1 – Sample questions and constraints to consider before creating a prototype – Taken from: Mike Fishbein and Josh Wexler – What Product Managers Need To Know About Rapid Prototyping, pp. 19 – 21
- What’s my or my organisation’s risk or treshold?
- What’s my capability given my current budget?
- What’s my organisation’s domain of discretion?
- Am I over-constraining?
Fig. 2 – Balancing simulation of value with iteration time and resources – Taken from: Mike Fishbein and Josh Wexler – What Product Managers Need To Know About Rapid Prototyping, pp. 54 – 55
- Stay in scope – It’s not about creating beautiful designs for your prototype, the goal is to learn about the user and their interactions with a product’s value proposition as early and often as possible. Staying in scope is therefore a key notion.
- Adapt the methodologies – No (prototyping) methodology is set in stone. Instead, Fishbein and Wexler recommend understanding the high-level purpose of each prototyping step and then adapting it to your organisation.
- Select technology that supports your best practices – When it comes prototyping, there’s a thousand and one tools to choose from. The important part is to make sure that prototyping is an efficient and waste-reducing mechanism for learning about users and validating new product concepts.
Related links for further learning:
It was interesting to read a very detailed breakdown of how Spotify build products by Henrik Kniberg. Kniberg is a Swedish Agile expert who consults with a lot of different companies on how to best implement Agile and Lean practices. Kniberg is well respected when it comes to iterative development and Spotify is growing rapidly within the music industry (an industry which I also happen to work in with 7digital). Kniberg’s article gives a great insight into how companies can successfully apply some of the principles around “releasing early and often”.
In this blog post, I’ll highlight both Spotify’s underlying philosophy and the 4 key stages that form its product development process. Let’s start with Spotify’s core philosophy first:
- “We create innovative products while managing risk by prototyping early and cheaply” – Rather than taking a product through a whole product development cycle before finding out whether it provides value or not, Spotify prefers to release early and often instead. True to the philosophy of “Agile” and “Lean” development methodologies, Spotify’s focus is on launching new products or features as soon as possible to see how they perform in real-time. The main benefits of this approach are twofold. Firstly, risk is managed more effectively and companies are likely to get real-time product feedback (from real users) a lot sooner.
- “We don’t launch on date, we launch on quality” – I guess not all businesses have the luxury of not having too worry about deadlines – think of companies that have to oblige contracts or other time-led constraint – but Spotify clearly seem to value quality over time. I guess the main point of this principle is the focus on quality above anything else. The challenge for any product person and developer is to strike the right balance between releasing early and often AND launching quality products. For instance, at 7digital we aim to release continuously whilst making sure that every singly release is fully tested to ensure set quality standards (I’ve written about this practice earlier).
- “We ensure that our products go from being great at launch to becoming amazing, by relentlessly tweaking after launch” – “Continuous Improvement” is the key term here. I wrote about this earlier and highlighted its roots in Japanese car manufacturing. In practice, this means that Spotify will try to ensure a certain quality standard upon launch, followed by further improvements based on real-time feedback and performance. A product is never ‘finished’ and as a product person you can never rest on your laurels; I believe that iterative development is effectively a continuous loop of releasing, measuring, learning, iterating, releasing and measuring again.
Now look at the 4 product development stages that are applied at Spotify:
- “Think It” – The first stage of Spotify’s product development process is all about figuring out what type of product should be built and why. The “Think It” stage can apply to both completely new product ideas or improvements to existing Spotify products. What’s interesting about this stage is that the overriding emphasis seems to be on creating a compelling narrative and less on on coming up with convincing metrics or a tight business case. Like they do at Amazon, a product’s narrative is written well before its being built (see also the the scope of a standard Spotify “product definition” in Fig. 1 below). Prototyping is the other aspect of the “Think It” stage that I like very much; a dedicated “Think It Squad” (typically a designer, a developer and a product owner) will create a number of prototypes to kick things off, varying in fidelity.
- “Build It” – With Spotify’s “Build It” stage the focus is on creating (and shipping) a product that’s “narrative complete” and not feature complete. Going back to the focus on delivering a compelling narrative in the previous “Think It” stage, the aim is is to release a product that fulfils the basic narrative to the user. At Spotify, they don’t talk about a minimum viable product (‘MVP’) but about a “minimum loveable product” instead. It’s about creating an initial product that real users will love and that fulfils the narrative.
- “Ship It” – True to the “Lean” approach, in the “Ship It” stage, Spotify will gradually roll out the product to all of its users. Instead of one big bang release, Spotify will start by releasing to a small percentage of all users (typically 1-5%), in order to collect data. This is the best bit in my opinion; the use of data to incrementally improve a product, compare groups of users and spread risk. This approach is a clear example of “data driven product development” as advocated by the likes of Eric Ries, Ash Maurya and Steve Blank. This ‘staggered’ approach is used for continuous measuring and improving, making product improvements as it’s being rolled out to more and more users. This also means, as Kniberg points out, that by the end of the “Ship It” stage a product is by no means “feature complete”. It just means that the product (= MVP + necessary improvements) has been 100% rolled out and that it will continue to evolve.
- “Tweak It” – This is a critical part of the product lifecycle since this is where the product is likely to spend most of its time. The product is now live and being used by all users and ‘the squad’ will continue to experiment with the product. The squad will continue doing A/B tests to make improvements (big or small) until a point is reached where the product as whole needs to be evaluated, especially when the product is starting to reach the point of diminishing returns (see also Spotify’s definition of “done” for this stage in Fig. 2 below). Do we keep tweaking the product, adding minor improvements? Or do we rethink the product as a whole?
Main learning point: Kniberg’s article on how Spotify builds products provides in my opinion mandatory reading for most product managers or developers, especially for those with an interest in lean or iterative development. The 4 different product development stages used at Spotify are well defined and really help in both spreading the risks involved in launching a new product and getting real user feedback as soon as possible. Don’t worry if you don’t have the resources that Spotify have; you’ll still be able to apply a lot of the underlying principles and techniques!
Fig. 1 – Scope of a “product definition” document at Spotify
The product definition is a short document that answers questions such as:
- Why should we build this?
- Who will benefit from this and how?
- What are the key metrics that we expect this product to improve?
- What are the hypothesis? How will we know if this product is successful?
- Is this a “step change” (that is, a product that we expect will give at least a 2x improvement on the chosen metric)? If we expect only minor improvement on the metrics, there better be some other strong reason for building it, for example a strategic reason.
Fig. 2 – Definitions of “done” for the 4 stages of Spotify’s product development
- Think It – definition of done: The Think It stage ends when Spotify’s management and ‘the squad’ jointly believe that this product is worth building (or that the product will never be worth building and should be discarded).
- Build It – definition of done: The Build It stage ends when Spotify’s management and ‘the squad’ jointly believe that this product fulfils the basic narrative and is good enough to start releasing to real users.
- Ship It – definition of done: The Ship It stage ends when the product is available to all users.
- Tweak It – definition of done: The Tweak It stage ends when a product has reached a point of diminishing returns. The product is great and the most important improvements have been made. The cost/benefit ratio of new feature development, however, is less attractive. Looking at the metrics, new features and improvements don’t seem to be moving the needle a lot.
Fig. 3 – Scaling Agile at Spotify – Taken from: http://techcrunch.com/2012/11/17/heres-how-spotify-scales-up-and-stays-agile-it-runs-squads-like-lean-startups/
Related links for further learning: