My product management toolkit (36): Google’s HEART framework

March 6, 2019 marcabrahamLeave a comment

Measure. Measure. Measure. Tracking the impact of a product is crucial if you wish to learn about your product and your customers. I’ve written before about the importance of spending time on defining the right metrics to measure, avoiding the risk of succumbing to data overload. That’s all well and good, but what do you do when the key things to measure aren’t so tangible!? For example, how do you measure customer feelings or opinions (a lot of which you’ll learn about during qualitative research)?

A few years ago, Kerry Rodden – whilst at Google – introduced the HEART framework which aims to solve the problem of measuring less tangible aspects of the products and experiences we create (see Fig. 1 below). The HEART framework consists of two parts:

The part that measures the quality of the user experience (the HEART framework)
The part that measures the goals of a project or product (the Goals-Signals-Metrics process)

Fig. 1 – The HEART framework combined with the Goals-Signals-Metrics process – Taken from: https://medium.com/@dhruvghulati/google-s-heart-framework-a-critical-evaluation-a6694421dae

Both parts are very helpful tools to have in one’s product management toolkit as they’ll help you to measure product performance through the lens of the person using your product:

HEART framework

Happiness – Measure of user attitudes, often collected via surveys or interviews. For example: satisfaction, perceived ease of use, and net promoter score.
Engagement – Measures the level of user involvement, typically via behavioural proxies such as frequency, intensity, or depth of interaction over some time period. Examples include the number of visits per user per week or the number of photos uploaded per user per day.
Adoption – New users of a product, feature or a service. For example: the number of accounts created in the last seven days, the number of people dropping off during the onboarding experience or the percentage of Gmail users who use labels.
Retention – The rate at which existing users are returning. For example: how many active users from a given time period are still present in some later time period? You may be more interested in failure to retain, commonly known as “churn.”
Task success – This includes traditional behavioural metrics with respect to user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate. This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow.

Certainly, the HEART framework isn’t bullet proof (nor does it have to be in my humble opinion). For example, Dhruv Ghulati has written up some valid concerns about how the HEART metrics could easily contradict each other or shouldn’t be taken at face value. I do, however, believe that the HEART framework is a valuable tool for the following reasons and use cases:

Learning how customers feel about your product.
Correlating these learnings with actual customer behaviours.
Does the product help achieve key customer tasks or outcomes? Why (not)?
What should we focus on? Why? How to best measure?

The HEART framework thus works well in measuring the quality of the user experience, making intangible things such as “happiness” and “engagement” more tangible.

Goals-Signals-Metrics process

The HEART framework goes hand in hand with the Goals-Signals-Metrics process, which measures the specific goals of a product. I came across a great example of the Goals-Signals-Metrics process, by Usabilla. This qualitative user research company applied the HEART framework and the Goals-Signals-Metrics when they launched a 2-step verification future for their users.

Fig. 2 – Usabila’s application of the HEART framework – Taken from: https://usabilla.com/blog/how-to-prove-the-value-of-your-ux-work/

This example clearly shows how you can take ‘happiness’, a more intangible aspect of Usabilla’s authentication experience, and make it measurable:

Question: How to measure ‘happiness’ with respect to Usabilla’s authentication experience?
Goal: The overarching goal here is to ensure that Usabilla’s customers feel satisfied and secure whilst using Usabilla’s product.
Signals: Positive customer feedback on the feature – through a survey – is a strong signal that Usabilla’s happiness goal is being achieved.
Metrics: Measuring the percentage of Usabilla customers that feels satisfied and secure after using the new authentication experience.

The Usabilla example of the HEART framework clearly shows the underlying method of taking a fuzzy goal and breaking it down into something which can be measured more objectively.

Main learning point: The HEART framework is a useful tool when it comes to understanding and tracking the customer impact of your product. As with everything that you’re trying to measure, make sure you’re clear about what you’re looking to learn and how to best interpret the data. However, the fact that the HEART framework looks at aspects at ‘happiness’ and ‘engagement’ makes it a useful tool in my book!

Related links for further learning:

Michael Margolis: “user research, quick and dirty” (1)

June 26, 2018 marcabraham1 Comment

Why do I keep coming across businesses that struggle to engage with their (prospective) customers, to learn about their needs and behaviours? Too often for my liking, I hear comments like:

“Marc, we’re a startup, we don’t have the time and budget to do customer research!”

“I’m not allowed to talk to customers.”

“In my old place, we used to have a dedicated user research team and they’d just give me their research report on a platter, after them having spoken to users.”

It therefore felt quite timely when a colleague pointed me in the direction of Michael Margolis, UX Research Partner at Google Ventures. Back in 2013, Margolis delivered a great Startup Lab workshop in which he covered the ins and outs of “User research, quick and dirty”. The recording of the 90 minute workshop is available on YouTube and you can find Margolis’ slides here (see also Fig. 1 below).

Fig. 1 – Michael Margolis’ Startup Lab workshop: “User Research, Quick ‘n’ Dirty” – Published on 26 February 2013 on https://youtu.be/WpzmOH0hrEM

I watched Margolis’ workshop in full and these are my main takeaways:

Seeing through users’ eyes

Margolis started off his session by talking about the importance of continuously learning about users, seeing things through their eyes. In a subsequent Medium post, Margolis writes that in his experience, startups will typically use UX research to achieve one of these objectives:

Improve a process or worklflow
Better understand customer shopping habits
Evaluate concepts
Test usability
Refine a value proposition

Two types of user interviews

It’s great to hear Margolis making a distinction between two types of interviews:

Usability: A usability interview is all about learning whether users can actually use your product and achieve their goals with it. Can users do it? Can they understand it? Can they discover features?
Discovery: Discovery type user interviews tend to be more contextual, and delve more into the actual user. Who? Where? When? Why? How? All key questions to explore as part of discovery, as well as the user’s existing behaviours, goals, needs and problems.

Margolis then talks about combining the two interview types and highlight two sample questions to illustrate this combination:

“How do you do things now?”

“How do you think about these things?”

The distinction between “usability” and “discovery” isn’t just an artificial one. I love Margolis’ focus on objectives, acknowledging that objectives are likely to vary depending on the type of product, its position within the product lifecycle and the learnings that you’re looking to achieve. I’ve found – at my own peril – that it’s easy to jump straight into defining user tasks or an interview script, without thinking about your research objective and what Margolis calls “North Star questions” (see Fig. 2 below).

Fig. 2 – Michael Margolis’ 5 studies startups needs most- Taken from: https://library.gv.com/field-guide-to-ux-research-for-startups-8569114c27fb – Published on 5 May 2018

Margolis provides some very useful pointers about discovery and usability questions, which you can use to create a research plan and an interview guide:

Sample discovery questions – as suggested by Michael Margolis:

What are users’ behaviours, attitudes and expectations towards the product?
Who are the key user groups? What are their needs and behaviours?
What are the pros/cons of different designs? Why?
What are the pros/cons of competitor products?
How are people using existing/competitor products? What features are mots important and why?
What barriers hinder users from adopting <product>?

Sample usability questions – as suggested by Michael Margolis:

Can users discover feature X?
Are users able to successfully complete primary tasks? Why (not)?
Do users understand feature X? Why (not)?

In a similar vein, I believe it’s important to distinguish between problem and solution interviews. There’s a risk of your customer insights becoming muddled when you mix problem and solution interviews, especially if you alternate problem questions with solution questions.

In a problem interview, you want to find out 3 things:

Problem – What problem are you solving? For example, what are the common frustrations felt by your customers and why? How do their problems rank? Ask your customers to create a top 3 of their problems (see the problem interview script in Fig. 1 below).
Existing alternatives – What existing alternatives are out there and how does your customer perceive your competition and their differentiators? How do your customers solve their problems today?
Customer segments – Who has these problems and why? Is this a viable customer segment?

Fig. 3 – Outline of a problem interview script – Taken from: Ash Maurya – “Running Lean”

In a solution interview, you want to find out 3 things:

Early adopters – Who has this problem and why? How do we identify and engage with early adopters? (see Fig. 3 below)
Solution – How will you solve their problems? What features do you need to build as part of your solution, why?
Pricing/Revenue – What is the pricing model for your product or service? Will customers pay for it, why?

Fig. 4 – Outline of a solution interview script – Taken from: Ash Maurya – “Running Lean”

Main learning point: In his Startup Lab workshop, Michal Margolis, drops a lot of very valuable tips on how to best keep customer research quick and simple, whilst still learning the things about your customer and/or product that you’re keen to learn. So much so that Michael Margolis’ tips warrant another blog post, which I’ll share soon!

Related links for further learning:

Book review: “Inspired: How To Create Tech Products Customers Love”

May 4, 2018 marcabrahamLeave a comment

About four years ago I read and reviewed Inspired: How To Create Products Customers Love by Marty Cagan, who I regard almost as the ‘founder’ of modern tech product management – along with Steve Jobs of course 🙂 Cagan has now released a second edition of “Inspired” in which he captures two aspects he’s uncovered since writing the first edition.

The first aspect is a critical need to focus on the specific job of the product manager, aiming to clarify which elements constitute the role of a product manager in a tech company. The second aspect is the importance of creating the right product culture for success, and understanding the range of product discovery and delivery techniques available to solve customer and business problems.

Whilst the book contains a wealth of valuable content about product management and how to create great products; in this review I’ll primarily focus on Cagan’s recommendations with respect to product discovery and delivery. Before I do that, it’s important to first look at Cagan’s take on the “root causes of failed product efforts” (Fig. 1).

Fig. 1 – “Root Causes of Failed Product Efforts” by Marty Cagan – Taken from: https://www.mihneadb.net/notes-on-craft-conf-2015/

Cagan sees a very sequential, “Waterfall” type approach as the underlying reason why many products fail. This approaches comes down to companies using a ‘feature heavy’ and preplanned roadmaps, as well as and using regular planning sessions to negotiate and prioritise the roadmap. Cagan shares some home truths to explain why this approach is now obsolete (Fig. 2).

Fig. 2 – 10 problems that companies using a Waterfall type approach suffer – Adapted from: Marty Cagan, Inspired: How To Create Tech Products Customers Love, pp. 17-21

Stakeholder-driven products: It’s a top down approach which leads to stakeholder-driven products and teams that don’t feel empowered
Business cases are mostly fictitious: I couldn’t agree more with Cagan when he argues that the two main business case inputs – how much money we’ll make and how much it will cost – are complete unknowns. We can’t know how much money we’ll make because that depends entirely on how good the solution turns out to be. In contrast, a lot of products end up making no money whatsoever! One of the most critical lessons in product, Cagan explains, is “knowing what we can’t know.”
Product roadmaps – There are two problems with traditional, feature led product roadmaps. Firstly, the reality is that half of our product ideas are simply not going to work. I always cringe when I see product roadmaps that contain detailed features prioritised prioritised for an entire year … In my experience, until you start discovering, implementing and launching product ideas, you’re not going to know whether your product is actually going to work. Secondly, even when ideas do prove to have potential they’re likely to need several iterations to reach the point where they deliver tangible business value. Cagan introduces the term “time to money” to refer this evolutionary process.
It’s not about gathering requirements for engineers to implement – I recently came across an organisation where they employed an entire team of project managers and business analysts whose main job it was to gather stakeholder requirements, and document them for designers and engineers to implement. Cagan rightly makes the point that “this is 180 degrees away from the reality of modern tech product management.”
UX designers are getting involved way too late – Don’t involve designers only once the requirements have been gathered, it’s simply too late as the designer won’t be able to add much value add this stage.
Engineers are getting involved way too late – If you’re just using your engineers to code, you’re only getting about half their value. I love the ‘little secret’ that Cagan shares with us: “engineers are typically the best single source of innovation.” He’s totally right!
Agile for delivery only – Cagan talks about “Agile for delivery”, whereby product development teams work in an Agile fashion, but the rest of the organisation isn’t.
Project-centric processes – The company usually funds projects, pushes projects through the organisation, and finally launches projects. Unfortunately, projects are output and product is all about outcome. I’d add that most projects are one-off pieces of works whereas products have a continuous lifecycle, until the product is being discontinued.
Customer validation happens way too late – Cagan points out the biggest shortcoming of the old waterfall process, which is that all the risk is concentrated right at the end and that customer validation happens way too late. Instead, customer validation or discovery should be continuous and needs to happen early and often.

Cagan offers three overarching principles which help overcome the aforementioned root causes of failed product efforts:

Risks are tackled upfront, instead of at the end
Products are defined and designed collaboratively, rather than sequentially
Finally, it’s all about solving problems, not implementing features

“Continuous Discovery and Delivery” is a great way to translate these three principles into a process and mindset for people to adhere too (Fig. 3). You can see how Cagan has taken the eight steps involved in the traditional waterfall approach (Fig. 1) and condensed them into to three, continuous stages: Objectives – Discovery – Delivery (Fig. 3).

Fig. 3 – Continuous Discovery and Delivery – Taken from: Marty Cagan, Process vs Model, https://svpg.com/process-vs-model/, 7 August 2017

Ultimately, this process enables you to to get answers to four critical questions:

Will the user buy this (or choose to use this)?
Can the user figure out how to use this?
Can our engineer build this?
Can our stakeholders support this?

Apart from these four critical questions, I like the emphasis Cagan puts on business context over a traditional product roadmap. In the book, Cagan covers two main components that provide this business context:

The product vision and strategy
The business objectives

The “risk” aspect feels like a crucial one to me, and thinking about ways to identify and mitigate risks early and often. For example, I’ve found the pre-mortem technique to be a great way to unearth key risks right at the outset. Cagan describes some common risks to consider:

Financial risk – Can we afford this solution?
Business development risk – Does this solution work for our partners?
Marketing risk – Is this solution consistent with our brand?
Sales risk – Is this solution something our sales staff is equipped to sell?
Legal risk – Is this something we can do from a legal or compliance perspective?
Ethical risk – Is this solution something we should do?

Cagan then goes on to describe three of his favourite discovery framing techniques:

1. Opportunity Assessment

The idea is to answer four key questions about the discovery work you’re about to undertake:

Objective – What business objective is this work intended to address?
Key results – How will you know if you’ve succeeded?
Customer problem – What problem will this solve for our customers?
Target market – What type of customer are we focused on?

2. Customer Letter

Cagan refers to Amazon and their working backward process, where you start the product effort with a fictitious press release.

3. Startup Canvas

The “Startup Canvas” is particularly useful when you work at an early stage startup and are staring from scratch, both with the business and your product or service. There are lots of these canvases around for you to have a closer look at; I’d suggest having a look at the Business Model Canvas (by Alex Osterwalder) and the Lean Canvas (by Ash Maurya; Fig. 4).

Fig. 4 – Ash Maurya’s “Lean Canvas” – Taken from Ash Maurya, “Running Lean”:

Cagan explains how you can use a canvas for any product change, no matter the size, but you would likely quickly find a risk of duplication once you’ve got an existing business and product. I agree that the law of diminishing returns kicks in once you’ve already established your business and products, since you’ll have already figured out things like your cost structure or distribution strategy.

Finally, Cagan explains about “testing value” as a key thing to consider when planning your customer discovery. The main thing here, Cagan stresses, is to learn whether customers perceive your product to be substantially better than the competition. So many companies and product teams think all they need to do is match the features of the competitive alternatives. This idea of “feature parity” being enough to woo customers has proven to be a false one. The reality is that for customers to switch from an existing product, they need to perceive the new product as a much better alternative.

For example, sometimes it’s not clear whether customer want what it is that we’re going to build and it can be very risky to simply think “we’ll build it and customers will come.” In the book, Cagan talks about how you can quickly and cheaply test whether there’s demand for instance through launching just a landing page. On the landing page, we describe the new offering exactly as we would if we were really launching the service. The difference is that if the user clicks the call to action, rather than getting the expected outcome, the users sees a message that explains that you’re thinking of launching the new service and that you’d love to get initial input from the user. It all falls under the mantra “Do Things that Don’t Scale”, first introduced by Y Combinator Founder Paul Graham. A good example is this one from Buffer:

Fig. 5 – Buffer example of a ‘smoke and mirrors’ landing page – Taken from: Christopher Bank, 15 ways to test your minimum viable product, https://thenextweb.com/dd/2014/11/12/15-ways-test-minimum-viable-product/, 12 November 2014

Main learning point: Marty Cagan has written a great followup to his first edition of “Inspired”. In this edition, he offers valuable tips and examples in relation to important themes as product discovery and delivery. Whether you’re new to product management or have got some good product management experience under your belt, “Inspired: How To Create Tech Products Customers Love” is a great and valuable read.

Book review: Sprint (Part 3 – Day 2)

September 3, 2016 marcabraham1 Comment

Once you’ve set a target as part of Day 1 of your sprint, the next step is for you and your team to look at solutions. On the second day of the sprint, you’ll be coming up with solutions, and sketching them. This day consists of two key activities: (1) review ideas to remix and improve, and (2) create solution sketches to feed into your plan for a prototype and customer testing.

Remix and Improve

In “Sprint”, Jake Knapp, John Zeratsky and Brad Kowitz suggest a good technique to collate and assess ideas: Lightning Demos. With this exercise, your team will take three-minute tours of their favourite ideas or products: from other products, from different domains, and from within your own company. The idea here is to encourage the team to throw everything in the mix, but to do so in a short and snappy way. Each person who has suggested an idea will do a three minute demo, showing the team what’s great about his or her solution. As a facilitator, you might want to use a timer to make sure each team member sticks to the their three minute time slot.

The key thing with these three minute ‘lightning demos’ is that you capture the big ideas from each presentation. Start by asking the person who’s doing the tour, “What’s the big idea here that might be useful?” You can then make a quick drawing of this big idea, write a simple headline above it and add the source underneath. I’ve included an example of a way to capture big ideas in Fig. 1 below.

Fig. 1 – An example of capturing ‘big ideas’ from lightning demos, by Karsten Neben – Taken from: https://medium.com/@karstenn/what-we-learned-in-just-5-days-our-design-sprint-report-b9ada5b7f19a#.27wb5fv4e

Lightning Demos

Sketch

In the afternoon of the second day of your sprint, the focus is on coming up with solutions. Instead of doing collective brainstorming sessions – which in my experience run the risk of becoming shouting matches or can be dominated by very vocal people – Knapp, Zeratsky and Kowitz suggest each team member coming up with ideas on their own. People will work individually, thinking about and sketching solutions. I’ve included a simple example of a sketch in Fig. 2 below.

The sketches that people create will act as an important driver for the rest of the sprint. On Wednesday (Day 3), you’ll critique everyone’s sketches and pick the best ones.

If you’re worried about the quality of your sketches, don’t! Knapp, Zeratsky and Kowitz introduce the four step sketch technique. This approach makes it easy for everyone to take some rough solutions and turn them into a detailed solution sketch (see Fig. 2 below):

Notes – As a first step, the team walks around the room and takes notes from looking at all the post-it notes, whiteboards, flip-charts that you have collated over the first day and a half of your sprint.
Ideas – Each team member individually will look at his or her notes and jot down rough ideas, simply filling a sheet with doodles, headlines, etc. The aim here is not to come up with fully fledged ideas or solutions. It’s purely a way for each person to drop down their thoughts.
Crazy 8s – Crazy 8s is a fast-paced exercise. Each person will take his or her strongest ideas and rapidly sketches eight variations in eight minutes. What I like about Crazy 8s is that it stops you from dwelling on your first possible solution for too long. Instead, the eight minute deadlines forces you to quickly decide whether to move on from your first reasonable solution or to stick with it but iterate (see Fig. 3 and 4 below). I’ve found Crazy 8s to work particularly well if you end up sketching several variations of the same idea, exploring alternative versions. Similarly, you can use Crazy 8s to refine a marketing headline or messaging.
Solution sketch – The solution sketch is each person’s best idea, put down on paper in detail. Up to this point, each team member will have worked individually on creating notes, ideas and Crazy 8s, and won’t have shared anything with the rest of the team. This all changes with the solution sketch; each solution sketch is an opinionated hypothesis for how to solve the challenge at hand. These sketches will be looked at – and judged – by the rest of the team. They will therefore need to be detailed, thought-out, and easy to understand (see Fig. 5 and 6 below).

Once each team member has put together a solution sketch, the sprint facilitator will collate them all and put them in a pile. The team will only be allowed to start looking at these solution sketches on the third day of the sprint.

Fig. 2 – Four step sketch technique – Taken from: https://www.fastcodesign.com/3057076/google-ventures-on-how-sketching-can-unlock-big-ideas

4-stage sketch

Fig. 3 – How to do Crazy 8s – Taken from: “Sprint”, pp. 112-113

Each person begins Crazy 8s with a single sheet of letter-size paper.
Fold the paper in half three times, so you have eight panels.
Set a timer to sixty seconds.
Hit “start” and begin sketching – you have sixty seconds per section, for a total of eight minutes to create eight miniature sketches.
Go fast and be messy: As with the notes and ideas, Crazy 8s will not be shared with the team.

Fig. 4 – Crazy 8s example – Taken from: https://www.fastcodesign.com/1672917/the-8-steps-to-creating-a-great-storyboard

Fig. 5 – Important rules to keep in mind when creating a solution sketch – Taken from: “Sprint”, pp. 114-118

Make it self-explanatory – Your solution sketch needs to explain itself. Think of this sketch as the first test for your idea. If no one can understand it in sketch form, it’s not likely to do any better when it’s polished.
Keep it anonymous – Don’t put your name on your sketch, and be sure that everyone uses the same paper and the same black pens.
Ugly is okay – Your sketch does not have to be fancy (boxes, stick figures, and words are fine), but it does have to be detailed, thoughtful, and complete.
Words matter – Strong writing is especially necessary for software and marketing, where words often make up most of the screen. So pay extra close attention to the writing in your sketch. Don’t use “lorem ipsum” or draw those squiggly lines that mean “text will go here.” That text will go a long way to explain your idea – so make it good and make it real!
Give it a catchy title – Since your name won’t be on your sketch, give it a title. Later, these titles will help you keep track of the different solutions as you’re reviewing and choosing. They’re also a way to draw attention to the big idea in your solution sketch (see the example in Fig 6 below).

Fig. 6 – A solution sketch from Blue Bottle Coffee’s sprint; each sticky note represents one screen – Taken from: https://www.fastcodesign.com/3057076/google-ventures-on-how-sketching-can-unlock-big-ideas

Sketch example

Main learning point: The second day of the sprint is very solution oriented. Instead of long brainstorm sessions, the day is filled with more individually oriented activities, encouraging team members to think about ideas and to come up with their own solution sketch.

Related links:

Book review: Sprint (Part 1 – Setting the stage)

August 29, 2016 marcabraham1 Comment

I personally find it very encouraging to see that more and more companies go down the route of experimentation and continuous discovery. Businesses are starting to realise that committing to a single solution upfront and implementing it in the hope that it will be successful can be a very risky strategy. “Sprint – How To Solve Big Problems and Test New Ideas in Just Five Days” builds on this change by introducing the concept of a 5-day sprint in which to identify problems, explore possible solutions AND get feedback from real customers.

Jake Knapp and two of his colleagues at Google Ventures, John Zeratsky and Braden Kowitz, have successfully applied ‘sprints’ for a wide range of companies, helping the likes of Pocket and Blue Bottle to tackle difficult problems in just five days. This is how the five days are broken down:

Monday (day 1) – ‘Start at the end’; agree to a long-term goal and pick a problem to solve during the sprint
Tuesday (day 2) – Review and improve existing ideas and sketch possible solutions
Wednesday (day 3) – Select a solution to focus on and create a storyboard
Thursday (day 4) – Build a prototype, a realistic ‘facade’
Friday (day 5) – Learn through customer interviews

Sprint 1

Fig. 1 – Steven Nguyen “Reflecting on our first design sprint” – Taken from: https://sprintstories.com/reflecting-on-our-first-design-sprint-2d58519eefad#.ej6ivw4ma

The “Sprint” book contains a wealth of great techniques to utilise as part of a sprint. I want to do it justice and will probably devote a couple of posts to this great book. Before delving into each of the days of the sprint, let’s start by looking at ‘setting the stage’ before kicking off the sprint. It’s important to have the right challenge and the right team before you begin a sprint:

Challenge – As readers of my posts might know; I’m quite obsessed about understanding the problem(s) worth solving before exploring solutions. I therefore believe that picking the right problem or challenge to solve is absolutely critical to a successful sprint. Knapp, Zeratsky and Kowitz suggest three challenging situations where sprints can help: high stakes, tight deadlines or when you’re simply stuck.
Team – The key thing when assembling a sprint team is to get a ‘Decider’ in the team; someone who’s in a position to make important decisions. This can be the CEO or another important stakeholder. I like how the book provides a number of arguments one can use when a Decider is reluctant to get involved in the sprint (see Fig. 2 below). You should end up with a well balanced team, made up of people who can implement as well as subject matter experts (see Fig. 3 below).

On top of picking a team, it’s also important to have a designated facilitator who can manage time, conversations, and the overall process. Naturally, this can be someone from within your company or someone external. For example, I know lots of digital agencies that facilitate sprints as part of a piece of work for their clients. As much this is beneficial to the client, this also helps the agency by creating a shared and robust understanding of what’s going to be built and why.

Fig. 2 – Get a Decider (or two) involved – Taken from: “Sprint”, pp. 31-32

Rapid progress – Emphasise the amount of progress you’ll make in your sprint: In just one week, you’ll have a realistic prototype.
It’s an experiment – Consider your first sprint an experiment. When it’s over, the Decider can help evaluate how effective it was.
Explain the tradeoffs – Show the Decider a list of big meetings and work items you and your team will miss during the sprint week.
It’s about focus – Be honest about your motivations. If the quality of your work is suffering because your team’s regular work schedule is too scattered, say so. Tell the Decider that instead of doing an okay job on everything, you’ll do an excellent job on one thing.

Fig. 3 – Recruit a team of seven (or fewer) – Taken from: “Sprint”, pp. 34-35

Decider – Examples: CEO, founder, product manager, head of design
Finance expert – Examples: CEO, CFO, business development manager
Marketing expert – Examples: CMO, marketer, PR, community manager
Customer expert – Examples: researcher, sales, customer support
Tech/logistics expert – Examples: CTO, engineer
Design expert – Examples: designer, product manager

Main learning point: I recommend everyone doing a ‘sprint’ before committing to a specific solution. “Sprint” is a great book, with a lot of helpful guideance as to how to best solve big problems in five days. I’d argue that some of the techniques to use as part of sprint shouldn’t be constrained to a 5-day period or at the start of a piece work; I’ll outline in the coming posts how some of the sprint exercises can be used on a continuous basis.

Sprint 2

Related links for further learning:

Looking at key omni-channel analytics – Part 1

December 13, 2014 marcabrahamLeave a comment

Over the past few weeks I’ve been learning about retailers and how they sell via a multitude of channels. The next thing for me now is to learn about some key omni-channel analytics. Let’s start with some questions to ask when measuring omni-channel retail and marketing:

What is the impact of online channels on offline and vice versa? – Given the fluid nature of consumer decision-making, alternating between online and offline, it’s important to measure the impact of online activities on offline and vice versa.
What does the conversion path look like? – How and where do we convert people into paying customers? Where do we lose people and why? Which channels do contribute to conversion and to which degree?

I’ll start by looking at the impact of online activities on offline conversion. I learned an awful lot from a 2008 blog post on tracking offline conversions by data guru Avinash Kaushik. Before I delve into some of Kaushik’s great suggestions, I want to take a step back and think about potential things to measure and why:

What is the impact of online channels on offline conversion? – As a product person, I’m keen to understand the relationship between online activities and actual purchases in-store. This understanding helps me to focus on the right online and offline elements of the value proposition, comprehending which things can be optimised inline to achieve a specific outcome in-store.
How do I best measure revenue impact of my website or mobile app in an omni-channel world? – For example, I’ve got a nice eCommerce site or app with a decent amount of traffic, 20% of which gets converted into actual online purchases. However, what happens with the remaining 80% of traffic that doesn’t get converted? Is my website or app delivering some value to this 80%!? If so, how? Can we measure this?

Now, let’s look at some practical tips by Kaushik in this respect:

Track online store locator or directions – If I track in an analytics tool the interactions with the URL for Marks & Spencer’s store locator, I can start learning about the number of Unique Visitors that are using the store locator in a certain time period (see Fig. 1 below). In addition, I can look at the number of visits or visitors where a certain post code or town has been entered into the store locator. I can take this insight as a starting point to learn more about the people within a certain geographical area that have a tendency to use the Marks & Spencer site and its store locator. Once a user then goes on to click on “Show on map” or “Enter an address for directions to this store” you can make some inferences about the user’s intentions to actually visit the M&S store in question.
Use of a promo code – Using an online voucher or promo code is an obvious way to combine online tactics with offline conversion (see a John Lewis example in Fig. 2 below). One can use the promo code as an event in an analytics tool and capture data on e.g. the number of codes or vouchers exchanged in-store vs the number of vouchers sent. I guess the only downside is that you’re unable to capture many interesting insights if a user doesn’t redeem her voucher or code.
Controlled experiments – Running controlled experiments was the bit in Kaushik’s piece that intrigued me the most. The idea behind these experiments is to validate retail ideas in the real world (the same as “experimentation” in a ‘lean’ context, which I’ve written about previously). As Kaushik explains, “the core idea is to try something targeted so that you can correlate the data to your offline data sources (even if you can’t merge it) and detect a signal (impact).” I’ve included some prerequisites for successful experiments in Fig. 3 below. One of them is to isolate the experiment to different states that are far from each other. As Kaushik explains, this way you are isolating “pollutants” to your data (things beyond your control that might give you sub optimal results).

Main learning point: Learning about how online can affect offline conversion felt like a good starting point for my getting a better understanding of the world of omni-channel analytics. The next step for me is to find out more about the impact of offline on online conversion: how can we best measure the impact of what happens offline on the conversion online?

Fig. 1 – Screenshot of the results of Mark & Spencer’s store locator

Fig. 2 – Sample John Lewis voucher – Taken from: http://www.dontpayfull.com/at/johnlewis.com

Fig. 3 – Some points on prerequisites on controlled experiments by (online) retailers:

Clearly defined customer segments of a decent size to quantify the impact of the experiment.
Design the experiment in such a way that the results can be isolated and compared in a meaningful way (e.g. IKEA umbrella sales on a rainy vs on a sunny day).
Random selection of customers in the control group (who get the current offering) and the treatment group (who get the experimental offering).
Clear assumptions and hypotheses which underpin the experiment.
Create a feedback loop which allows you to measure or observe how customers respond to different experiments.

Related links for further learning:

Priya Prakash explains about Design Principles at Mobile Academy

October 22, 2014 marcabraham4 Comments

As part of the Mobile Academy curriculum, I recently attended a class by Priya Prakash on “design principles”. Priya is a very experienced designer and has founded Design for Change, a London-based urban experience design studio.

Priya started off the session by explaining that design principles describe the experience of core values of a product or a service. Design principles help in making decisions on your product. She referred to a great definition of design principles by Luke Wroblewski (see Fig. 1 below). The important part of Luke’s definition is that all decisions can be measured against design principles.

“Design is what you decide not to do” was one of the key points that Priya raised in this class. It’s all about doing less and simplifying things. She talked about Spotify and Google Glass as good examples in this respect:

Content first – Focus on the content, and remove any unnecessary user interface elements.
Get familiar – Even though there is a clear distinction between a “lean forward” mode (Spotify desktop app) and “lean back” mode (Spotify mobile app), there’s a unified design language which has been executed consistently, irrespective of the device that you access Spotify from.
Don’t get in the way – Google Glass is designed to be there when you need it and to be out of the way when you don’t. The goal is to offer engaging functionality that supplements the user’s life without taking away from it.
Keep it relevant – Deliver information at the right place and time for each Google Glass user.

Priya then talked about motion user interface design principles:

Personality – For example, the Pitchfork app has a magazine like feel. It’s about understanding what the content is and translating this into appropriate behaviours.
Responsive – Priya talked about the Clear app as being very responsive, explaining how this app gracefully expands or contracts.
Context – Motion should give context to the content on screen by detailing the physical state of those assets and the environment they reside in.
Emotive – This principle is all about evoking a positive emotional response. This kind of response can be triggered by wide range of user interface elements, for example smooth transition or a nice animation. Yelp‘s app is a good example in this regard.
Orientation – Motion should help ease the user through the experience. The “orientation” principle means that motion should establish the “physical space” of the app by the way objects come on and off the screen or into focus. The key is to get the flow of actions right, guiding the user on her journey and make sure she doesn’t feel lost or confused. Mobile apps like Yelp and Evernote do this pretty well in my opinion.
Restraint – Keep it simple! Similar to the abovementioned “orientation” principle, it’s important not to bombard the user wity too much animation or confuse them with too many interactions to choose from. This is one of the reasons why I’m so a big fan of single purpose apps; I like the simplicity that they offer and the level of design restraint that they tend to apply.

Main learning point: I learned a lot from Priya Prakash’s class on design principles, particularly with respect to motion user interface design principles. Design principles can provide valuable guidance for the design of any software product or service and should therefore not be taken lightly. Thanks to Priya for a great class!

Fig. 1 – Definition of design principles by Luke Wroblewski – Taken from: http://www.lukew.com/ff/entry.asp?854

“Design principles are the guiding light for any software application. They define and communicate the key characteristics of the product to a wide variety of stakeholders including clients, colleagues, and team members.”

“Design principles articulate the fundamental goals that all decisions can be measured against and thereby keep the the pieces of a project moving toward an integrated whole.”

Fig. 2 – What makes a good design principle? – Taken from Priya’s lecture at the Mobile Academy on 14 October ’14:

Specific enough to help make a choice
Focuses the team – avoid being broad
Measurable against user need or product/business goal

Related links for further learning:

As I learn …

This site is about everything digital, giving an update on new things as I learn

Category: Google

My product management toolkit (36): Google’s HEART framework

HEART framework

Goals-Signals-Metrics process

Michael Margolis: “user research, quick and dirty” (1)

Book review: “Inspired: How To Create Tech Products Customers Love”

Book review: Sprint (Part 3 – Day 2)

Remix and Improve

Sketch

Book review: Sprint (Part 1 – Setting the stage)

Looking at key omni-channel analytics – Part 1

Priya Prakash explains about Design Principles at Mobile Academy