Book review: “Designing with Data”

I’d been looking forward to Rochelle King writing her book about using data to inform designs (I wrote about using data to inform product decisions a few years ago, which post followed a great conversation with Rochelle).

Earlier this year, Rochelle published “Designing with Data: Improving the User Experience with A/B Testing”, together with Elizabeth F. Churchill and Caitlin Tan. The main theme of “Designing with Data” the book is the authors’ belief that data capture, management, and analysis is the best way to bridge between design, user experience, and business relevance:

  1. Data aware — In the book, King, Churchill and Tan distinguish between three different ways to think about data: data driven; data informed and data aware (see Fig. 1 below). The third way listed, being ‘data aware’, is introduced by the authors: “In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions.” If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.
  2. How much data to collect? — The authors make an important distinction between “small sample research” and “large sample research”. Small sample research tends to be good for identifying usability problems, because “you don’t need to quantify exactly how many in the population will share that confusion to know it’s a problem with your design.” It reminded me of Jakob Nielsen’s point about how the best results come from testing with no more than 5 five people. In contrast, collecting data from a large group of participants, i.e. large sample research, can give you more precise quantity and frequency information: how many people people feel a certain way, what percentage of users will take this action, etc. A/B tests are one way of collecting data at scale, with the data being “statistically significant” and not just anecdotal. Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance.
  3. Running A/B tests: online experiments — The book does a great job of explaining what is required to successfully running A/B tests online, providing tips on how to sample users online and key metrics to measure (Fig. 2) .
  4. Minimum Detectable Effect — There’s an important distinction between statistical significance — which measure whether there’s a difference — and “effect”, which quantifies how big that difference is. The book explains about determining “Minimum Detectable Effect” when planning online A/B tests. The Minimum Detectable Effect is the minimum effect we want to observe between our test condition and control condition in order to call the A/B test a success. It can be positive or negative but you want to see a clear difference in order to be able to call the test a success or a failure.
  5. Know what you need to learn — The book covers hypotheses as an important way to figure out what it is that you want to learn through the A/B test, and to identify what success will look like. In addition, you can look at learnings beyond the outcomes of your A/B test (see Fig. 3 below).
  6. Experimentation framework — For me, the most useful section of the book was Chapter 3, in which the authors introduce an experimentation framework that helps planning your A/B test in a more structured fashion (see Fig. 4 below). They describe three main phases — Definition, Execution and Analysis — which feed into the experimentation framework. The ‘Definition’ phase covers the definition of a goal, articulation of a problem / opportunity and the drafting of a testable hypothesis. The ‘Execution’ phase is all about designing and building the A/B test, “designing to learn” in other words. In the final ‘Analysis’ phase you’re getting answers from your experiments. These results can be either “positive” and expected or “negative” and unexpected (see Fig. 5–6 below).

Main learning point: “Designing with Data” made me realise again how much thinking and designing needs to happen before running a successful online A/B test. “Successful” in this context means achieving clear learning outcomes. The book provides a comprehensive overview of the key considerations to take into account in order to optimise your learning.

Fig. 1 — Three ways to think about data — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 3–9

  • Data driven — With a purely data driven approach, it’s data that determine the fate of a product; based solely on data outcomes businesses can optimise continuously for the biggest impact on their key metric. You can be data driven if you’ve done the work of knowing exactly what your goal is, and you have a very precise and unambiguous question that you want to understand.
  • Data informed — With a data informed approach, you weigh up data alongside a variety of other variables such as strategic considerations, user experience, intuition, resources, regulation and competition. So adopting a data-informed perspective means that you may not be as targeted and directed in what you’re trying to understand. Instead, what you’re trying to do is inform the way you think about the problem and the problem space.
  • Data aware — In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions. If you are aware there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that might be available to you.

Fig. 2 — Generating a representative sample — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 45–53

  • Cohorts and segments — A cohort is a group of users who have a shared experience. Alternatively, you can also segment your user base into different groups based on more stable characteristics such as demographic factors (e.g. gender, age, country of residence) or you may want them by their behaviour (e.g. new user, power user).
  • New users versus existing users — Data can help you learn more about both your existing understand prospective future users, and determining whether you want to sample from new or existing users is an important consideration in A/B testing. Existing users are people who have prior experience with your product or service. Because of this, they come into the experience with a preconceived notion of how your product or service works. Thus, it’s important to be careful about whether your test is with new or existing users, as these learned habits and behaviours about how your product used to be in the past can bias in your A/B test.

Fig. 3 — Know what you want to learn — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, p. 67

  • If you fail, what did you learn that you will apply to future designs?
  • If you succeed, what did you learn that you will apply to future designs?
  • How much work are you willing to put into your testing in order to get this learning?

Fig. 4 — Experimentation framework — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 83–85

  1. Goal — First you define the goal that you want to achieve; usually this is something that is directly tied to the success of your business. Note that you might also articulate this goal as an ideal user experience that you want to provide. This is often the case that you believe that delivering that ideal experience will ultimately lead to business success.
  2. Problem/opportunity area — You’ll then identify an area of focus for achieving that goal, either by addressing a problem that you want to solve for your users or by finding an opportunity area to offer your users something that didn’t exist before or is a new way of satisfying their needs.
  3. Hypothesis — After that, you’ll create a hypothesis statement which is a structured way of describing the belief about your users and product that you want to test. You may pursue one hypothesis or many concurrently.
  4. Test — Next, you’ll create your test by designing the actual experience that represents your idea. You’ll run your test by launching the experience to a subset of your users.
  5. Results — Finally, you’ll end by getting the reaction to your test from your users and doing analysis on the results that you get. You’ll take these results and make decisions about what to do next.

Fig. 5 — Expected (“positive”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 227–228

  • How large of an effect will your changes have on users? Will this new experience require any new training or support? Will the new experience slow down the workflow for anyone who has become accustomed to how your current experience is?
  • How much work will it take to maintain?
  • Did you take any “shortcuts” in the process of running the test that you need to go back and address before your roll it out to a larger audience (e.g. edge cases or fine-tuning details)?
  • Are you planning on doing additional testing and if so, what is the time frame you’ve established for that? If you have other large changes that are planned for the future, then you may not want to roll your first positive test out to users right away.

Fig. 6 — Unexpected and undesirable (“negative”) results — Taken from: Rochelle King, Elizabeth F. Churchill and Caitlin Tan — Designing with Data. O’Reilly 2017, pp. 228–231

  • Are they using the feature the way you think they do?
  • Do they care about different things than you think they do?
  • Are you focusing on something that only appeals to a small segment of the base but not the majority?

Related links for further learning:

  1. https://www.ted.com/watch/ted-institute/ted-bcg/rochelle-king-the-complex-relationship-between-data-and-design-in-ux
  2. http://andrewchen.co/know-the-difference-between-data-informed-and-versus-data-driven/
  3. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
  4. https://vwo.com/ab-split-test-significance-calculator/
  5. https://www.kissmetrics.com/growth-tools/ab-significance-test/
  6. https://select-statistics.co.uk/blog/importance-effect-sample-size/
  7. https://www.optimizely.com/optimization-glossary/statistical-significance/
  8. https://medium.com/airbnb-engineering/experiment-reporting-framework-4e3fcd29e6c0
  9. https://medium.com/@Pinterest_Engineering/building-pinterests-a-b-testing-platform-ab4934ace9f4
  10. https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166

 

Design with Data.jpg

 

 

 

 

Book review: “Product Leadership”

A few months ago, Richard Banfield, Martin Eriksson and Nate Walkingshaw published “Product Leadership”, a great book which shares stories and insights from top product managers who’ve built world class products and highly performant product teams. Banfield, Eriksson and Walkingshaw are all well established product management experts, and as part of writing “Product Leadership” they interviewed some of the best product managers in the field about their ways of working.

The authors distinguish between product management and product leadership, stressing the need to focus more on product leadership. This book isn’t about just managing a product team; as the book states, “A product leader is ultimately responsible for the success and failure of a product, and by extension, the company itself.” This raises the question about what makes a true product leader:

  1. “Product is people” – The book makes a great point about the importance of people within a product management context. More and more organisations are moving away from a traditional top-down approach, whereby senior management determines ‘upstream’ what people ‘downstream’ should be doing. Companies like Spotify have been very successful in creating autonomous and cross-functional teams. The success of this approach doesn’t only come from the cross-functional and collaborative nature of Spotify teams; people within these teams are also very close to the customer. When Banfield, Eriksson and Walkingshaw talk about ‘people’, they talk about people both within and outside of a product organisation.
  2. Focus on customer problems – I love how the book stresses the importance of focusing on customer problems. I liked the comment in the book from User Interface Engineering’s Jared Spool. He suggests having themes on the roadmap instead of features: “Themes are a promise to solve a problems not to build features.” Once you’ve truly understood customer problems, you can start prioritising. Banfield, Eriksson and Walkingshaw suggest that prioritisation is best done through the lens of the core product management principles: first, is it valuable; second, is it usable and third, is it feasible?
  3. Creating a successful product team – The book covers the ingredients of successful product teams (see Fig. 1 below). The authors point out that none of the people they interviewed mentioned hard skills, such as engineering, design or even specific product management expertise. In contrast, with product managers, the focus is much more on soft skills. One of my favourite product people, Drift CEO and cofounder David Cancel, believes that the traits of a successful product leader will always be weighted towards softer skills.”What we’re looking for are things that fit more on the qualitative side, which people hate because it’s so squishy”, Cancel explains.
  4. How to identify product leaders – The book rightly makes the point that hiring product leaders is hard. Experienced product people with strong soft skills are low in supply in most markets. One can derive the characteristics of good product leaders from the high level traits of a successful team (see previous point), and the book contains some helpful characteristics of good product leaders (see Fig. 2).
  5. Aligning team members – I’ve heard some product leaders talking about “treating your product team as a product”, and the book talks along similar lines about successful product teams. One key aspect in this respect is a product leader’s ability to align team members around a shared goal or vision.

Main learning point: “Product Leadership” is a great book for anyone interested in making the leap from being a product manager to a product leader. The book isn’t prescriptive in what makes a true product leader or how to become one; the authors have interviewed a wide range of global product leaders who share a wide range of experiences and perspectives. Because of its real life examples and lessons learned, “Product Leadership” isn’t your average ‘how to’ book. Instead, it helps readers reflect on what what product management is and what excellence in this field looks like.

 

Fig. 1 – Common characteristics of successful product teamsTaken from: Richard Banfield, Martin Eriksson and Nate Walkingshaw. Product Leadership, pp. 74-76

  • Lifelong learning – Actively seeking new information, insights and understanding is a cornerstone to successful product management. Connected to this is an open-mindedness and coachability in all respected leaders; they don’t assume they know all the answers and constantly refactor their thinking to take in new knowledge and feedback.
  • Strong communication – This is a big bucket that includes several soft skills like listening and presenting, but communication is fairly self-explanatory.
  • Empathy – Deeply connected to communication is empathy – not only empathy for team members inside and outside the organisation, but for customers as well.
  • Diversity – Strong leaders know that a diversity of backgrounds, experiences, and demographics provides the range of perspectives needed to build a complete product experience.
  • Business savvy – This characteristic is related to how product leaders understand their role in the value delivery process and their grasp of the broader business context.
  • Cross-functional representation – Teams without representation see the world with strong biases. The more a product leader can bring the functional areas of a product together, the more likely the team will act in a coherent manner.
  • Collocation – Working side by side with your team makes communication easier and faster.
  • Autonomy – The best teams have the ability to solve problems on their own without having to run everything up the ladder. Decision-making skills and the authority to implement those decisions are critical to a team’s speed and effectiveness.
  • Interdependence – This might seem almost contradictory to the autonomous point, but successful teams don’t wall themselves off from the rest of the company. Autonomy is a reference to the ability to make decisions, not a reflection on their avoidance of others.
  • Accountability – The best teams place guardrails in place that allow them to see clearly when either the qualitative research isn’t paying back or the product releases are not hitting the desired outcomes.

Fig. 2 – How to identify product leaders – Taken from: Richard Banfield, Martin Eriksson and Nate Walkingshaw. Product Leadership, pp. 76-82

  • Plays well with others – Product management and product leadership are people-first roles. Teams are made up of individuals who each bring their own personalities, perspective, and opinions. Understanding what makes people tick makes it easier to get the best out of them and identify when there is a problem. Being an empathic product leader doesn’t mean you have to be all things to all people but you’re able to engage emphatically with others, even if they don’t have a lot in common with them.
  • Always acts and thinks “team first” – The kind of leader that needs constant recognition and praise is not likely to be the best person for this job. The top product leaders hardly ever get recognition for their hard work and dedication. Furthermore, they tend to turn any limelight they receive back on their teams.
  • Is comfortable wearing lots of hats – Product leaders are typically comfortably wearing the hats of marketer, manager, technologist, customer advocate, and facilitator, to name a few. This doesn’t mean they are experts at each of these roles; rather, they are comfortable working across all areas as the role evolves or as the product demands.
  • Displays curiosity – This trait relates to several others mentioned here, but needs to be addressed specifically. Having a deep interest in learning, enthusiastically accepting new challenges, encouraging others to speak up, and listening to their perspectives are all actions driven by curiosity.
  • Communicates well – This might seem so obvious it’s not worth mentioning, but it’s surprising how many senior managers and leaders are poor communicators. Having the ability to share information in a clear, concise manner is a necessary and desirable skill for a product leader. Whether they’re talking, listening, writing, planing, sharing, or facilitating, the goal is always to be understood.
  • Possesses selling skills – This might be controversial, but when you consider what product leaders do each day, it’s no surprise that selling is one of the things to look out for in product leaders. To be clear, this is not the type of selling associated with business development; rather, it’s not the type that works to change minds and get buy-in from others.
  • Has exceptional time management skills – Shipping product is a race against time. Managing that race is a series of little decisions that collectively make up the product roadmap.
  • Shows equanimity / grace under fire – Being equanimous is tightly connected to the “plays well with others” trait mentioned above. “We practice equanimity – the idea of taking all this emotional anxiety and trying to structure it, understand it, and channel it into productive output,” says Jeff Veen, Design Partner at True Ventures. Who is on the team matters much less than how the team interacts, and the number one dynamic that defined successful teams was a sense of psychological safety – a sense that the team would not embarrass, reject, or punish someone for speaking up. Being able to instil this sense of psychological safety and trust is a key part of the product leader’s job.

 

My product management toolkit (24): ‘pre-mortem’

As helpful as ‘post-mortem’ sessions can be, how often do you find yourself wondering: “if I’d only thought about that beforehand!” In a medical environment, a post-mortem creates the opportunity for health professionals and the family to establish the patient’s cause of death. With digital products, a post-mortem typically happens at the end of a project or after a big product release. It’s similar to Agile retrospectives which often take place more regularly; the goal is to reflect on areas for improvement and related actions.

With a pre-mortem, the idea is to think about a project or a piece of work before commencing it. A pre-mortem can be done either as a group or as an individual exercise, and participants need to imagine the product or project to have failed (see Fig. 1 below).

For example:

“This product has failed because nobody wanted to pay for it”

“The product was never released in the end because the regulator refused to approve the product and its underlying business model” 

“The product has failed because many of its users complained about its poor user experience”

“The product failed because we the design & build project had to be rushed in the end to get it over the line?”

A pre-mortem thus enables you to start working backwards, and look at the individual factors that could lead to failure as well as the actions to take in order to pre-empt failure. Once you’ve identified these factors, you can start thinking about how to best avoid or mitigate these risks from occurring.

Naturally, it’s up to you to figure out how to best structure your pre-mortem, but these are some aspects that you might want to consider:

  • The product has bombed – Rather than consider the possible failure of a product or a project, in the pre-mortem the product is assumed to have failed. Similarly, if you’re doing a pre-mortem at the start of a project, it’s really helpful if all the participants in the pre-mortem think of any reason why the project could fail dramatically.
  • Categorise failures – When getting people to think about the reasons for failure, I always find it helpful to think about failures in specific areas, dependent on your product, market or organisation (see Fig. 2 below).
  • Does this mean that nothing will go wrong!? – Of course, things will not go according to plan and unexpected things will still happen. However, right from the get go, you’re creating an environment that allows people to think about and continuously talk about risk. You’re effectively creating an environment where people can speak freely about potential risk scenarios, without fear of being seen as impolite or difficult.
  • Identify threats and risks – There are a number of ways to elicit risks and threats from the pre-mortem. You could do a simple ‘brain dump’ for example. Alternatively, you could try a more structured approach whereby people need to come up with risks within certain areas (see “Categorise failures” above).
  • Analyse likelihood – Once you’ve collated threats and risks, you can use a simple scale ranging from “highly unlikely” to “highly likely” to assess likelihood of a specific risk or threat materialising. I’ve found that the activity of placing risks against the aforementioned scale, helps creating a shared understanding of the biggest risks. It also focuses the mind on the actions that need to be taken or things that need to be in place to mitigate risk.
  • Are the right people in the room? – When running a pre-mortem session with a number of people, it’s important to make sure you’ve got the right people in the room. For example, if you’re looking to work on a digital product, it’s important to have developers and designers present. Similarly, if you’re working in a highly regulated industry, having a compliance person brainstorming upfront about potential risks will go a long way.
  • Brainstorm and assess preventative actions – Once you’ve collectively established your highest risks, you can start thinking about ways to mitigate these risks. Realistically, you might not be able to stop all risks from happening  (think about a garden party being impacted by a sudden rain downpour …). In these scenarios, you can still figure out how to best reduce the impact of a risk happening and come up with a ‘plan B’. For example, if the heavens open during your garden party, people could always seek shelter in the tent that you put up ‘just in case’.
  • Not a one off exercise! – The risks and actions that you identify during your pre-mortem are something that you can use as ongoing reference point. “How are we doing we against risk X?” or “We can see risk Y materialising soon, how can best pre-empt things?” A regular product retrospective offers a great opportunity to take a step back from the day-to-day and look at the risks involved with your project or product.

Fig. 1 – Pre-Mortem by Dave Gray – Taken from: http://gamestorming.com/pre-mortem/

Fig. 2 – Possible failure categories to consider:

  1. Marketing – Think upfront about what could go wrong with your product’s go-to-market strategy and the marketing channels that you’re looking to use. For example, what would be the impact be if product messaging fails to resonate with your target audience?
  2. Customer – What if customers don’t want to pay for our product? What if we’re addressing the wrong customer segment? What if we’ve got customer needs wrong?
  3. Technology – The chosen technology implementation doesn’t work, what do we do? The technology isn’t scalable!? At this stage, it’s very valuable to explore the complexities or unknowns related to the technical implementation. As a group, you might agree on running discovery sprints to test certain technologies or non-happy path scenarios to deal with some of the ‘unknown unknowns’ being discovered early on.
  4. Stakeholders – I know some people like stakeholder mapping, but the pre-mortem can also help in figuring out who the relevant stakeholders are and how they’d need to be communicated with. Also, when look at the different risk areas, it can be very helpful which stakeholders have the most knowledge in a particular domain.
  5. Cost – What if our budget gets slashed halfway through the project? What if costs spiral because we failed to budget properly? What if the technology we’ve chosen turns out to be far too expensive or complex to implement?
  6. Compliance – Compliance and regulatory aspects are easy ones to forget at the start of a project. However, compliance can have a massive impact on a product or a project. For instance, I was involved in a project where we only found out halfway through that launching in China would be very tricky from an internal compliance and external regulation point of view. This proved to be a major stumbling block and it would have been good if we could have identified this as a major project risk beforehand.
  7. Competition – What if your direct competitor launcheds the same product as yours, but just a bit quicker? Or we imagine a situation where the product has failed because the competitor lowered its product price. Do we need a plan B for when this scenario happens? If so, what should this plan look like?

Main learning point: Running a pre-mortem isn’t a substitute for fully planning projects upfront. You can do the best pre-mortem possible but things will still happen, and to which you’ll need to adapt. A good pre-mortem, however, will get people to think and be open about risk. It will help significantly in identifying and addressing risk early and often.

Related links for further learning:

  1. https://zapier.com/blog/project-retrospective-postmortem/
  2. https://www.cmcrossroads.com/article/when-postmortems-meet-retrospectives-improving-your-agile-process
  3. http://retrospectivewiki.org/index.php?title=Retrospective_Plans
  4. https://en.wikipedia.org/wiki/Pre-mortem
  5. https://hbr.org/2007/09/performing-a-project-premortem
  6. https://simplicable.com/new/premortem
  7. http://gamestorming.com/pre-mortem/
  8. http://www.springer.com/gb/book/9783319490663

Book review: “Customers Included”

In the book “Customers Included”, Mark Hurst and Phil Terry make a great case for listening to the customer. In the book, Hurst and Terry look at why customers get overlooked by companies and explain how to best engage with customers:

  1. Why do customers get overlooked? – “The problem with customers is that they don’t always know what’s best for them” is a quote from Netflix CEO Reed Hastings referred to in the book. Similarly, Harvard Business School professor Clayton Christensen, warns that paying too much attention to today’s customers could lead a company to avoid the necessary step of disrupting itself to prepare for tomorrow’s market. These are commons reasons for why customers don’t always get involved or listened to when it comes to creating or improving products.
  2. Listening and disrupting can go hand in hand – Hurst and Terry argue that listening to customers isn’t as black and white as the likes of Hastings and Christensen portray it to to be. There’s room for nuance; accounting for different types of customers and different ways of listening to them. They make the point that “being disruptive requires knowing how to listen, in the right ways, to the right customers.” I totally agree that even in disruptive environments, it’s still essential to include the customer.” The point being that innovation should be focused on creating benefits for the customer, measuring innovation by its impact on the customer.
  3. Difference what people think and what they actually do – There’s typically a big difference between what people think (or say they think) and what they actually do. In my experience, this phenomenon raises its head particularly in focus groups, where people get together to give their feedback on a product. Hurst and Terry make the point that the very structure of a focus group fails to approximate real-world usage of a product, simply because having a number of people talking about a product doesn’t equal actual usage.
  4. The power of direct observations – The risk with research methods like focus groups is that customers give hypothetical answers, speculating about how they might behave, or how they could feel. I don’t find this feedback particularly helpful as it doesn’t give me a reliable indiction of how people actually behave or how they really feel. This is the key reason why Hurst and Terry advocate the use of direct observations; observing people in the appropriate environment, watching what they (don’t) do. For example, if you’re looking to learn more about people’s grocery shopping behaviours, you’re most likely to learn the most from observing people whilst they’re shopping at the supermarket.
  5. Doubts about personas – Hurst and Terry argue that “personas prioritise the hypothetical over the actual, and fiction over fact”. A user persona is a fictitious person with a fictitious profile. These aren’t real life people and I agree that if you do work with personas, you should always validate your made up user traits with real people. If you don’t do this validation, there’s a big risk of making product decisions solely based on hypothetical data.
  6. Limitations of task-based usability testing – Similar to the aforementioned point about personas, Hurst and Terry explain about the limitations of task-based usability testing (see Fig. 1 below). The overarching problem with only doing usability testing is that you might miss out on larger, more strategic insights. At is core, usability testing is tactical and helps to learn about how people use your product and identify any points of friction.
  7. Discovering unmet needs – “Unmet needs” are the antidote to the concept of “customers not knowing what they want” or “build it and they (customers) will come.” By just focusing on set usability tasks, Hurst and Terry argue, you’re unlikely to develop more strategic insights into your customers and their needs. To solve this, Hurst and Terry suggest direct observations and so-called “listening labs” as a better way of uncovering unmet needs.

Main learning point: “Customers Included” offers some good primers to use when convincing others of the importance of engaging with customers. More than that, the book also provides a ‘nuanced’ overview of the different user research methods to use, explaining pros and cons of each method.

Fig. 1 – Drawbacks of task-based usability testing – Taken from: Mark Hurst and Phil Terry, Customers Included, pp. 70-71

  1. The user tasks are all determined by the researchers beforehand
  2. The insights gained from the usability test are limited by those tasks
  3. The focus of task-based usability on tactical design elements

App review: ipagoo

“Accounts designed for the 21st century” is the main strapline of ipagoo, a Fintech startup that offers multi-country and cross-currency accounts. I was intrigued by this concept and decided to download ipagoo’s new iOS app and write a quick review:

My quick summary of ipagoo (before using it) – I expect ipagoo to offer services similar to the World Account that World First, the company I work for, launched a few months ago. The World Account makes it easy for customers to open accounts in specific local currencies, making it easy to receive and pay out in difference currencies.

How does the app explain itself in the first minute? – When I open ipagoo’s iOS app, the first screen is a login one. I’m somewhat thrown by this as I don’t have an ipagoo account and the calls to action aren’t as clear as they could have been.

After finding and tapping on “new to ipagoo?” I land on a screen which explains the different services that ipagoo offers:

  • Accounts – Open and manage multiple accounts in different countries from your smartphone.
  • Payments – Keep payments under control and manage counter parties from the app.
  • Money Transfer – Move money between accounts instantly.
  • Currency Exchange – Switch funds between currencies in real time.

Despite looking a bit clunky, the app does a good job in letting its users know at the start of the onboarding process about the documents required for a successful registration. There’s a clear explanation as to how take a geotagged selfie, and that users need to make sure their phones are enabled for geolocation (complete with short videos for both Android and iOS users). The two-factor authentication process ipagoo applies feels relatively straightforward and seamless; I validate my email address on my desktop, after which a “complete my credentials” screen appears on my mobile app.

However, I’m slightly daunted by the 6-step registration process and the proofs of my ID that I need to upload. The look and feel of the form isn’t compelling, and fields like “previous name(s)” make me wonder why these are necessary in the first place. I do upload my passport, but give up on the registration process when asked for a recent proof of address, a picture of myself with my ID and detailed info about my financial status. I realise that I simply don’t know enough about ipagoo and the benefits of using its services, so I decide to explore things further before deciding whether or not sign up.

 

Perceived benefits of ipagoo – I understand that ipagoo uses traditional banking services to provide its users with a single portal to all their bank accounts, standing orders, debit cards, etc. The ease of accessing multiple accounts and data through a single app should make life a lot easier for customers. Currently, people like me – with accounts and debit cards in different countries – are currently constrained in account access, and having to have multiple logins.

Points of differentiation (existing and future) – Whilst I believe the frictionless aspect of ipagoo will soon become a ‘hygiene factor’ (given that the likes of Curve and Varo Money offer a similarly seamless experience) in future, I expect ipagoo become more of a ‘financial hub’ for customers, using APIs and the opportunities that PSD2 offer. I wouldn’t be surprised if ipagoo do more to address cross-border payments, as well as traditional standing orders, bank transfers, etc. I was therefore pleased to see that some of these aspects are represented on ipagoo’s roadmap. What I didn’t see on ipagoo’s roadmap was predictive analytics and recommendations, being able to understand customer profiles and recommend other financial products accordingly.

Main learning point: I really like ipagoo’s proposition and see plenty opportunities for ipagoo to make the (financial) life of their customers easier. However, I do believe there’s a need to improve the onboarding and user experience of the app, before integrating new services. As it stands, there’s scope to simplify the app experience, making ipagoo a truly ‘sticky’ proposition for its customers.

Related links for further learning:

  1. https://www.ipagoo.com/services
  2. https://moven.com/solutions/
  3. https://about.holvi.com/
  4. http://www.bankingtech.com/277222/new-entrant-ipagoo-targets-businesses-with-pan-european-current-account/
  5. https://marcabraham.com/2017/05/22/how-psd2-is-set-to-change-banking-up-as-we-know-it/
  6. https://ibsintelligence.com/ipagoo-brings-borderless-banking-app-uk/
  7. http://www.thebanker.com/Techvision/Ipagoo-joins-the-EU-dots-to-beat-the-regulators

Book review: “Radical Candor”

In my experience, as you further your career, you’re likely to lead other people in some capacity or another. Whether you’re managing people or simply interacting with them, giving and receiving feedback can often be tricky.  I believe that being able to both share and receive feedback is a true skill that only few people have truly mastered. I for one, feel that I still have a lot to learn about how to best give constructive feedback, especially since I’d rather not use the age old “sh*t sandwich” since I don’t believe in dressing up negative feedback, and most people tend to see through the sh*t sandwich anyway.

Fig. 1 – The “Sh*t Sandwich” by Lighthouse – Taken from: https://getlighthouse.com/blog/give-feedback-team-sh-t-sandwich/

This prompted me to read “Radical Candor”, a book published earlier this year by Kim Scott. The main premise of “Radical Candor” is that you don’t need to cuss or shout or act rude to be a great boss. In contrast, the book encourages leaders to create relationships based on trust with the people that you work with.

These are the main things that I learned from reading “Radical Candor”:

  1. What do bosses do? – I really like Kim Scott’s definition of a boss’ responsibility: “bosses guide a team to achieve results.” Bosses are ultimately responsible for achieving results. Rather than doing all the work themselves, bosses rely on other people to achieve results, and will guide them accordingly. Scott goes on to unpick the aforementioned definition further, which I found very valuable (see Fig. 2 below).
  2. Trusting relationships are the key – For me, Scott’s point about the importance of building and maintaining “trusting relationships” is probably the crux of the book. Once a relationship of trust has been established, it becomes so much easier to practise “radical candor” on a daily basis. Unfortunately, there’s no set formula for developing trust. Scott, however, has identified two dimensions that help people move in the right direction: “Care Personally” and “Challenge Directly”.
  3. Care Personally – I was really pleased to read about Scott slashing the idea of people having two radically different personas – with people’s work persona being radically different to their private persona. Scott makes the point that you need to be your whole self to have a good personal relationship. She also talks about genuinely caring for the people who work for you as a critical prerequisite for a strong relationship. Unfortunately, I too often come across managers who regard the people that work for them as “resources” and treat them accordingly. Getting people to think more deeply about the “Care Personally” part of the trust relationship equation should help in stopping employees being referred to and treated as “resources.”
  4. Challenge Directly – “Challenge Directly” involves telling people that their work isn’t good enough. I personally have often found this the hardest part to do, as I’ve found there to be a fine line between challenging directly and (passive) aggression. Scott argues that challenging people “is often the best way to show them that you care when you’re the boss.” As counterintuitive as it may sound; challenging people directly can be a great way to establish a relationship. Challenging people, in a clear but constructive way, is often appreciated – despite it feeling hard initially (for both the poser and the receiver of the challenge). It shows (1) you care enough to point out both the things that are going well and the things that aren’t and (2) that you’re willing to admit when you’re wrong and that you’re committed to fixing mistakes that you or others have made. At the end of the day, it’s all about fixing a problem in my opinion.
  5. “Operationalising” good guidance – The book introduces a helpful matrix, which has four quadrants to consider in light of how to best care personally and challenge directly: “Ruinous Empathy”; “Manipulative Insincerity”; “Obnoxious Aggression” and – the desired one – “Radical Candor” (see Fig. 4 below). Scott stresses that each quadrant refers to guidance, not to personality traits. These quadrants are not used to label people, but to learn about the types of guidance we are or should be providing to the people we interact with. Having reflected on each of these quadrants, I found them to be very useful and ‘true’ (see Fig. 5 below).
  6. How to criticise without discouraging? – Scott mentions a number of useful tips on how to criticise people without discouraging the person. Also, it’s important to ask for criticism before giving it. As hard as it can sometimes feel, it’s important to actively and continuously ask for feedback, as a way of building a two-way relationship (see point 4. above). Scott provides some pointers to make it easier to ask for guidance, particularly from people that report into you (see Fig. 6 below). Secondly, be humble and helpful, offer guidance in person and immediately, criticise in private, and don’t personalise. Thirdly, make it clear that the problem isn’t due to some inflexible personality flaw, and share stories when you’ve been criticised for something similar.

Main learning point: Being radically candid doesn’t mean that you can just be rude and upset people. Instead, “Radical Candor” does a great job of offering readers with lost of valuable tips about how to care personally and challenge directly.

Fig. 2 – Former Secretary of State Colin Powell’s famous comment about sometimes having to piss people off – https://www.pinterest.co.uk/pin/389983648959114509/ 

 

Fig. 3 – Unpicking the responsibilities that come with being a boss – Adapted from: Kim Scott. Radical Candor. pp 6 -7

  1. Guidance: Guidance is often called “feedback”. People dread feedback – both the praise, which can feel patronising, and especially the criticism. However, in order to solve problems or make the most of opportunities, people do need to solicit guidance from others, and encourage it between them.
  2. Team-building: Building a cohesive team means figuring out the right people for the right roles: hiring, firing, promoting. Once you’ve got your team in place, the focus should be on engaging with your team (without micro-managing) and keep people motivated.
  3. Results: Ultimately, it’s all about achieving results. As a boss it’s your responsibility to guide your team towards achieving key results.

Fig. 4 – Radical Candor’s “Care Personally Change Directly” matrix – Taken from: https://www.radicalcandor.com/about-radical-candor/

Fig. 5 – Examples of the four quadrants of Radical Candor’s “Care Personally Change Directly” matrix – Adapted from: Kim Scott, Radical Candor, pp. 22-42

Radical Candor:

“I admire that about that you” is a great example of radical candid praise. It’s relatively easy to say “thank you” or “you’re awesome”, but it can be much harder to really think about the praise you want to give, personalise and contextualise it. For example, “I think the mentoring that you do is really impressive, I admire the way in which you take your own learnings and share them with people who are the stage that you were at a few year ago.”

Coming up with criticism when you’re being successful is probably a great time to apply radical candid criticism. I recently spoke to a senior executive whose company had just gone through a difficult patch, probably for the first time in its existence. “We’ve had it easy for so long” he explained to me. This comment made me wonder whether he and his colleagues would have benefited from a healthy dose of radically candid criticism whilst they were still winning. For example, “we just achieved over $10 million in revenue, and it has been a record year, but I think it’s important that we look at how to reduce our operational margins in the coming year so that this growth can become more sustainable.”

Obnoxious Agression:

A word of warning: “Radical Candor” isn’t about offering bosses a blank cheque to be rude or aggressive and act like a jerk. This is a lesson that I’ve been trying to take to heart, as I’ve experienced that there’s often a very thin line between being assertive and aggressive. Whilst I believe in directness over sugarcoating things , I’ve learned that 100% directness doesn’t work for everyone and can easily be perceived as aggressiveness. Scott’s point about the debilitating nature of Obnoxious Aggression therefore really resonated with me.

Manipulative Insincerity:

Manipulatively insincere guidance happens when you don’t care enough about a person to challenge directly. People give praise and criticism that’s manipulatively insincere when they are too focused on being liked or think they can gain some sort of political advantage by being fake – or when they’re just to tired to care or argue anymore. When you challenge directly, as Scott explains, you truly care about the people that you challenge; “let go of vanity and care personally.” The flip side happens when you don’t care and end up simply wasting your and everybody else’s time by trying to fake it.

Ruinous Empathy:

Scott claims that most people want to avoid creating tension or discomfort at work. Purely based on personal experience, I think Scott’s right; over the years, I’ve seen quite a few managers who actively try to make everyone happy. Whilst this is a laudable intention, I believe it hardly ever works like that. My personal mantra is that healthy tension doesn’t have to be a bad thing, it can actually help people grow and make teams more effective. You can’t be friends with all your colleagues nor can you make people happy all the time. Scott’s points about Ruinous Empathy made me think about how to best solicit feedback from team members, and ask for criticism. Scott urges all bosses to “start by asking for criticism, not by giving it!”

Fig. 6 – Soliciting impromptu guidance – Adapted from: Kim Scott, Radical Candor, pp. 130-136

  • Have a go-to question: In order to make it easier and less awkward to ask your direct reports for performance feedback or guidance, Scott suggest using a go-to question. She learned this technique from Fred Kofman, who used to be her coach at Google and is the author of “Conscious Business”. “Is there anything I could do or stop doing that would it make it easier to work with me?” This is just a sample to go-to question, the key goal here is to get the conversation going and to remove any feelings of awkwardness.
  • Embrace the discomfort: In case your go-to question fails to have the desired effect, and the other person answers that everything is fine or struggles to come up with something, remain quiet. It can be tempting to say “We’ll that’s great then” (or something along those lines) but that’s not going to help anyone in my opinion. Leaving some silence or suggesting to rearrange can help in getting your direct reports over their – understandable – hurdle.
  • Listen with the intent to understand, not to respond: If you’re anything like me, i.e. not super comfortable with asking people for feedback, your initial response might well be to act defensively and respond to the criticism. Scott urges us not to do that; don’t start criticising the criticism! Instead, she suggests saying something like “So what I hear you saying is …”
  • Reward criticism to get more of it: If you did get feedback, the next important thing is to follow up and show that you really welcomed the feedback. If you agree with what was said, you should make a change as soon as possible. If the necessary change will take time, do something visible to show you’re trying.
  • Gauge the guidance you get: I love Scott’s suggestion to try and keep a tally of the number of times people reporting to you have criticised you. Equally, measure how often they praise you. Scott mentions, that you should be weary it it’s all praise and no criticism! It means that you’ll have to work harder to get people to criticise you.

 

Related links for further learning:

  1. https://www.radicalcandor.com/about-radical-candor/
  2. http://codingwithempathy.com/2016/08/23/examples-of-radical-candor/
  3. https://www.radicalcandor.com/blog/radical-candor-not-brutal-honesty/

App review: Receipt Bank

It isn’t often that one of the apps that I use on a regular basis attracts a large round of funding but it happened recently with Receipt Bank, a London based started which “makes your bookkeeping, faster, easier and more efficient.” Last month, Receipt Bank received a Series B investment worth $50 million from New York based Insight Venture Partners.

Receipt Bank, which started in 2010, targets accountants, bookkeepers and small businesses. It offers them an online platform through which users can submit their invoices, receipts, and bills by taking a picture and uploading it through Receipt Bank’s mobile app (see Fig. 1), desktop app (see Fig. 2), or an email submission. Receipt Bank’s system then automatically extracts relevant data, sorts and categorises it. Apart from viewing your processed expenses online, Receipt Bank also publishes everything to the user’s accounting software of choice, FreshBooks or Xero for example.

Fig. 1 – Screenshot of Receipt Bank iOS app

 

 

Fig. 2 – The entry in Receipt Bank for one of my receipts

Given that I’ve been using Receipt Bank for a while now; instead of just reviewing existing functionality, I’ve also had a think about how I’d use a $50m war chest to further build out the Receipt Bank product:

  1. Faster! Faster! Faster! – When I started using Receipt Bank last year, I emailed the customer support team enquiring about the wait between submitting a picture of a receipt and it being “ready for export”. I got a friendly reply explaining that “we ask for a maximum of 24 hours to process items, but we are usually much faster than that.” The customer support adviser also explained that “the turnaround time also depends on the number of items waiting to be processed by the software and also their quality.” I’m sure Receipt Bank uses some form of machine-learning, algorithms to automatically interpret and categorise the key data fields from the picture of a receipt. As the field of Artificial Intelligence continues to evolve, I expect Receipt Bank to be able to – eventually – process receipts and invoices within seconds, with no need for the user to add or edit any info processed. Because I envisage machine learning to be the core driver of Receipt Bank’s proposition, I suggest spending at least half of its latest investment on AI technology and engineers specialised in machine learning.
  2. Not just tracking my bills and invoices – Yes, everybody is jumping on the chatbot wagon (and some of the results are frankly laughable). However, I do believe that if Receipt Bank can learn a sufficient amount about its customers and their spending and accounting behaviours, it will be able to provide them with tailored advice and predictions. For example, if I pay my supplier in China a fixed amount per month to keep my stock up, I’d like to ask Receipt Bank’s future “Expense Assistant” how my supplier payments will be affected if there’s massive volatility in the exchange rate between the British Pound and the Chinese Yuan. Similarly, when I look at most of today’s finance departments, the people in these teams seem to spend on matching the right payments received to the relevant invoice(s) sent out. I realise that the machine learning around multiple invoices wrapped into a single payment is easier said than done, but I don’t think it will be impossible and the $25m investment into AI (see point 1. above) should help massively.
  3. What if the days of paper bills are numbered!? – Now that I’ve effectively spent $25m on AI technology, I’ve got $25m left. The first thing I’d do with this remaining money is to prepare for scenarios where invoices or receipts are no longer issued on paper but provided orally. At the moment, capability like Alexa Expense Tracker is mostly used for personal expenses, but I do envisage a future where people use Alexa or Siri to add and track their expenses. Given that voice technology is still very much in its infancy, I suggest restricting Receipt Bank’s investment into this area to a no more than $1m.
  4. Integrate more (and please don’t forget about Asia) – If I were Receipt Bank I’d probably use about $10m of the remaining fund to enter new geographies and integrate with additional systems. For example, I like how Sage’s Pegg hooks into any expenses you record on your mobile, whether it’s via Slack, Facebook, Skype, WhatsApp, etc. I don’t know whether Receipt Bank is looking to enter the Asian market, but I feel there’s great opportunity to integrate with messenger apps like WeChat and Hike, without spending more than $2m on development and marketing. Also, integrating with payment processors, like Finsync did recently with Worldpay, is an integration avenue worth considering! 
  5. But don’t forget about the current product! – I feel Receipt bank would be remiss if it were to forget about improving its current platform, both in terms of functionality and user experience. For example, I can’t judge how well Receipt Bank does in retaining its customers, but I feel there are a number of ways in which it can make the existing product ‘work harder’ (see Fig. 3 below). In my experience, some of my proposed improvements and features shouldn’t break the bank. By spending about $1m on continuous improvements over a number of years, Receipt Bank should have at least $20m left in the bank, as a buffer for difficult times and any new opportunities that might arise during the product lifecycle.

Fig. 3 – Suggestions to make Receipt Bank’s existing product work harder:

  1. Some touches of gamification – I’d argue that the longevity of the relationship between Receipt Bank and an individual user is determined by how often the users uploads bills onto the platform. I assume that most users will most probably not view managing their expenses as fun, I think it would be good to look at ways to make the experience more fun. For example, I could get a gold star from my accountant once I’ve successfully synced my month’s expenses into my accounting system. I feel that there’s plenty of room to reinforce the current gamification elements that Receipt Bank uses. For example, the message that Receipt Bank managed to save 27 minutes of my time doesn’t really do it for me (see Fig. 4 below). Instead, the focus could be on the productivity gain that I’ve made for billable work (if I’m a freelancer for example).
  2. Better progress and status updates – Even if it does continue to take up to 24 hours. to categorise and process my expenses, it would be great if Receipt Bank could make its “in progress” status more intuitive and informative.
  3. Clearer and stronger calls to action – For example, I can see that I’m not making the best use of my Receipt Bank subscription (see Fig. 5 below). However, there are no suggestions on specific actions I can take to get more value from my Receipt Bank plan.

Fig. 4 – Screenshot my Receipt Bank usage

Fig. 5 – Screenshot of my Receipt Bank “Usage summary”

Main learning point: Having thought about Receipt Bank’s current product offering, and my understanding of their target market, I suggest investing a good chunk of the recent investment into optimising the machine learning algorithms in such a way that both processing speed and accuracy are significantly increased. By doing this, the customer profile and behavioural data generated, will create additional opportunities to further retain customers and offer adjacent products and services.

Related links for further learning:

  1. http://uk.businessinsider.com/receipt-bank-raises-50-million-from-insight-venture-partners-2017-7
  2. https://venturebeat.com/2017/07/20/receipt-bank-raises-50-million-insight-venture-partners/
  3. https://itunes.apple.com/gb/app/receipt-bank-business-expense-scanner-tracker/id418327708?mt=8
  4. https://play.google.com/store/apps/details?id=com.receiptbank.android&hl=en_GB 
  5. https://www.forbes.com/sites/bernardmarr/2017/07/07/machine-learning-artificial-intelligence-and-the-future-of-accounting/#49bb42ac2dd1
  6. https://hellopegg.io/
  7. http://uk.pcmag.com/cloud-services/87846/feature/23-must-have-alexa-skills-for-your-small-business
  8. https://www.accountingweb.co.uk/tech/accounting-software/case-study-receipt-banks-rapid-growth
  9. https://www.finextra.com/pressarticle/70263/finsync-connects-with-worldpay-us
  10. http://www.bankingtech.com/520502/symitars-episys-core-system-integrated-with-amazon-echo-baxter-cu-an-early-taker/