Book review: “Who – The A method for hiring” by Geoff Smart and Randy Street

Almost everyone I speak to complains about how hard it is to hire good product people. We talk about the low return on investment of lengthy interview processes and those product hires that “didn’t work out”. I’ve written about hiring product managers previously and shared my lessons learned about recruiting good product people. Who – The A method for hiring, published back in 2008 by Geoff Smart and Randy Street has been a very welcome addition to my learnings about hiring:

  1. It’s about “who” not about “what” – In the book, “who” refers to the people you put in place to make the “what” decisions. Who’s running your sales force? Who manages your product? Who engages with your customers? In the book, Smart and Street stress the importance of making the right hiring decisions, and how as a manager your success is largely determined by the quality of the people around you.
  2. When ‘who’ mistakes happen – Smart and Street list a number of instances where “who” mistakes happen; when managers end up hiring the wrong person for the job. They also talk about “voodoo hiring” (see Fig. 1-2 below). Their book aims to help people make better “who” decisions, and provides a proven method, dubbed the “A method” to help readers on their way.
  3. The limited value of CVs – There’s only so much candidate CVs will tell you as a hiring manager. In my experience, a CV can be a useful snapshot but often not more than that. Smart and Street describe a CV as “a record of a person’s career with all of the accomplishments embellished and all the failures removed.”
  4. Finding A players – Smart and Street define an A player as “a candidate who has at least a 90 percent chance of achieving a set of outcomes that only the top 10 percent of possible candidates could achieve.” To help find A Players, Smart and Street and their colleagues at ghSMART have developed the “A Method for Hiring” or the “A Method” for short. There are four steps to the A Method: (1) Scorecard (2) Source (3) Select and (4) Sell (see Fig. 3 below). Naturally, the hiring manager needs to feel confident about having A Players around him or her, which might not always be the case. Equally, I’d argue that it pays off to look for good B (and perhaps even C) players too. These candidates might not have achieved their A game yet, but still stand 70 percent chance of achieving key outcomes (and can learn as they go).
  5. Scorecard – The idea of having a predetermined scorecard for each role is a valuable one, since a scorecard describes the mission for the position, outcomes that must be accomplished, and competencies that fit with both the culture of the company and the role. What do you really want the person that you’re hiring to accomplish, and why? What’s the phase that your company is in, and which the person you’re hiring needs to adapt to? The scorecard consists of three parts: (1) the job’s mission (2) outcomes and (3) competencies (see Fig. 4 below).
  6. Source – The idea of sourcing is all about constantly looking for talented people, irrespective of whether your company has an imminent hiring need. As some of the people interviewed for the book explain, ‘source’ is all about constantly asking people we know to introduce us to the talented people they know and maintaining a relationship with high-potential candidates (see Fig. 6 below). You thus build up talent pool which you can use as your first port of call when hiring, thus increasing the overall velocity of your hiring process.
  7. Select – The ‘select’ element of the A Method comes down to interviewing well. Smart and Street recommend doing structured interviews in order to avoid what they call ‘voodoo hiring methods’ (see Fig. 2 below). They suggest the following steps for selecting the right candidate:  (1) Screening Interview (2) Who Interview (3) Focused Interview (4) Reference and (5) Skill-Will Bull’s-Eye (see Fig. 7 below).
  8. Sell – In their book, Smart and Street stress the importance of putting yourself in the candidate’s shoes as the key to successful selling your candidate to join your company. Care about what they care about. The book explains how candidates typically care about five things, and encourage you to make sure that you address each of these five areas until you get the person to sign on the dotted line (see Fig. 12 below). Selling doesn’t happen just at the end of the process. Instead, you ‘sell’ throughout: When you source; When you interview; The time between your offer and the candidate’s acceptance; The time between the candidate’s acceptance and his or her first day and The new hire’s first one hundred days on the job.

Main learning point: Hiring good people is easier said than done in my experience. “Who – The A Method for hiring”, however, does a great job in offering every hiring manager with key considerations to make and techniques to apply when look for  new people.

 

Fig. 1 – When ‘who’ mistakes happen – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

Who mistakes happen when managers:

  • Are unclear about what’s needed in a job;
  • Have a weak flow of candidates;
  • Do not trust their ability to pick out the right candidate from a group of similar-looking candidates;
  • Lose candidates they really want to join their team.

Fig. 2 – Voodoo Hiring – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

Top ten voodoo hiring methods:

  1. The Art Critic – Hiring people based on gut instinct.
  2. The Sponge – A common approach among busy managers is to let everybody interview a candidate, with the risk of interviewers asking candidates exactly the same questions.
  3. The Prosecutor – Many managers act like the prosecutors they see on TV and aggressively question candidates, attempting to trip them up with trick questions and logic problems.
  4. The Suitor – Rather than rigorously interviewing a candidate, some managers spend all of their energy selling the applicant on the opportunity. Suitors are more concerned with impressing than assessing their capabilities.
  5. The Trickster – Then there are the interviewers who use gimmicks to test for certain behaviours. They might throw a wad of paper on the floor, for example, to see if a candidate is willing to clean it up.
  6. The Animal Lover – Many managers hold on stubbornly to their favourite pet questions – questions they think will reveal something uniquely important about a candidate.
  7. The Chatterbox – This technique has a lot in common with the “la-di-da” interview. The conversation usually goes something like this: “How about them Yankees! Man, the weather is rough this time of year. You grew up in California? So did I!”
  8. The Psychological and Personality Tester – Asking a candidate a series of bubble-test questions like “Do you tease small animals?” or “Would you rather be at a cocktail part or the library on a Friday night?” is not useful (although both are actual questions on popular psychology tests), and it’s certainly not predictive of success on the job.
  9. The Aptitude Tester – Tests can help managers determine whether has the right aptitude for a specific role, such as persistence for a business development position, bit they should never become the sole determinant in a hiring decision.
  10. The Fortune-Teller – Just like a fortune-teller looking in a crystal ball to predict the future, some interviewers like into the future regarding the job at hand by asking hypothetical questions: “What would you do? How would you do it? Could you do it?”

Fig. 3 – “The ghSMART A Method for Hiring” – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

 

  • Scorecard – The scorecard is a document that describes exactly what you want a person to accomplish in a role. It is not a job description, but rather a set of outcomes and competencies that define a job well.
  • Source – Finding great people is getting harder, but it is not impossible. Systematic sourcing before you have slots to fill ensures you have high quality candidates waiting when you need them.
  • Select – Select talent in the A Method involves a series of structured interviews that allow you to gather the relevant facts about a person so you can rate your scorecard and make an informed hiring decision. These structured interviews break the voodoo hiring spell.
  • Sell – Once you identify people you want on your team through selection, you need to persuade them to join. Selling the right away ensures you avoid the biggest pitfalls that cause the very people you want the most to take their talents elsewhere.

Fig. 4 – The three parts of the A Method scorecard – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  • The mission – The mission is an executive summary of the job’s core purpose. It boils the job down to its essence so everybody understands why you need to hire someone into the slot. You’ll know you have a good mission when candidates, recruiters, and even others from your team understand what you are looking for without having to ask clarifying questions. A good mission statement could for instance read: “To serve as a visionary leader who helps the bank capture market share from the competition by analysing the market and devising successful new strategies and product offerings.” Mission statements also help you avoid one of the most common hiring traps: hiring a generalist over a specialist.
  • Outcomes – Outcomes, the second part of a scorecard, describe what a person needs to accomplish in a role. If you are hiring for a sales person for instance, the scorecard should read” “Grow revenue from $25 million to $50 million by end of year three.” This is a clearly defined outcome which a sales person either can or can’t achieve. An outcome is something which a person must get done.
  • Competencies – Competencies flow directly from the first two elements of the scorecard. The mission defines the essence of the job to a high degree of specificity. Outcomes describe what must be accomplished. Competencies describe how you expect a new hire to operate in the fulfilment of the job and the achievement of the outcomes. The book lists some critical competencies for A Players (see Fig. 5 below).

Fig. 5 – Critical competencies for A Players – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  • Efficiency – Able to produce significant output with minimal wasted effort.
  • Honesty / integrity – Does not cut corners ethically. Earns trust and maintains confidences.
  • Organisation and planning – Plans, organises, schedules, an budgets in an efficient, productive manner. Focuses on key priorities.
  • Aggressiveness – Moves quickly and takes a forceful stand without being overly abrasive.
  • Follow-through on commitments – Lives up to verbal and written agreements, regardless of personal cost.
  • Intelligence – Learns quickly. Demonstrates ability to quickly and proficiently understand and absorb new information.
  • Analytical skills – Able to structure and and process qualitative and quantitative data and draw insightful conclusions from it. Exhibits a probing mind and achieves penetrating insights.
  • Attention to detail – Does not let important details slip through the cracks or derail a project.
  • Persistence – Demonstrates tenacity and willingness to go the distance to get something done.
  • Proactivity – Acts without being told what to do. Brings new ideas to the company.
  • Ability to hire A Players (for managers) – Sources, selects and sells A Players to join a company.
  • Ability to develop people (for managers) – Coaches people in their current roles to improve performance, and prepares them for future roles.
  • Flexibility / adaptability – Adjusts quickly to changing priorities and conditions. Copes effectively with complexity and change.
  • Calm under pressure – Maintains stable performance when under heavy pressure or stress.
  • Strategic thinking / visioning – Able to see and communicate the big picture in an inspiring way. Determines opportunities and threats through comprehensive analysis of current and future trends.
  • Creativity / innovation – Generates new and innovative approaches to problems.
  • Enthusiasm – Exhibits passion and excitement over work. Has a can-do attitude.
  • Work ethic – Possesses a strong willingness to work hard and sometimes long hours to get the job done. Has track record of working hard.
  • High standards – Expects personal performance and team performance to be nothing short of the best.
  • Listening skills – Lets others speak and seeks to understand their viewpoints.
  • Openness to criticism and ideas – Often solicits feedback and reacts calmly to criticism or negative feedback.
  • Communication – Speaks and writes clearly and articulately without being overly verbose or talkative.
  • Teamwork – Reaches out to peers and cooperates with supervisors to establish and overall collaborative working relationship.
  • Persuasion – Able to convince others to pursue a course of action.

Fig. 6 – How to source – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  1. Referrals from your professional and personal networks.
  2. Referrals from your employee
  3. Deputising friends of the firm
  4. Hiring recruiters
  5. Hiring researchers
  6. Sourcing systems

Fig. 7 – Select: interview steps – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  1. Screening Interview – The screening interview is a short, phone-based interview designed to clear out B and C Players from your roster of candidates. In the screening questions you can ask some of the following questions: What are your career goals? What are you really good at professionally? What are you not good at or interested in doing professionally? Who were your last five bosses, and how will they each rate your performance on a 1-10 scale when we talk to them? The whole point of the screening interview is to weed people out as quickly as possible.
  2. Who Interview – The Who Interview is designed to give you more confidence in your selection because it uncovers the patterns of somebody’s career history, which you can match to your scorecard (see Fig. 8 below for a sample Who Interview Guide). The Who Interview is a chronological walkthrough of a person’s career.
  3. Focused Interview – The Who Interview is comprehensive and will get you most of the way toward the right answer of who to hire. In the Focus Interview, you can gather additional, specific information about your candidate. In essence, you’re turning the magnification up another notch so you can give would-be hires one last look with a finer degree of granularity (see a sample Focused Interview Guide in Fig. 10 below).
  4. Reference Interview – There are three things you have to do to have successful reference interviews. First, pick the right references. Second, ask the candidate to contact the references to set up the calls. Third, conduct the right number of interviews; you personally do about four and ask your colleagues to do three, for a total of seven reference interviews (see sample Reference Interview Guide in Fig. 11 below).
  5. Skill-Will Bull’s-Eye – The goal of the “Select” step of the A Method is to gather the facts you need to decide if somebody’s skill (what they can do) and will (what they want to do) match your scorecard. This is a person’s skill-will profile. When a candidate’s skill-will profile matches up perfectly with the requirements outlined on your scorecard, your candidate hits the the skill-will bull’s-eye.

Fig. 8 – Who Interview Guide – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  1. What were you hired to do?
  2. What accomplishments are you most proud of?
  3. What were some low points during that job?
  4. Who were the people you worked with?
  5. Why did you leave that job?

Fig. 9 – ‘Master Tactics’ for the ‘Who Interview’ – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  • Interrupting – You have to interrupt the candidate. If you don’t, he or she might talk for then hours straight about things that are not at all relevant. The bad way to interrupt somebody is to put up your hand like a stop sign gesture and say, “Wait, wait, wait. Let me stop you there. Can we get back on track?” The good way to interrupt somebody is to smile broadly, match their enthusiasm level, and use reflective listening to get them to stop talking without demoralising them.
  • The Three P’s – The three P’s are questions you can use to clarify how valuable an accomplishment was in any context. The questions are: (1) How did your performance compare to the previous year’s performance? (2) How did your performance compare to the plan? (3) How did your performance compare to that of peers?
  • Push Versus Pull – People who perform well are generally pulled to greater opportunities. People who perform poorly are often pushed out of their jobs. Do not hire anybody who has been pushed out of 20 percent or more of their jobs. Push: “It was mutual.” “It was time for me to leave.” “My role shrank.” Etc. Pull: “My biggest client hired me.” “My old boss recruited me to a bigger job.” “The CEO asked me to take a double promotion.” Etc.
  • Painting a Picture – This is all about empathic imagination, which helps you move away from generic answers that don’t mean anything and toward specific details that give you real insight. You’re really trying to put yourself in the candidate’s shoes and get curious to truly understand.
  • Stopping at the Stop Signs – One of the advantages of the Who Interview in person is that you can watch for shifts in body language and other inconsistencies. The idea isn’t to gather dirt. Think of yourself instead as a biographer interviewing a subject. You want both the details and the broad pattern, the facts and the texture.

Fig. 10 – Who Interview Guide – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  1. The purpose of this interview is to talk about _______________ (Fill in the blank with specific outcome or competency such as the person’s experience selling to new customers, building and leading a team, creating strategic plans, etc).
  2. What are your biggest accomplishments in this area during your career?
  3. What are your insights into your biggest mistakes and lessons learned in this area?

Fig. 11 – Reference Interview Guide – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  1. In what context did you work with the person?
  2. What were the person’s biggest strengths?
  3. What were the person’s biggest areas for improvement back then?
  4. How would you rate his/her overall performance in that job on a 1-10 scale? What about his or her performance causes you to give that rating?
  5. The person mentioned that he/she struggled with ________________ in that job. Can you tell me more about that?

Fig. 12 – Five Things Candidates Care About – Taken from: Geoff Smart and Randy Street, “Who – The A method for hiring”:

  • Fit ties together the company’s vision, needs, and culture with the candidate’s goals, strengths, and values. “Here is where we are going as a company. Here is how you fit in?”
  • Family takes into account the broader trauma of changing jobs. “What can we do to make this change as easy as possible for your family?”
  • Freedom is the autonomy the candidate will have to make his or her own decisions. “I will give you ample freedom to make decisions, and I will not micromanage you.”
  • Fortune reflects the stability of your company and the overall financial upside. “If you accomplish your objectives, you will likely make [compensation amount] over the next five years.”
  • Fun describes the work environment and personal relationships the candidate will make. “We like to have a lot of fun around here. I think you will find this is a culture you will really enjoy.”

My product management toolkit (37): product / market fit survey

Since venture capitalist Marc Andreessen introduced the concept of “product / market fit” back in 2007 there have been many different interpretations of what this concept actually means. From people using sales projections to companies applying customer satisfaction to determine product / market fit, I’ve seen companies utilising different yardsticks. I therefore thought it would be good to go back to Marc Andreessen’s original definition and look at a great approach by Sean Ellis – who specialises in growth strategies – to identify product / market fit.

First, let’s look at Marc Andreessen’s original definition of product / market fit:

“Product/market fit means being in a good market with a product that can satisfy that market.”

Andreessen has taken this definition from Andy Rachleff – another successful startup founder and venture capitalist – who describes getting to product / market fit as “the only thing that matters” when you’re a startup. Product / market fit thus means being in a good market with a product that can satisfy that market.

The million dollar question then becomes how one knows that product / market fit has been achieved. Andreessen explains how you can feel when product / market for isn’t happening:

“The customers aren’t quite getting value out of the product, word of mouth isn’t spreading, usage isn’t growing that fast, press reviews are kind of “blah”, the sales cycle takes too long, and lots of deals never close.”

Andreessen argues that you can feel when product / market fit is happening if:

“The customers are buying the product just as fast as you can make it – or usage is growing just as fast as you can add more servers. Money from customers is piling up in your company checking account. You’re hiring sales and customer support staff as fast as you can. Reporters are calling because they’ve heard about your hot new thing and they want to talk to you about it.” 

Another way of establish whether you’ve achieved product / market fit comes from Sean Ellis and his product / market fit survey. Ellis’ survey consists of one simple question:

“How would you feel if you could no longer use this product?”

People can respond to this survey in one of the following ways:

  • Very disappointed
  • Somewhat disappointed
  • Not disappointed (it really isn’t that useful)
  • N/A – I no longer use this product

Ellis’ rule of thumb is that if you’re above 40% of the people saying they’d be very disappointed, you’ve found product / market fit, and if you’re less than that, you haven’t.

Through the survey, you can learn about a person’s underlying reasoning behind their response by including a free text field which says something like “Please help us understand why you’ve selected this answer.”

In addition, you could conduct user interviews to learn more about the products which people consider as an alternative to your product or to understand the perceived benefits of your product. For example, those survey respondents who responded that they’d be “somewhat disappointed” it might be that you haven’t delivered on the product’s intended value proposition. Similarly, it would be good to find out from those people who indicated “not disappointed” why they wouldn’t care if your product ceases to exist.

Main learning point: Understanding whether (and why) your product has achieved product / market fit is critical for any startup. Learning why a product has or hasn’t achieved product / market fit through a survey or an interview will fuel further product development and decision making.

Related links for further learning: 

  1. https://www.slideshare.net/hiten1/measuring-understanding-productmarket-fit-qualitatively
  2. https://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp
  3. https://open.buffer.com/measure-productmarket-fit-product-feature/
  4. http://www.startup-marketing.com/getting-to-product-market-fit/
  5. https://growthhackers.com/
  6. https://www.pisano.co/en/blog/sean-ellis-test-figure-out-product-market-fit/
  7. https://pmarchive.com/guide_to_startups_part4.html
  8. https://foundr.com/product-market-fit-andy-rachleff-wealthfront/

Book review: The Making of a Manager by Julie Zhuo

“The Making of a Manager” is the first book by Julie Zhuo, VP of Product Design at Facebook. In “The Making of a Manager”, Julie shares her experiences and learnings with regard to her transition from being a personal contributor to becoming a manager. “This is a book about how someone with no formal training learned to become a confident manager” is the starting point for Julie’s book.

When she started her first role as a manager at Facebook, Julie had very little experience under her belt and she describes what she thought a manager’s job was:

  • have meetings with reports to help them solve their problems,
  • share feedback about what is or isn’t going well, and
  • figure out who should be promoted and who should be fired.

Without wanting to spoil the rest of “The Making of a Manager”, this is how Julie sees a manager’s job today:

  • build a team that works well together,
  • support members in reaching their career goals, and
  • create processes to get work done smoothly and efficiently.

Julie summarises that “Your job, as a manager, is to get better outcomes from a group of people working together.” She puts a great focus on outcomes and refers to her former manager Chris Cox, ex VP of Product at Facebook, who explained that half of what he as a manager looks at were his team’s results and the other half was based on the strength and satisfaction of his team.

I liked Julie’s inclusion of Richard J. Hackman’s research into what helps create successful teams (see Fig. 1 below). She uses a similar approach to managers creating the right conditions for their teams:

  • Purpose – The purpose is the outcome your team is trying to accomplish, otherwise known as the why. The first big part of your job as a manager, Julie writes, is to ensure that your team knows what success looks like and cares about achieving it.
  • People – This is all about the who. Are the members of your team set up to succeed? Do they have the right skills? Are they motivated to do great work? To manage people well, Julie explains, you must develop trusting relationships with them, understand their strengths and weaknesses (as well as your own – see below), make good decisions about who should do what (including hiring and firing when necessary), and coach individuals to do their best.
  • Process – This describes how your team works together. Julie clarifies that process in her mind isn’t about stacks of paperwork and frameworks for everything, but enabling teams to make decisions and work together effectively: “In a team setting, it’s impossible for a group of people to coordinate what needs to get done without spending time on it. The larger the team, the more time is needed.”

Staying on the topic of defining management, Julie provides a useful distinction between leadership and management. Manager is a specific role, with clear principles outlining what a manager does and how his success is measured. Leadership, on the other hand, is the particular skill of being able to guide and influence other people. Julie makes the point that a leader doesn’t have to be a manager; “Anyone can exhibit leadership, regardless of their role.”

In “The Making of a Manager”, Julie covers a lot of different facets of becoming and being a manager. From recounts her first couple of months as a manager to breaking down her views on strong management, Julie offers a ton of insights and tips for those of us who are managers or would like to take on this role. Let’s pick some aspects that resonated with me most:

  • Trust is the most important ingredient – It may sound obvious, but the importance of investing time and effort into creating / maintaining trusting relationships can get easily overlooked. Julie mentions that the hallmark of a trusting relationship is that people feel they can share their mistakes, challenges, and fears with you.
  • Giving and receiving critical feedback – Similar to Kim Scott and Amy Edmondson, Julie talks about how managers and their direct reports need to be able to give each other critical feedback regularly without it being taken personally. If your report does work that you don’t think is great, are you comfortable saying that directly? Similarly, would your report tell you if you if he thinks you’ve made a mistake?
  • Be honest and transparent about your report’s performance – As a manager, your perspective on how your report is doing carries far more weight than his perspective on how you’re doing. After all, you’re the one who determines what he works on and whether he should get a promotion or be fired.
  • Admit your own mistakes and growth areas – Julie shares how she tries to admit when she doesn’t have the answers or when she’s working through her own personal challenges, and shares a number of useful phrases that she’ll typically use when doing so (see Fig. 2 below).
  • Managing yourself – Here, Julie talks about the so-called imposter syndrome, i.e. where you doubt your accomplishments and worry being exposed as a “fraud”.  She raises the question why imposter syndrome hits managers particularly hard and gives two main reasons. Firstly, because managers are often looked to for answers. Secondly, managers are constantly put in the position where they’re put in the position if doing things they haven’t done before. She also talks about managers identifying their own strengths and weaknesses, and “being brutally honest with yourself”.
  • Amazing meetings – I liked Julie’s points about meetings, the bane of most managers’ lives. She distinguishes between decision-making meetings and informational meetings and explains how being clear about the meeting objective (and structuring the meeting accordingly) can lead to much more effective and enjoyable meetings (see Fig. 3 below).

Fig. 1 – Richard J. Hackman, Hackman’s 5 Factor Model:

Being a Real Team – One with clear boundaries and stable membership.

Compelling Direction – Provide the team with clear goals, which are both challenging and consequential.

Enabling Structure – Where possible, offering the team variety in the tasks they undertake improves the team’s success. Within the team’s structure it’s also key to ensure that team members have strong social skills.

Supportive Context – A supportive context is essential for companies and organisations, as they are made up of small groups which when combined form a larger group.

Expert Coaching – This is about coaching and mentoring the team to help achieve the outcomes they need to achieve and support team members developing their individual skills.

 

Fig. 2 – Julie Zhou, The Making of a Manager: Sample things to say when you don’t have the answer or are working through personal challenges:

  • “I don’t know the answer. What do you think?”
  • “I want to come clean and apologise for what I did/said the other day …”
  • “One of my personal growth areas this half is …”
  • “I’m afraid I don’t know enough to help you with that problem. Here’s someone you should talk to instead …”

 

Fig. 3 – Julie Zhou, The Making of a Manager: Decision-Making Meetings and Informational Meetings:

A great decision-making meeting does the following:

  • Gets a decision made (obviously)
  • Includes the people most directly affected by the decision as well as a clearly designated decision-maker.
  • Presents all credible options objectively and with relevant background information, and includes the team’s recommendation if there’s one.
  • Gives equal airtime to dissenting opinions and makes people feel that they were heard.

A great informational meeting does the following:

  • Enables the group to feel like they learned something valuable.
  • Conveys key messages clearly and memorably.
  • Keeps the audience’s attention (through dynamic speakers, rich storytelling, skilled pacing, interactivity).
  • Evokes and intended emotion – whether inspiration, trust, pride, courage, empathy, etc.

Main learning point: “The Making of a Manager” provides an honest, no bullsh*t account of what it means to be manager and how to best transition into a managerial role. Definitely worth a read if you’re manager or looking to become one.

Related links for further learning:

  1. http://www.juliezhuo.com/
  2. https://medium.com/the-year-of-the-looking-glass
  3. https://hbr.org/2009/05/why-teams-dont-work
  4. https://marcabraham.com/2018/03/12/book-review-the-no-asshole-rule/
  5. http://www.free-management-ebooks.com/faqld/development-03.htm
  6. https://www.cnbc.com/2017/01/30/sallie-krawcheck-says-a-lack-of-diversity-leads-to-bad-decision-making.html
  7. https://marcabraham.com/2019/01/27/my-product-management-toolkit-35-effective-one-on-one-meetings/
  8. https://www.youtube.com/watch?v=ZQUxL4Jm1Lo
  9. https://hbr.org/product/becoming-a-manager-how-new-managers-master-the-challenges-of-leadership/1822-PBK-ENG
  10. https://www.tmbc.com/
  11. https://strengthsprofile.com/

 

Book review: “High Output Management” by Andrew Grove

“High Output Management” by the late Andrew Grove, ex Chairman and CEO of Intel, is a must read management book in my opinion. It’s easy to be quite cynical about most management and business books as a lot of them seem to introduce one central theme (or buzzword) right at the beginning of the book, followed by endless repetition throughout the remainder of the book …

 

 

In contrast, “High Output Management” contains valuable advice and tips from the beginning right to the end of the book. First published back in 1983, the book applies production principles to management. Some of these principles really resonated with me and I feel strongly about all (product) managers benefiting from these principles as part of their day-to-day working practices:

Identifying the “limiting step” – Grove defines the “limiting step” as the step in the overall shape of the production flow that will determine the overall shape of a company’s operations. In the book, Grove uses the simple example of a breakfast company, and highlights the time required to boil an egg is the critical component or the ‘limiting step’ in the production flow (see Fig. 1 below). The key idea here is to construct your production flow by starting with the longest (or most difficult, or most sensitive, or most expensive) step and work our way back. As a (product) manager you’ll thus focus on the limiting step within your context, e.g. in the workflow of your team, the customer funnel or the decision-making process.

 

Fig. 1 – Example of a limiting step when creating a breakfast – Taken from: http://clarkeching.com/ccblog/2018/1/21/what-are-bottlenecks-andy-grove

 

Detect and fix issues at the “lowest-value stage” possible – If there’s a problem with your product, you want to find out about it as early on in the production process as possible so that you can minimise risk. In the production world, I witnessed lowest-value stage thinking first hand at the assembly line of a Toyota factory. Here, employees can pull the   “Andon” cord to (temporarily) bring things to a halt as soon as they come across an issue. It’s an easy way of escalating things, making sure that a problem or bottleneck is dealt with before proceeding with the rest of the assembly process. Think about when you last pulled an imaginary Andon cord to flag a product or team issue early!?

 

Fig. 2 Using the Andon cord to raise an issue and stop production – Taken from: https://www.lean.org/lexicon/fixed-position-stop-system

 

 

Using (leading) indicators to measure and predict – In order to run a production process well you’ll need a set of indicators which help you monitor and improve the efficiency of the production line. Grove stresses that for these indicators to be useful, “you have to focus each indicator on a specific operational goal.” He goes on to list a number of relevant production indicators (see Fig. 3 below). Leading indicators give you one way to look inside the production process by showing you in advance what the future might look like.

 

Fig. 3 – Indicators related to the production process – Taken from: Andrew S. Grove, “High Output Management”, p. 16:

  • Sales forecast
  • Raw material inventory
  • Manpower
  • Quality

 

Leverage – Grove introduces the concept of “leverage”.  This is the output generated by a specific type of work activity. An activity with high leverage will generate a high level of output; an activity with low leverage, a low level of output. This raises the question about what qualifies as managerial leverage and output. Grove’s distinction between activities and output really helps to bring the concept of leverage to life (see Fig. 4 below). The overarching idea is that with each activity that the manager performs, the organisational output should increase.

 

Fig. 4 – Managerial activities vs output – Taken from: Andrew S. Grove, “High Output Management”, pp. 39 – 54:

Managerial activities:

  • Judgments and opinions
  • Direction
  • Allocation of resources
  • Mistakes detected
  • Personnel trained and subordinates developed
  • Courses taught
  • Products planned
  • Commitments negotiated

Managerial output:

A manager’s output is the output of all of the people and the teams that report into her. For example, if someone manages a design team, then his output consists of completed designs that are ready to be implemented.  That output can take many different forms depending on the type of role and industry. Regardless of the form of output, managers must measure its quantity and quality:

A manager’s output = The output of his organisation + The output of the neighbouring organisations under his influence

 

High leverage activities – We’ve already touched on the impact of highly leveraged activities on an organisation’s output, and Grove explains how these activities can be achieved (see Fig. 5 below). For example, to maximise the leverage of his or her activities, a manager must keep timeliness firmly in mind. Equally, managers micro-managing or ‘meddling’ are examples of negative leverage activities. A big part of a manager’s work is to supply information and know-how, and to share a sense of the preferred method of handling things to the teams under his control or influence. A manager also makes and helps to make decisions.

 

Fig. 5 – Three ways in which to achieve high leverage activities – Taken from: Andrew S. Grove, “High Output Management”, pp. 54 – 55:

  • When many people are affected by one manager.
  • When a person’s activity or behaviour over a long period of time is affected by a manager’s, well focused-set of words or actions.
  • When a large group’s work is affected by an individual supplying a unique, key piece of knowledge or information.

 

Applying production principles to management – In the book, Grove rightly points out how time management techniques are commonly used to improve managerial output. He then uses production principles to improve on some of these time management techniques (see Fig. 6 below).

 

Fig. 6 – Ways to improve on time management techniques – Taken from: Andrew S. Grove, “High Output Management”, pp. 62 – 63:

  • Identify our limiting step: determine which things that have to happen on a schedule that’s absolute, and which can’t be moved. You can then plan more flexible activities around it and thus work more efficiently.
  • Batching similar tasks: group similar activities, e.g. performance reviews, as these activities tend to require (mental) set-up time. You can thus maximise the set-up time needed for the task and reduce duplication of effort.
  • Forecasting your activities: Your calendar can be a valuable production-planning tool (and not a dumping ground for random meetings or “orders” by others). If you want to use your calendar as better forecasting and planning tools, Grove suggest that two conditions should be met. Firstly, you should move toward the active use of your calendar, taking the initiative to fill the holes between the time-critical events with non-time critical though necessary activities. Secondly, you should say “no” at the outset to work beyond your capacity to handle.

 

Meetings, the output of managerial work – A lot of managerial tasks (see “High leverage activities” above) are best suited for face-to-face interactions, and more often than not, for meetings. Grove provides a useful distinction between two basic kinds of meetings: process-oriented and mission-oriented meetings (see Fig. 7 below). I love how at the end of the book’s chapter about meetings, Grove reminds us of a quote by the late management guru Peter Drucker who said that “If people spend more than 25 percent of their time in meetings, it’s a sign of malorganisation.”

 

Fig. 7 – Process-oriented and mission-oriented meetings – Taken from: Andrew S. Grove, “High Output Management”, pp. 72 – 87:

  • Process-oriented meetings: Knowledge is shared and information is exchanged during process-oriented meetings, which take place on a regular, scheduled basis. One-on-ones and team meetings are good examples of process-oriented meetings.
  • Mission-oriented meetings: The purpose of mission-oriented meetings is to solve a specific problem and often produce a decision. These meetings are ad hoc affairs, not scheduled long in advance, because they usually can’t be. The key to the success of a mission-oriented meeting is what the chair of the meeting does. The person leading the meeting needs to have a clear understanding of the meeting’s objective – what needs to happen and what decision needs to be made.

 

 

Planning: today’s actions for tomorrow’s output – In High Output Management, Grove offers three simple steps of a  planning process (see Fig. 8 below). He describes planning as an ordinary everyday activity, something we all do – both in our professional and personal lives. The planning process enables you to reflect on what’s needed, the gap with the current situation and the specific actions necessary to close the gap.

 

Fig. 8 – Three steps your planning process should consist of – Taken from: Andrew S. Grove, “High Output Management”, pp. 103 – 120:

Step 1 – Establish projected need or demand: What will the environment demand from you, your business, or your organisation?

Step 2 – Established your present status: What are you producing now? What will you be producing as your projects in the pipeline are completed? 

Step 3 – Compare and reconcile steps 1 and 2: What more (or less) do you need to do to produce what your environment will demand?

 

Main learning point: “High Output Management” is probably one of the most valuable management books I’ve read in the last couple of years. If you’re looking for an inspiring but practical book about management, I suggest you look no further: High Output Management is the book for you!

Review: Shift

Having worked on a number of online marketplace products, I’m always curious about other online marketplaces out there. So you might be able to imagine my excitement when I came across Shift, a US-based marketplace for new and used cars. Having bought used cars before, I feel that the used car industry is ripe for disruption and my hunch is that Shift is aiming to do just that.

I can see plenty of room to improve transparency and trust when it comes to buying and selling used cars and I’m keen to learn more about how Shift tries to tackle both areas:

My quick summary of Shift before using it: I expect a platform that enables consumers to discover, compare and buy used cars. Unsure whether cars are bought from dealerships or from Shift directly. Also, wondering whether I can get finance through Shift to help purchase my car.

How does Shift explain itself in the first minute? The landing page of the site shows two women, seated in a car and looking happy. The main strap-line on the site reads “Simplified car buying”, followed by “Great cars. Better prices. Test drives delivered to you.” The main navigation bar in the top right hand corner of the page shows “Financing” as one of the options for people to consider.

 

 

How does Shift work? Shift’s “Concierges” deliver test drives to customers on-demand. After a test drive one can arrange finance and purchase the car on the spot. Shift applies three driving principles to its business, as it aims to “bring trust and simplicity to the peer-to-peer used car market”: convenience, value and trust. Shift sees the Concierge as a pivotal actor as part of this experience as it’s the role of the Concierge “to be your guide. It’s not their job to sell you a car, it’s to help you buy one.”

 

 

When, for instance, I look at a used Mercedes GLE 350 to buy (see screenshot below), a few things stand out to me:

“No-haggle list price” – So there’s no room for a potential buyer to bring the price down!? From a peer-to-peer perspective, I can see how a fixed price creates a lot of clarity and trust for both parties involved in the transaction, car buyer and seller.

 

 

Compare price – I would have loved to compare prices for the specific car I’m interested in. When, however, I click on “Compare” for a a number of different vehicles on Shift’s site, I keep getting a message stating that price comparison info isn’t available.

 

 

Mechanical inspection – Would love to learn more about Shift’s process that precedes the mechanical inspection as shown for each model on the site. I deliberately looked for cars that didn’t just have a perfect list, i.e. all green marks, and I found one (see below). This Toyota Prius (2010) has three body related issues. When I click to see details, the three issue are being explained clearly, as well as their impact on both the exterior and the drivability of the car.

 

Wear & tear photos – For this nine year old Toyota Prius, Shift offers seven wear and tear photos so that I can see clear evidence of the body related issues listed in the mechanical inspection report. I can thus make up my mind – before arranging a test drive – whether I can live with these issues or not.

 

 

Having looked into buying car, I now want to see how one can sell a car through Shift:

These three steps involved in selling a car through Shift feel very similar to selling through Vroom:

 

 

Get an estimate – Getting a Shift estimate for a car to sell feels pretty straightforward (see screenshot below). My only question is how car sellers can quickly figure out whether they’re getting a good price for their car, and how this estimated price compares to what they could get elsewhere.

 

 

How and when do I get paid? Shift will initiate payment to the the car seller at the end of the appointment in which they evaluate one’s car to sell. This approach made me think of real estate platforms such as Opendoor and Nested. These companies will buy your property off you (Opendoor) or pay an advance (Nested) after they’ve thoroughly inspected and valued your home. The comparison with real estate made me wonder whether Shift refurbishes the interior of car or improves the exterior once it has bought the car off you.

 

Screen Shot 2019-03-11 at 09.18.27.png

 

Did Shift deliver on my expectations? Yes. Refreshing to see the level of simplicity and transparency into an experience which has traditionally put the (uninformed) car buyer or seller on the back foot.

 

Related links for further learning:

  1. https://www.autogravity.com/
  2. https://www.lingscars.com/
  3. https://www.vroom.com/
  4. https://shift.com/cars/
  5. https://www.drivemotors.com/
  6. https://broadspeed.com/

My product management toolkit (36): Google’s HEART framework

Measure. Measure. Measure. Tracking the impact of a product is crucial if you wish to learn about your product and your customers. I’ve written before about the importance of spending time on defining the right metrics to measure, avoiding the risk of succumbing to data overload. That’s all well and good, but what do you do when the key things to measure aren’t so tangible!? For example, how do you measure customer feelings or opinions (a lot of which you’ll learn about during qualitative research)?

A few years ago, Kerry Rodden – whilst at Google – introduced the HEART framework which aims to solve the problem of measuring less tangible aspects of the products and experiences we create (see Fig. 1 below). The HEART framework consists of two parts:

  • The part that measures the quality of the user experience (the HEART framework)
  • The part that measures the goals of a project or product (the Goals-Signals-Metrics process)

 

Fig. 1 – The HEART framework combined with the Goals-Signals-Metrics process – Taken from: https://medium.com/@dhruvghulati/google-s-heart-framework-a-critical-evaluation-a6694421dae

 

Both parts are very helpful tools to have in one’s product management toolkit as they’ll help you to measure product performance through the lens of the person using your product:

HEART framework

  • Happiness – Measure of user attitudes, often collected via surveys or interviews. For example: satisfaction, perceived ease of use, and net promoter score.
  • Engagement – Measures the level of user involvement, typically via behavioural proxies such as frequency, intensity, or depth of interaction over some time period. Examples include the number of visits per user per week or the number of photos uploaded per user per day.
  • Adoption – New users of a product, feature or a service. For example: the number of accounts created in the last seven days, the number of people dropping off during the onboarding experience or the percentage of Gmail users who use labels.
  • Retention – The rate at which existing users are returning. For example: how many active users from a given time period are still present in some later time period? You may be more interested in failure to retain, commonly known as “churn.”
  • Task success – This includes traditional behavioural metrics with respect to user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate. This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow.

Certainly, the HEART framework isn’t bullet proof (nor does it have to be in my humble opinion). For example, Dhruv Ghulati has written up some valid concerns about how the HEART metrics could easily contradict each other or shouldn’t be taken at face value. I do, however, believe that the HEART framework is a valuable tool for the following reasons and use cases:

  • Learning how customers feel about your product.
  • Correlating these learnings with actual customer behaviours.
  • Does the product help achieve key customer tasks or outcomes? Why (not)?
  • What should we focus on? Why? How to best measure?

The HEART framework thus works well in measuring the quality of the user experience, making intangible things such as “happiness” and “engagement” more tangible.

Goals-Signals-Metrics process

The HEART framework goes hand in hand with the Goals-Signals-Metrics process, which measures the specific goals of a product. I came across a great example of the Goals-Signals-Metrics process, by Usabilla. This qualitative user research company applied the HEART framework and the Goals-Signals-Metrics when they launched a 2-step verification future for their users.

Fig. 2 – Usabila’s application of the HEART framework – Taken from: https://usabilla.com/blog/how-to-prove-the-value-of-your-ux-work/

This example clearly shows how you can take ‘happiness’, a more intangible aspect of Usabilla’s authentication experience, and make it measurable:

  • Question: How to measure ‘happiness’ with respect to Usabilla’s authentication experience?
  • Goal: The overarching goal here is to ensure that Usabilla’s customers feel satisfied and secure whilst using Usabilla’s product.
  • Signals: Positive customer feedback on the feature – through a survey – is a strong signal that Usabilla’s happiness goal is being achieved.
  • Metrics: Measuring the percentage of Usabilla customers that feels satisfied and secure after using the new authentication experience.

The Usabilla example of the HEART framework clearly shows the underlying method of taking a fuzzy goal and breaking it down into something which can be measured more objectively.

Main learning point: The HEART framework is a useful tool when it comes to understanding and tracking the customer impact of your product. As with everything that you’re trying to measure, make sure you’re clear about what you’re looking to learn and how to best interpret the data. However, the fact that the HEART framework looks at aspects at ‘happiness’ and ‘engagement’ makes it a useful tool in my book!

Related links for further learning:

  1. https://www.interaction-design.org/literature/article/google-s-heart-framework-for-measuring-ux
  2. https://www.dtelepathy.com/ux-metrics/
  3. http://www.rodden.org/kerry
  4. https://medium.com/uxinthe6ix/how-we-used-the-heart-framework-to-set-the-right-ux-goals-4454df39db94
  5. https://library.gv.com/how-to-choose-the-right-ux-metrics-for-your-product-5f46359ab5be
  6. https://www.appcues.com/blog/google-improves-user-experience-with-heart-framework
  7. https://clevertap.com/blog/google-heart-framework/
  8. https://medium.com/@dhruvghulati/google-s-heart-framework-a-critical-evaluation-a6694421dae
  9. https://gofishdigital.com/heart-framwork-plan-track-measure-goals-site/
  10. https://www.youtube.com/watch?v=vyZzbsL_fsg
  11. http://www.jjg.net/elements/
  12. https://usabilitygeek.com/combining-heart-framework-with-goals-signals-metrics-process-ux-metrics/

Book review: “The Fearless Organization” by Amy C. Edmondson

Before you read this review of “The Fearless Organization” by Amy Edmondson, I’d encourage you to watch Amy’s Tedx Talk in which she talks about how to build psychological safety. Edmondson is a management professor at Harvard Business School and has done a tremendous amount of work in the area of psychological safety.  In her Tedx Talk, she describes psychological  safety as “a shared belief that the team is safe for interpersonal risk taking.” I believe that psychological safety is a critical yet often overlooked concept, and one which underpins Edmondson’s latest book, The Fearless Organization – Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth.

 

 

These are the things that I took away from reading The Fearless Organization:

  1. Starting with “Personal and Organizational Change through Group Methods” – The aforementioned concept of psychological safety dates back to a 1965 book titled “Personal and Organizational Change through Group Methods” by Edgar Schein and Warren Bennis, which addresses the need for psychological safety for to help people cope with the uncertainty and anxiety of organizational change. Schein later noted that psychological safety was vital for helping people overcoming the defensiveness and “learning anxiety” they face at work, especially when something doesn’t go as they’d hoped or expected.
  2. Psychological safety isn’t a personality trait or difference – Based on her extensive research, Edmondson observes that psychological safety “is not a personality difference but rather a feature of the workplace that leaders can and must help create.” This observation made me think about the conditions that leaders can and must ‘enable’ to create a culture of psychological safety within the organisation, establishing a climate that supports learning. Edmondson mentions a number of other things which psychological safety is not, and which I’ve captured in Fig. 1 below.
  3. Measuring psychological safety – Perhaps you think of psychological safety as quite a fluffy idea, but it can actually be measured. I like the seven survey items which Edmondson introduced in her original research dissertation and which I’ve included in Fig. 3 below. She uses a seven-point Liker scale to obtain responses (from strongly agree to strongly disagree), and three out of seven items are expressed positively. Agreement with these items indicates greater psychological safety, whilst those items items expressed negatively (highlighted with an “R” for reverse), such that disagreement is consistent with higher psychological safety.
  4. Adopting an agile approach to strategy – I loved Edmondson’s point about viewing a company strategy as a hypothesis, to be tested continuously, rather than a plan. This perspective ties in with Edmondson’s broader theme around organisational learning. She argues that organisational learning – championed by company leaders but enacted by everyone – requires actively seeking deviations that challenge the assumptions underpinning a current strategy.
  5. Set the stage for psychological safety – In the book, Edmondson offers some useful tips with respect to ‘facilitating’ psychological safety, sharing a valuable toolkit (see Fig. 4 below).
  6. Proactive inquiry – Being able to say that you don’t know and driving participation through inquiry are two strong tenets of psychological safety. Edmondson shares some rules of thumb for asking a good question: one, you don’t know the answer; two, you ask questions that don’t limit responses to Yes or No, and three, you phrase the question in a way that helps others share their thinking in a focused way (see also Fig. 5 below).

Main learning point: In “The Fearless Organization”, Edmondson has written a valuable book about psychological safety. Even if you’re unable to create a truly fearless organisation anytime soon, Edmondson offers a number of valuable starting points with respect to critical aspects such as questioning, conflict and speaking up.

 

Fig. 1 – What Psychological Safety Is Not – Taken from: Amy Edmondson, The Fearless Organisation, pp. 15-19

  • Psychological safety isn’t about being nice – Working in a psychologically safe environment doesn’t mean that people always agree with one another for the sake of being nice. It also doesn’t mean that people offer unequivocal praise or unconditional support for everything you have to say. Psychological safety is about candour, about making it possible for productive disagreement and free exchange of ideas. Conflict inevitably arises in any workplace. Psychological safety enables people on different sides of a conflict to speak candidly about what’s bothering them.
  • Psychological safety isn’t a personality factor – Some have interpreted psychological safety as a synonym for extroversion. They might have previously concluded that people don’t speak up at work because they’re shy or lack confidence, or simply keep to themselves. Psychological safety, however, refers to the work climate, and climate affects people with different personality traits in roughly similar ways. In a psychologically safe climate, people will offer ideas and voice their concerns regardless of whether they tend to toward introversion or extroversion.
  • Psychological safety isn’t just another word for trust – Although trust and psychological safety have much in common, they aren’t interchangeable concepts. A key difference is that psychological safety is experienced at a group level. Further, psychological safety describes a temporally immediate experience.
  • Psychological safety isn’t about lowering performance standards – Psychological safety is not an “anything goes” environment where people aren’t expected to adhere to high standards or meet deadlines. It isn’t about becoming “comfortable” at work (see Fig. 2 below). Psychological safety enables candour and openness and, as such, thrives in an environment of mutual respect.

 

Fig. 2 – How psychological safety relates to performance standards – Taken from: Amy Edmondson, The Competitive Imperative of Learning, https://hbr.org/2008/07/the-competitive-imperative-of-learning

 

 

Fig. 3 – A survey measure of psychological safety – Taken from: Amy Edmondson, The Fearless Organisation, p. 20

  1. If you make a mistake on this team, it is often held against you. (R)
  2. Members of this team are able to bring up problems and tough issues.
  3. People on this team sometimes reject others for being different. (R)
  4. It is safe to take a risk on this team.
  5. It is difficult to ask other members of this team for help. (R)
  6. No one on this team would deliberately act in a way that undermines my efforts.
  7. Working with members of this team, my unique skills and talents are valued and utilised.

 

Fig. 4 – The leader’s tool kit for building psychological safety – Taken from: Amy Edmondson, The Fearless Organisation, p. 159 

Setting the stage:

Leadership tasks:

  • Frame the work – Set expectations about failure, and interdependence to clarify the need for voice
  • Emphasise the purpose – Identify what’s at stake, why it matters, and for whom

Accomplishes:

  • Shared expectations and meaning

Inviting participation:

Leadership tasks:

  • Demonstrate situational humility – Acknowledge gaps
  • Practice inquiry – Ask good questions and model intense listening
  • Set up structures and processes – Create forums for input and provide guidelines for discussion

Accomplishes:

  • Confidence that voice is welcome

Responding productively

Leadership tasks:

  • Express appreciation – Listen, acknowledge and thank
  • Destigmatise failure – Look forward, offer help. Discuss, consider and brainstorm next steps
  • Sanction clear violations

Accomplishes:

  • Orientation toward continuous learning

 

Fig. 5 – Attributes of a powerful question – Taken from: Amy Edmondson, The Fearless Organisation, p. 171

  • Generates curiosity in the listener
  • Stimulates reflective conversation
  • Is thought-provoking
  • Surfaces underlying assumptions
  • Invites creativity and new possibilities
  • Generates energy and forward movement
  • Channels attention and focuses inquiry
  • Stays with participants
  • Touches a deep meaning
  • Evokes more questions

 

Related links for further learning:

  1. https://www.businessinsider.com/amy-edmondson-on-psychological-safety-2015-11
  2. https://www.tandfonline.com/doi/abs/10.1080/00207284.1967.11642993
  3. https://hbr.org/2008/07/the-competitive-imperative-of-learning
  4. https://marcabraham.com/2017/08/17/book-review-radical-candor/
  5. https://www.huffingtonpost.com/matt-tenney/be-a-dont-knower-one-of-e_b_7242468.html
  6. https://hbr.org/2014/07/the-fukushima-meltdown-that-didnt-happen
  7. https://www.mindtheproduct.com/2018/05/how-to-improve-your-teams-conflict-competence-by-julia-whitney/
  8. https://marcabraham.com/2018/03/12/book-review-the-no-asshole-rule/