Write a readout that carries a decision

Warm-up

From last time. Which trap could each number fall into?

1. Conversion rose, while phones and computers both fell.

   Segment mixing Who's missing

   Right, it's segment mixing. The mix of visits changed, not each group.

   Not quite. It's segment mixing. The mix of visits changed, not each group.

2. Current users love the group cart.

   Segment mixing Who's missing

   Right, it's survivorship bias. Dana stopped using it. She's not a current user, so you'd never hear her.

   Not quite. It's survivorship bias. Dana stopped using it. She's not a current user, so you'd never hear her.

3. Average lunch price rose after more big offices joined.

   Segment mixing Who's missing

   Right, it's segment mixing. Each office may pay the same as before. The mix changed.

   Not quite. It's segment mixing. Each office may pay the same as before. The mix changed.

Last time, you learned to read numbers before trusting them. Now it’s Thursday. Mia has to give Peter, the head of sales, one page by Friday. Everything the team learned since his email, and what to do next.

She’s written the finding. One line is still missing.

Think first

Mia’s finding: Office managers who pay for colleagues’ lunch need to stop being owed money.

Finish her next line: We’d be wrong about this if…

Your line Hint: name something you could see in the people the finding is about. Say the finding is Members quit because classes are full. You might write: We'd be wrong if members who get a place quit just as often.

Lock in my answer Skip, show me

You

Which of these is closest to your answer?

...the office pays for the lunch.

Shirish

That’s what I said, when I was asked this. But the finding is already about managers who pay. Ray’s bank was never part of it.

So this says who the finding is for. Does it say what would prove it wrong?

...managers who get paid at once still keep their notebook.

Shirish

Yes. That’s something you could see, in the very people the finding is about. If it happened, the finding would be wrong.

What else could the next test show, that would go against it?

...Peter disagrees with it.

Shirish

Peter might. But his opinion is rung 0. Would it change what’s true for Anna?

What would you need to see, among the managers themselves?

Your reply

Continue Skip

You

What this means

There’s a difference between saying who a finding is for and saying what would prove it wrong. If the office pays is the first kind. It’s useful as a size check: if most offices are company-paid, the finding is true but small.

The line Mia needs is the second kind. Something you could see, among the people the finding is about, that would go against it. Most readouts leave it out. That was your bet.

Reset

Mia’s readout

Here’s the page Mia gives Peter on Friday. It fits on one screen.

• Question. Should Tiffin build group ordering this quarter?
• Who we heard from. Five office managers and one colleague, one morning watching Sara, and 40 help requests from July to September.
• Groups. One person pays and chases: Anna, Dana. Take turns and settle monthly: Ken. The company pays: Ray. Each pays by request: Sara.
• Finding. The need isn’t group ordering. Managers who pay for colleagues’ lunch need to stop being owed money. Three of five are owed money, and Anna is $157 short this month. One office gave up: I stopped ordering for everyone in August. (Dana). Seven of ten group help requests ask about money.
• The case that doesn’t fit. Ray’s bank pays for lunch. His need is a daily tax invoice, not getting paid back.
• How sure. Rung 2 to 3: six people telling us what they do, and one morning of watching.
• Next test. A link where each person pays for their own meal within 30 minutes. Three offices where one person pays, or they take turns. One week. We’re wrong if fewer than 6 in 10 pay in time. At 6 in 10, Anna chases 4 people instead of 11. We also count who misses lunch. We decide on Friday, 23 October.
• What would change this. Managers who can be paid instantly keep their notebook anyway. Or the test offices end the week owed as much as before. And a size check: most offices turn out to be company-paid.
• Don’t lose. Ben’s peanut allergy at hand-out. Rare and serious, and nothing above solves it.

Look at where this started. Peter asked for group ordering in lesson 2.1. The research found a different need, and sized the next step to the proof. And it set a date to look again: 23 October, from lesson 2.4.

Proves it wrong, or says who it’s for?

Practice

Each line finishes "We'd be wrong if...". Could it prove Mia's finding wrong, or does it only say who the finding is for?

1. The office pays for lunch.

   Could prove it wrong Says who it's for

   Right, it's a line that only says who the finding is for. The finding is only about managers who pay. This just repeats that.

   Not quite. It's a line that only says who the finding is for. The finding is only about managers who pay. This just repeats that.

2. The test offices end the week owed as much as before.

   Could prove it wrong Says who it's for

   Right, it's a line that could prove the finding wrong. If paying for yourself doesn't stop the debts, the finding is wrong.

   Not quite. It's a line that could prove the finding wrong. If paying for yourself doesn't stop the debts, the finding is wrong.

3. The office is a bank.

   Could prove it wrong Says who it's for

   Right, it's a line that only says who the finding is for. A kind of office, not a result. Ray's bank was never part of it.

   Not quite. It's a line that only says who the finding is for. A kind of office, not a result. Ray's bank was never part of it.

4. Managers who can be paid instantly still chase people.

   Could prove it wrong Says who it's for

   Right, it's a line that could prove the finding wrong. Then the money wasn't really the need.

   Not quite. It's a line that could prove the finding wrong. Then the money wasn't really the need.

Keep it where others can check it

A readout is only as good as what’s behind it. So Mia keeps it all in one shared folder, linked from the page:

• The plan, from lesson 3.5.
• The notes and recordings from each interview.
• The coded cards, and the counts by person.
• The readout itself.

A shared place like this is a research repository. It means anyone can check a claim, and the next person can run the same study. That’s the last step of the check from lesson 3.5. Could a stranger repeat this, and get the same kind of answer?

Words from this lesson

Readout

One page that turns research into a decision, with its proof and its limits. In this lessonMia's page: question, who, finding, how sure, next test, what would change it.

Research repository

One shared place for a study’s plan, notes and findings, so anyone can check them. In this lessonMia's shared folder: the plan, the notes, the counts and the readout.

Back to your first answer

Your first answer

Your answer

Your answer will show here after you write one at the top of this page.

You skipped this at the top. Answer it now, with what you've learned.

You've read the lesson. Would you change your answer?

I'd keep it I'd change it

Good. Which part of this lesson backs your answer?

Who it's for, or what proves it wrong Mia's readout Proves it wrong, or says who it's for Keep it where others can check it Something else

Why? One line is enough. (optional) What would you say now?

Your bet: what is Peter doing?

Your bet

Peter reads Mia’s page on Friday. He nods.

Peter, head of sales: Getting paid back. I knew it all along, really.

What’s going on? Make your bet, and say how sure you are.

Your guess

He really did know all along Now that he knows, it feels obvious to him He hasn't read the readout

How sure are you?

50% 70% 90%

You'll find the answer in Chapter 5: Biases: theirs and yours.

Try this during the week: take a recent decision from your team. Add the line it’s missing: “We’d be wrong if…”

Sources

• Erika Hall, Just Enough Research, second edition, 2019. On writing research up so it gets used.
• Tomer Sharon, Validating Product Ideas, 2016. On research repositories.
• Tiffin and its people are made up for this course.