Watch what people do

Warm-up

From last time. Ken says: "Two of our juniors asked to come off the rota." Does each reply follow his thread, or drop it?

1. What happened with those two?

   Follows it Drops it

   Right, it's a reply that follows his thread. It asks what came next. They now eat alone, at a different time.

   Not quite. It's a reply that follows his thread. It asks what came next. They now eat alone, at a different time.

2. Great. How's the food?

   Follows it Drops it

   Right, it's a reply that drops his thread. It changes the subject, away from who drops out.

   Not quite. It's a reply that drops his thread. It changes the subject, away from who drops out.

3. (Say nothing, and wait.)

   Follows it Drops it

   Right, it's a reply that follows his thread. The silence gives him room to go on.

   Not quite. It's a reply that follows his thread. The silence gives him room to go on.

Last time, you learned to follow the thread people hand you. But some things never come up in an interview, however well you listen. People don’t notice them, or they’ve stopped seeing them.

On Thursday, Mia interviewed Sara, who orders lunch for 8 people at a travel agency. Sara said paying is easy. Her trouble is the menu, which she types out every morning. So on Monday, Mia goes back to watch, and takes you with her.

Think first

You’ll sit with Sara from ten in the morning until lunch is paid for.

What would you write down?

What you'd write down Hint: write things you could see, with a number. Watching a friend cook, you might note: Opens the recipe 4 times. Burns the onions at 6:42.

Lock in my answer Skip, show me

You

Which of these is closest to your answer?

Whether she seems stressed.

Shirish

That’s a guess about what’s inside her, a beetle again. Two people watching would write different things.

What would stress look like, that you could see and count?

How long each part takes, and every mistake she makes.

Shirish

Yes. Minutes and mistakes are things anyone watching would agree on.

Which part do you think will take longest?

What she thinks of the menu.

Shirish

She told you that already, in the interview. You’re there to see what she does.

What does she do with the menu, minute by minute?

Your reply

Continue Skip

You

What this means

Write down what you can see, with a number: a time, a count, a mistake. She seems stressed is a guess. She retyped one price is a record.

Reset

What Mia wrote down

Here are Mia’s notes from Monday morning, exactly as she wrote them.

• 10:02. Sara opens Tiffin’s menu on her phone. 9 dishes today.
• 10:03 to 10:17. Types each dish and price into the group chat. 14 minutes. Copies one price wrong, $12 instead of $13.
• 10:18 to 10:55. Replies come in. She checks the chat 6 times. One person changes their order.
• 10:58 to 11:04. Adds 8 meals to the cart, one by one, reading from the chat. Taps the wrong spice level once, and fixes it.
• 11:05. Pays on her card. Sends 8 payment requests. 2 minutes.
• By 12:10. Seven people have paid. The driver will pay cash tomorrow.

Think first

In the interview, Sara said: Easy. I send a payment request and everyone pays within the hour. And: The annoying part is the menu.

What do Mia’s notes add to what Sara said?

You

Pick the answer closest to yours.

Nothing. Sara already told Mia all of it.

Shirish

She told Mia where the trouble is. Did she say how long it takes, or that it causes mistakes?

What’s in the notes that wasn’t in the interview?

Numbers: the menu takes 14 minutes, and copying it causes mistakes.

Shirish

Yes. The interview said where. Watching said how much, and showed the typing causes errors further down.

Did watching agree with her about the money?

That Sara was wrong about the money.

Shirish

Look again. Seven of eight paid within the hour. On the money, she was right.

So what did watching add?

Your reply

Continue Skip

You

What this means

Watching agreed with Sara about the money. And it put numbers on the menu: 14 minutes, one wrong price, one wrong spice level. Sara is a different kind of office. Her money is solved. Her trouble is typing.

Reset

Seen, or guessed?

Practice

Here are notes two people might write while watching Sara. Is each one something you could see, or a guess?

1. She typed the menu for 14 minutes.

   Seen Guessed

   Right, it's something you could see. A time anyone watching would agree on.

   Not quite. It's something you could see. A time anyone watching would agree on.

2. She seems tired of the menu.

   Seen Guessed

   Right, it's a guess. It's a guess about how she feels. What did she do that made you think it?

   Not quite. It's a guess. It's a guess about how she feels. What did she do that made you think it?

3. She copied one price wrong.

   Seen Guessed

   Right, it's something you could see. You could point to it in the chat.

   Not quite. It's something you could see. You could point to it in the chat.

4. She finds the app confusing.

   Seen Guessed

   Right, it's a guess. Nothing she did showed confusion. She only tapped the wrong spice level once.

   Not quite. It's a guess. Nothing she did showed confusion. She only tapped the wrong spice level once.

5. She checked the chat 6 times before 11.

   Seen Guessed

   Right, it's something you could see. A count. It shows how the waiting goes.

   Not quite. It's something you could see. A count. It shows how the waiting goes.

What to watch for

When I planned to watch an office manager at work, I wrote a list before I went.

The last question is the best one. Whether she eats first, or does the money first, shows which job wins when the food arrives. Nobody would ever tell you that in an interview.

To make each one a record, give it a unit. The time she posts. How many reminders she sends. Minutes on the notebook.

You change what you watch

People act differently when someone is watching. They work more carefully, and skip the shortcuts they’d normally take. So how do you see a normal day?

I’d become part of it. In my words: I would order for myself as well, and become part of the office while taking notes from afar. Two more things help:

• Come back more than once. The first day is always the odd one. By the third, you’re part of the furniture.
• Look at what was there before you came. Last month’s chat messages, the notebook’s old pages. Your visit can’t change those.

A five-person test, in a morning

Watching someone do their normal work shows today. To see if a new design works, you watch people try it. That’s a usability test, and it fits in a morning.

1. Write a task, not a question. A goal with a clear end: Order your lunch for tomorrow, and pay for it. Not What do you think of the link?
2. Decide what counts as success before you start. Here: lunch ordered and paid, with no help.
3. Ask them to think out loud as they go. Then stay quiet, even when they struggle.
4. Watch five people. Jakob Nielsen found that five people find most of the big problems. Steve Krug suggests one morning a month.

For more, read the usability testing guide (opens in a new tab). It shows how many people a test needs, and why.

Words from this lesson

Observer effect

People change what they do when they know they’re being watched. In this lessonSara types the menu more carefully because Mia is watching.

Task success

The person reached the end you set, without help from you. In this lessonLunch ordered and paid for, with no help.

Think-aloud

Asking people to say what they’re thinking as they try something. In this lessonOkay, I'm looking for the spice level... where is it?

Back to your first answer

Your first answer

Your answer

Your answer will show here after you write one at the top of this page.

You skipped this at the top. Answer it now, with what you've learned.

You've read the lesson. Would you change your answer?

I'd keep it I'd change it

Good. Which part of this lesson backs your answer?

Mia's notes from Monday Seen, or guessed My list for one team lunch You change what you watch Something else

Why? One line is enough. (optional) What would you say now?

Your bet: what does Nora need from Mia’s plan?

Your bet

Next week, Mia is away. Nora Kim, the growth manager, will run two more interviews, using Mia’s plan.

Mia, research lead, to you: I want Nora to get the same kind of answers I would.

What does Mia’s plan most need? Make your bet, and say how sure you are.

Your guess

Mia's exact questions, and nothing else Who to talk to, what to ask, and what to write down Just the research question

How sure are you?

50% 70% 90%

You'll find the answer in Lesson 3.5: Write a plan someone else could run.

Try this during the week: watch someone you know do a small task on their phone, like paying a bill. Write down three things you saw, each with a number.

Sources

• Steve Krug, Rocket Surgery Made Easy, 2010. A usability test, one morning a month.
• Jakob Nielsen, Why You Only Need to Test with 5 Users, Nielsen Norman Group, 2000. Laura Faulkner’s 2003 study showed five can also miss a lot, so test again.
• The list is from my own practice. Tiffin, Mia, Sara, Ken and Nora are made up for this course.