Synthesising Notes Without Cherry-Picking
The quote that supports your plan is the one you will remember. Build a process that outvotes you.
You have done nine interviews. You are writing the summary. One participant said something perfect about the exact feature you were already leaning toward, and you can hear it in your head. That quote is going in the deck. The three people who described a completely different problem are going to become 'some users also mentioned'. You are not being dishonest. You are being human, and the process you are using offers no resistance.
Synthesis is where discovery usually fails, and it fails quietly. Bad interviews are recoverable because you can tell they were bad. Bad synthesis produces a confident, well written summary that is wrong, and nobody can see the fault from the outside. The defence is not willpower. It is structure that makes cherry-picking visible.
Count participants, not quotes
The four ways synthesis goes wrong
- Confirmation. You went in with a hypothesis, and the evidence for it is more memorable than the evidence against it, because it fits a story you already have.
- Vividness. The most articulate participant dominates. Being good at explaining a problem is not the same as having it most, and it correlates with being senior, which skews your sample toward people who do not do the work.
- Recency. Interviews seven, eight and nine outweigh one through three, purely because you remember them better.
- Narrative pressure. You have to present something coherent on Thursday. Messy findings feel like failure, so you tidy them, and tidying is where the contradictions get deleted.
Each of these gets worse the longer you wait. Notes written the same day contain observations. Notes written a week later contain a story you have constructed, with the inconvenient parts already softened. The single highest leverage habit in synthesis is writing up within the hour.
A synthesis process that resists you
- Capture observations, not conclusions
- During the interview, write what happened and what was said, as close to verbatim as you can manage. 'She opened a spreadsheet before answering' and not 'she finds the reporting inadequate'. Conclusions written during a call are guesses with a costume on.
- Separate one observation per note
- One idea per line or card. This sounds fussy and it is the whole mechanism. It lets you group by what was actually said rather than by which interview it came from, and it makes the count honest.
- Group without labels first
- Cluster notes that seem to belong together before you name the groups. Naming first makes you sort into buckets you already believed in, which is confirmation with a whiteboard.
- Count participants, not mentions
- A theme supported by nine quotes from two people is weaker than one supported by four quotes from four people. Write the participant count next to every theme, always, including the ones you like.
- Write down the disconfirming evidence
- Keep a standing section titled 'what did not fit'. Anything that contradicts your emerging story goes there and stays there in the final summary. This is the part people quietly drop, and it is the part that saves projects.
The participant count rule does more work than anything else on that list. It is mechanical, it takes no judgement, and it neutralises the vivid-participant problem automatically. Someone who says the same thing six different ways still counts as one.
Worked example
Northgate, the internal admin panel
Eight interviews with ops staff about a shift handover screen. The PM's draft summary led with a theme called 'staff want richer handover notes', built largely on two long, articulate interviews with team leads. A colleague applied the participant count rule to the raw notes. Richer notes: 2 of 8, both team leads, neither of whom does handovers themselves. A theme nobody had named, roughly 'I cannot tell if the previous shift actually completed the checks or just ticked the box', appeared in 6 of 8, mostly in short, unremarkable asides that had not made the draft because none of them was quotable. The project changed direction. The tell was that the winning theme had no good quote, which is exactly why it had almost been lost.
Cherry-picked synthesis
- Organised by interview, one summary per person.
- Led by the best quote.
- Themes named before grouping.
- Contradictions omitted for clarity.
- Written the week after.
Honest synthesis
- Organised by observation, regrouped across people.
- Led by the theme with the highest participant count.
- Groups formed first, named afterwards.
- A standing 'what did not fit' section in the final doc.
- Written within the hour, every time.
Two practices make this harder to fake. Have someone who was not in the interviews do the grouping with you, because they have no story to protect. And keep the raw notes linked from the summary so anyone can check your counts. You will find that knowing the notes are readable changes how you write the summary, which is the point.
Finally, be willing to publish a summary that says the findings were mixed. There is real pressure to arrive with an answer, and a team that punishes inconclusive research will get conclusive research every time, whether or not the evidence supports it. Saying 'four of eight described X, three described something unrelated, and we need four more interviews in this segment' is a legitimate outcome. It is also the sentence that separates people who do discovery from people who perform it.
Quick check
Why is counting participants per theme better than counting quotes or mentions?
The takeaway
Count participants rather than quotes, group before you name, and keep a permanent section for the evidence that did not fit your story.
Try this tomorrow
Go back to your most recent set of interview notes and rewrite every theme with the number of distinct participants who expressed it. Then add a 'what did not fit' section and put at least one real contradiction in it.
Answer the check above, then bank the day.
Where this comes from
- Continuous Discovery Habits, Teresa Torres
- The Mom Test, Rob Fitzpatrick
- Inspired, second edition, Marty Cagan
Discovery is one of six tracks. These lessons summarise and build on the work above, they do not reproduce it. Buy the books, they are better.