The Listening Cycle · episode 3
The pack that failed its own audit
Composite scenario. Unofficial analysis for illustration. Not a named firm.
The organisation
A UK NHS community provider — 1,800 staff, four localities. They field in Qualtrics. We do not replace Qualtrics. The June quality committee wants a pack in the Trust’s master. Last year a consultancy sent a deck in which Clinical Support was 54% on the league table and 56% on the cascade. The committee noticed. Trust in the listening died before the recommendations were read.
The constraint
Every number on the slide must match the dashboard. Every quote must be in the file. Fail closed on export and on share. Do not market k-anonymity — the privacy page refuses that guarantee. Complementary suppression is a professional obligation, not a unique claim.
Three decisions
1. Two slides, same people, two numbers is how trust dies
The 54-versus-56 defect is not a rounding story. One path averaged dimension scores; the other pooled items. Thirteen numerical invariants exist because a green build is not evidence. The pack now refuses to download until they agree.
2. A paraphrased quote is an invented ending
Verbatim or it does not ship. A contiguous span of a longer response is allowed if the cut is marked. ‘Managers say they listen, then everyone clapped’ is not in the file. The verifier is a product stamp, not a CLI the consultant might forget.
3. Headline 67% without ± is a pretend census
Finite-population margin of error, n-gate 30, on the stamp the committee can see. Method card: this is OLS, not causal. Pairwise Welch with Bonferroni is not Tukey. Over-claiming the test is the People Scientist smell.
The cycle
| Channel | When | Who fields |
|---|---|---|
| Qualtrics census (theirs) Head of OD | Already fielded — we stitch the workbook | Qualtrics — keep it |
| Quality committee pack Director of Nursing + OD | June, Trust master | Analysis here; export blocked until the stamp passes |
| Failed-audit path (deliberate) Practitioner | Shown once, then the repair | Clinical Support 54 vs 56, plus a caught paraphrase, then the green stamp |
- Week 1 — Stitch the Qualtrics workbook. Do not create a Pindrop campaign.
- Week 2 — Build the pack in the Trust master. Run the stamp. It fails. That is the point.
- Week 2, same afternoon — Repair the 54-versus-56 path. Replace the paraphrased quote with the verbatim span. Stamp passes: 13 invariants, N quotes, headline ± MOE, n-gate 30.
- June committee — The stamp is on the Export step, visible, not a script in a repo.
- July — Four actions as WOOP. Complementary suppression explained at board level without promising k-anonymity.
What we will not ask
- A pack that can download while two slides disagree.
- A quote the file cannot find.
- Tukey on a Welch+Bonferroni pairwise.
- A k-anonymity guarantee the privacy page refuses.
- Causal language on OLS.
90-minute workshop
20 min · Find the 54 and the 56
Two slides, same unit, two numbers. The room has to say out loud which number they would defend. Then we show why both were ‘correct’ locally and neither was safe.
20 min · Catch the paraphrase
A quote with an invented ending. The verifier fails. We put the verbatim span back. Fail closed is the product.
25 min · Read the stamp
13 numerical invariants passed · N quotes verified against source rows · headline 67% ±3.1 (finite-population MOE) · n-gate 30. That sentence is what the committee sees.
25 min · Method cards
OLS is not causal. Welch+Bonferroni is not Tukey. When a 7-point is not eNPS. Write the cards before the pack leaves the building.