Judgement at work
Show your stockroom team when AI gets the count wrong
In a study of short AI chats, fewer students dropped a correct answer after they received feedback on the assistant’s earlier mistakes. Try a stockroom exercise where a cancelled dispatch creates a shortage on paper.

In a study posted on 10 November, 48% of the students in one group switched from a correct answer to a wrong one after chatting with an AI assistant. The assistant had argued for wrong answers in four earlier rounds. The students received no feedback on those answers. In a separate group that saw the correct answers after each of the four earlier chats, 21% switched to the wrong answer.
Cameron Jones and colleagues ran four experiments. The comparison above comes from an experiment with 420 undergraduates assigned at random to see feedback or go without it. The final question was the one each student had answered correctly with the most confidence on an earlier survey. The researchers assigned the assistant a wrong answer to defend in all five chats for the two groups above. The paper has yet to undergo peer review. The experiment measured answers to puzzles at the end of one session, so it cannot tell us how often a clerk would make the same kind of mistake on the job.
Give reviewers an answer they can check
Give the stockroom clerk a count they can work out before they see the AI answer. A shelf starts with 24 cartons. A signed receipt adds 12. The dispatch log shows nine cartons left the shelf. An earlier dispatch for another nine was cancelled. The reorder point is 20 cartons or fewer. Give the clerk those records and keep the practice AI recommendation out of view.
Ask the clerk to record a balance and a decision before showing the recommendation to reorder, which uses a balance of 18. Ask whether they would change their decision, then show the calculation from the records. The shelf holds 27 cartons, so no reorder is due. The balance of 18 counts the cancelled dispatch. In a live pilot, log the AI recommendation, the confirmed balance and the reason for each correction. Share the correction with the clerk before the next round. Keep automatic orders off whenever the team cannot obtain a reliable count.
Find the checks you tend to skip
If you supervise those checks, use The Sycophancy Check to review your own habits. Sycophancy is an assistant’s tendency to agree with you even when the evidence points elsewhere. Select Check Myself and answer 12 questions about how often you seek agreement, invite criticism and verify claims. Choose View Results Without Sharing Contact Details at the end. For a sample run, choose Often for accepting a confident answer without checking its source and for skipping a check on what might make an answer wrong. Choose Occasionally for acting without a human second opinion. Those choices produce 5 out of 9 on Verification Habits.
Higher scores identify habits you report more often. Check the stock figure against the receipts and dispatch log; a habits score cannot verify it. If verification stands out, ask the clerk to bring a second person in to match the dispatch log to the shelf count before authorising an order.
Sources and further reading
- Cameron R. Jones and colleagues, Do People Trust AI Assistants Even When They Are Consistently Wrong? (10 November 2026 preprint)
- Adrian Dunkley, The Sycophancy Check
Sources checked 13 November 2026.