Language and display
BE THE BOSS OF AI.ADRIAN DUNKLEY ↗
← This newspaper edition

AI Found the Right Topic. Did It Find the Answer?

A volunteer list cannot tell you who is staffing an entrance. Before acting on an AI search, check that its source answers your exact question.

A slate-blue tab stops short of a rectangular opening cut into stacked cream card.
The Empty Recess · Editorial artwork · AI Boss Ltd

Suppose an arts festival team asks an assistant whether anyone is assigned to the east gate. It returns a list of volunteers but does not say where anyone is working. If the team reads that result as confirmation the gate is unstaffed, it might divert visitors from an entrance it could have used.

The right topic can lead to the wrong answer

In a 2018 study, Pranav Rajpurkar, Robin Jia and Percy Liang added 53,775 unanswerable questions to a reading test. The questions matched the topic of a Wikipedia paragraph and the paragraph contained something that looked like an answer. One question asked for a sea-level projection in the Fourth Assessment Report, while the passage supplied the Third Assessment Report’s figure.

The test asked whether a system could answer from the supplied paragraph or recognise that the paragraph did not support an answer. An assistant searching your files faces an extra problem: it must first find the document that could answer the question. A match on names or keywords is not enough.

Read the roster before calling the gate unstaffed

Ask the assistant which roster and version it found and which entries it checked. If the assistant cannot show the source, open the current approved roster yourself. If it found only a list of names, look for the gate assignments. If the approved roster explicitly marks the east gate as unstaffed, the assistant can report that finding. If the full roster has no entry for the gate, say so and check with the roster owner whether an assignment was left off or recorded elsewhere.

Before changing the festival’s gate directions, record the roster version and any assignment the roster owner confirms. Mark the east gate’s staffing unresolved while you wait.

Sources and further reading

Sources checked 10 November 2026. Edited and reviewed by AI Boss.