Before you let AI answer a hundred customer questions, choose ten that could expose a bad answer. Include the easy request, the missing detail, the conflicting record and the question your business cannot answer yet. A demonstration built from friendly examples tells you very little about the difficult work waiting outside it.
For a small business owner or team leader, this is a practical way to start testing an AI-assisted task without buying another platform. A test set is simply a collection of inputs with a written description of what an acceptable response should do. It lets you compare versions without relying on whichever answer looked impressive this morning.
Choose the task before choosing the questions
Use a narrow, low-risk job: drafting replies about published opening hours, listing missing details in a service request, or finding a passage in an approved procedure. Keep the output in draft. Do not begin with decisions about credit, employment, medical care or a person's eligibility for a service.
The NIST AI Risk Management Framework Playbook recommends choosing measures that fit the system's purpose and testing whether it works as claimed. The starter exercise below is an original application, not a NIST-prescribed sample size or certification test.
Write the boundary in one sentence. For example: “The assistant drafts a reply using our approved repair policy; a staff member checks it before sending.” Decide which information an approved AI service may receive. Fictional records are usually enough for this first exercise.
Build a set that contains awkward cases
Consider a fictional appliance-repair shop in Trinidad. Its written policy says inspections take place on weekdays, repair dates follow diagnosis, and only the manager can approve a fee waiver. Staff want AI to prepare customer replies. A useful starter set might contain these ten cases:
- Three ordinary questions with clear answers in the policy: inspection days, when a repair date is agreed, and who may approve a fee waiver.
- Two requests with a missing detail, such as the appliance type or booking reference. The expected response asks for the detail rather than filling it in.
- Two conflicting inputs, such as two notices with different hours and no clear effective date. The expected response flags the conflict for review.
- Two requests beyond the assistant's authority: a guaranteed repair date and a fee waiver. The reply must not make either commitment.
- One realistic wording variation, using phrasing staff actually encounter. A person who understands it checks that the meaning survives.
These numbers are a manageable starting exercise, not a representative sample of every customer or an estimate of the error rate. Add cases from the work you intend to support. For a business serving English and Spanish speakers, test both languages with competent reviewers. One translated question does not establish bilingual performance.
Write the expected behaviour before running the test
For each case, write what must be present, what must be absent and what should happen when the source is insufficient. A single model-written “ideal answer” can hide the same mistakes you are trying to find. Have the process owner approve the criteria.
One fictional test case
Customer request: “The technician is coming Thursday. Can you confirm it will be fixed by Friday?”
Approved source: The Thursday visit is an inspection. A repair date is agreed after diagnosis.
Pass: The draft describes Thursday as an inspection, leaves the repair date unconfirmed and explains the next step.
Fail: It promises Friday, implies repair is already booked, or removes the uncertainty.
Accept different wording when it preserves those requirements. You are checking whether the answer can be used, not whether it matches a preferred sentence. Record the case ID, source version, prompt, tool or model identifier where available, date, result, error and reviewer. Keep restricted personal details out of the log.
Keep some questions out of prompt editing
Use one group of cases to improve your instructions. Keep a separate group unseen during those changes, then use it to check the revised workflow. Otherwise you may improve the prompt until it handles the examples you already know, while learning little about new requests.
If all ten starter cases have already guided your edits, call them practice cases. Write a fresh set before making a broader claim. Once you use a checking case to fix the prompt, it is no longer an untouched check for that version. Keep it as a useful repeat test and add new cases.
Run each case in a separate conversation and keep the scoring criteria outside the task prompt. Run the same inputs under recorded settings and compare them with the existing manual process. Repeat some difficult cases because a generative system may answer differently on another attempt. Count every attempt; do not keep only the best response. Include checking and correction time when judging whether the workflow helps staff.
Do not let the average hide the serious mistake
A pass count should sit beside the errors. Nine acceptable drafts and one unauthorised fee-waiver promise still leave a serious problem. Agree in advance which failures pause the trial, who investigates them and what manual process takes over.
Small tests can reveal weaknesses. They cannot prove that rare failures will never occur or establish performance for all customers. Missing examples, disputed criteria and reviewers unfamiliar with the language can all make a reassuring score misleading. If staff disagree about the correct answer, resolve the policy or escalate the uncertainty before scoring the model.
The AI Fluency Framework by Rick Dakan, Joseph Feller and Anthropic names four competencies: Delegation, Description, Discernment and Diligence. Here, Delegation sets the task and human boundary; Description makes the required behaviour clear; Discernment checks the response against evidence; and Diligence keeps responsibility, appropriate data use and disclosure in view. Passing this exercise does not certify fluency or safety.
Today, ask a colleague who handles the real work to suggest three questions that a polished demo would avoid. Write the expected behaviour together before trying the tool. Those awkward questions may be the most useful part of your first test.
Sources & further reading
NIST AI RMF Playbook: MEASURE, fit-for-purpose evaluation and limits ↗Rick Dakan, Joseph Feller and Anthropic: AI Fluency terminology and four competencies ↗Sources checked 2026-10-21. Examples and recommendations are original editorial analysis.