Assess your AI program

A steel ball shatters a tilting stack of glass blocks above a thin base and a small red wedge.

Find AI tools that staff use without approval

A customer service supervisor approves an AI tool to draft replies without customer details. In this example, staff paste full complaints into the tool. When the tool slows down, staff switch to personal accounts. The company could lose control of the customer records.

Start with one task that handles customer data. Ask a staff member to show you how they completed it. Use sample data for the demonstration.

  1. Record the product and the type of account the staff member used.
  2. Ask what kind of data they entered.
  3. Check whether the approval covers the product, account and task.
  4. Check whether the approval covers the data the staff member entered.
  5. Report any suspected disclosure of customer data to your security team.

Use company purchase records and sign-in records to check what you find. Ask the teams about free tools and features built into software they already use. Record unapproved tools as pending review in the AI product register.

Build an AI product register to record approvals and data rules

An AI product register lists the products your teams use. It shows which tasks and data each approval covers. Use a shared spreadsheet to start. Keep one row for each product and task. Add a separate row if a team uses a different account or needs different data rules.

Fields for an AI product register
What to recordExample or evidence to keep
Product and accountAI writing assistant on a company-managed account.
Owner and taskCustomer service lead owns the product. Staff use it to draft replies.
Data rulesApproved public guidance and sample complaints. No customer records.
Supplier settingsRecord how long the supplier keeps entries. Save evidence of whether the supplier can use them to train its model.
Approval and review dateMark the use as approved, pending review or blocked. Record the approver and link to the decision. Set the next review date.
Person who can pause useCustomer service duty manager. Switch to the approved reply template if the tool fails.

Check the supplier’s terms and account settings before you approve a product. Keep approval pending if you cannot confirm the data rules. Record which staff can access it. Recheck the approval when the supplier changes its terms or the team wants to use new data. The US National Institute of Standards and Technology says, “Risk management should be continuous, timely”. Its framework calls for an inventory of AI systems and more attention as risks rise. NIST AI Risk Management Framework, Govern 1.6 and 6.1.

Check AI replies before customers receive them

An AI draft can give a customer the wrong deadline or promise a service the company does not provide. Name the person who must check each draft. Require that person to compare dates, promises and instructions with an approved source.

Give staff a sample reply with one wrong deadline. Ask the reviewer to find the error before the reply goes out. Record whether they found it. If a reviewer is unavailable, use the approved reply template.

Rank risks by missing checks and possible harm

A control is a rule or a check that helps prevent a problem. The assessment gives each area a score out of 100. Higher scores show where to check first. In the customer reply example, the risk to customer records puts tools and data first. ROI means return on investment.

Customer reply workflow · Example screening scores out of 100
  • Tools and data100
  • Staff training63
  • Inventory and monitoring63
  • Direction38
  • Risk and audit25
  • Value and ROI13

Use the risk matrix to compare gaps in controls with possible harm. A higher gap signal means more checks are missing or lack evidence. A material impact can mean substantial rework or disruption. Check past errors and the number of sensitive cases before you estimate how often a failure might occur.

3 × 3 risk matrix · Customer reply example
Gap signalMinor impactMaterial impactMajor impact
HigherModerateHighInventory and monitoringHighTools and data
ElevatedLowValue and ROIModerateDirectionHighStaff training
LowerLowLowModerateRisk and audit

Staff training and monitoring both score 63 in this example. Training has fewer gaps. A missed check can still let a wrong instruction reach a customer. Monitoring has more gaps. The team rates the harm from those gaps lower.

Choose who can pause AI use and who can approve a restart

Use a risk register to record what could go wrong and who must act. Record one risk per row. Name the person who owns the risk. Set a date to review the test results.

Mini risk register · Customer reply example
RiskSuggested ownerActionEvidence to review
Staff paste customer details into a personal account.Information security leadCheck which accounts staff use. Limit customer data to tools your team approves for it.Approved data rules and a test with sample data.
An AI draft gives a customer the wrong instruction.Customer service leadTeach staff which replies need a second check.A test reply with a known error and a record of who found it.
Teams adopt new tools or tasks without review.AI program ownerCheck a sample of team tasks against the AI product register.A list of exceptions and one change the risk team reviewed.

Set a clear rule for when to pause a task. For example, stop AI drafts if staff send customer data to an unapproved account. The duty manager switches the team to its approved reply template. Send the incident details to your security team.

Name the person who can approve a restart. Test the pause on a sample task. Check that staff can still serve customers during the pause. Restart only after the owner can show that staff fixed the problem and passed a new test. Record who approved the restart. NIST AI Risk Management Framework, Manage 2.3 and 2.4.

Count review and correction time before you report AI savings

Suppose staff handle 120 replies a month. Each reply took 22 minutes before the team used AI. With AI, it takes 12 minutes, including the time to check and correct the draft. At 10 minutes per reply, the team gains 20 hours a month for other work.

At J$2,500 an hour, those hours are worth J$50,000 if the team uses them for other work. Subtract J$25,000 in monthly tool and training costs to get J$25,000 in net value. Divide the net value by the cost. Multiply by 100 to get the percentage return. In this example, the return is 100%.

Measure the same type of reply before and after introducing AI. Compare the share of replies that contain errors. Check that spending fell before you report cash savings. Use the timesheets and error records when you decide whether to use AI for more tasks.

Bring one product register row and one test result to your next team meeting. Ask the team to name the person who will fix the gap. Set a date to check the fix.

Sources & further reading

NIST AI Risk Management Framework 1.0 ↗NIST AI Risk Management Framework Playbook ↗Adrian Dunkley, CAIRMC — AI Risk in the Caribbean ↗Adrian Dunkley — FAIR-AI puts usage and oversight into the loss estimate ↗Adrian Dunkley — AI Governance Comes Down to 25 Documents You Either Have or You Do Not ↗

Sources checked 2026-11-18.