Learn with AI. Then check what you can do without it.

If AI helps a colleague finish an unfamiliar task, you have evidence that the assisted workflow produced an output. You still need to establish what the colleague learnt. That distinction matters when the person will later have to spot a mistake, explain a recommendation or work through an exception.

In yesterday’s essay on token costs and human capital, the question was what a business loses when it counts documents but overlooks expertise.

The quiz came after the assistant was removed

In Shen and Tamkin’s coding experiment, 52 participants learnt an unfamiliar Python library called Trio. Participants were randomly assigned: one group could use an AI assistant during the coding task; the other could not. Neither group could use AI on the subsequent comprehension quiz. Participants had prior Python experience, so the experiment concerned learning a new library rather than learning to programme from nothing.

The group given AI assistance scored lower on the later quiz. The researchers did not find a statistically significant difference in average task completion time. This short experiment does not establish what happens after months of practice, or whether the same pattern occurs in every occupation.

The researchers also examined different ways participants used the assistant. Some asked conceptual questions or sought explanations while remaining engaged with the code. Those patterns were associated with stronger learning than handing the task over. The patterns were not separately randomised teaching methods, so they suggest approaches worth testing rather than a guaranteed recipe.

Give the learner a fresh case

For a low-risk training exercise, choose a task whose answer an experienced colleague can check. Let the learner use approved AI tools during practice, then give them a different case without the assistant. Ask them to explain a decision and identify an error you have deliberately inserted. Label the inserted error as part of the exercise when reviewing the result.

For example, in a hypothetical invoice-checking lesson, the practice case could involve matching a purchase order to an invoice. The independent case could include an agreed discount that was omitted from the total. A correct total copied from the practice case would tell you little; explaining where the discount belongs would show more about the learner’s understanding.

Protect the time needed to correct a mistake

Record assisted output quality, independent performance and the time spent checking and correcting. These answer different questions. An employee may deliver good assisted work while still needing supervised practice before handling exceptions.

Use the result to decide what to teach next. If a learner misses a discount rule, revisit that rule and test a new example. A training programme that records only completed outputs cannot tell you whether the team is becoming better able to supervise them.

Sources and further reading

• Shen and Tamkin: How AI Impacts Skill Formation, arXiv:2601.20245v1
• Anthropic: How AI assistance impacts the formation of coding skills, 29 January 2026

Sources checked 3 November 2026. Prepared with AI assistance and editorial review.