Work & judgement
GPT-4 helped plan a launch but hurt a brand decision
In a field experiment with consultants, GPT-4 helped with a product launch exercise. On a separate case, consultants with access to the same tool chose the right brand less often.
Fabrizio Dell’Acqua and colleagues reported the experiment in Organization Science in 2026. The study included 758 Boston Consulting Group consultants. The researchers randomly assigned them to two separate sets of exercises. Within each set, participants first completed an assessment without AI and then were randomly assigned to work without AI, with GPT-4, or with GPT-4 and a briefing on how to prompt it. Both sets used the version of GPT-4 available in April 2023.
The footwear exercise included 18 tasks, from coming up with a product to defining the market and drafting launch materials. Participants with GPT-4 completed more of the tasks, and human graders rated their responses more highly. In the brand exercise, participants had to make a recommendation from interviews and financial data. About 85% of the group without AI got the brand right, compared with about 71% in the GPT-4 group and 60% in the group also given the prompting briefing. Even when the recommendation was wrong, graders gave the AI-assisted responses higher marks for coherence and persuasiveness.
