AI agents went shopping for TVs and laptops—and messed up nearly 50% of the time, study shows
Source: Fortune ·
In a first of its kind experiment, Gemini and Muse agents scored the highest when it come to carrying out tasks like buying a TV.
From the source
As personal AI agents like Muse or Claude become part of daily life, more people are asking them to carry out shopping tasks. But there’s a problem: the agents aren’t very good at shopping. In a first-of-its-kind study, Visa instructed 22 different agents to purchase three everyday items—a TV, a laptop and a hair appliance—and, while some agents did significantly better than others, the overall success rate was only around 57%. The results of the study are available on a new website by Visa that shows the outcome of 15 separate shopping expeditions by each of the 22 agents. Unsurprisingly, the agents powered by more cutting edge AI models fared best, with the Aragon 4 version of Google’s Gemini and the Wonder version of Meta’s Making notching the highest scores. Both agents made a successful purchase 82% of the time. A Claude agent came in third with a 76% success rate, while an OpenAI agent finished 6th with a 64% success rate. The diagram below from Visa shows the performance of…
BestAIBroker analysis
- Finance-aware sentiment: slightly negative (-0.10 on a -1 to +1 scale).
- Event classification: MACRO.
- Market importance score: 54/100, weighting entity relevance, source credibility, multi-outlet coverage and recency.
- Symbols referenced: V.