Tech

Dimitri Masin, CEO of Gradient Labs, argues that companies using AI agents for customer support should only pay when the bot does its job.

"If you look at Salesforce, they price the automation per conversation," he told The Register in a phone interview. "So essentially, if you have a conversation with AI, no matter what it leads to, you pay $2."

According to Salesforce's own researchers, leading LLM-based agents tested on the CRMArena-Pro benchmark successfully complete single-turn (prompt and reply) tasks about 58 percent of the time and only about 35 percent of the time for multi-turn (back-and-forth conversation) requests.

That's both bad for customers and bad for the progress of AI agents overall. Paying regardless of results, said Masin, "doesn't create any incentive for Salesforce to actually make their agent better."

Salesforce did not respond to a request for comment. But the stats it touts in its marketing copy are, unsurprisingly, a lot better.