CX Users Adapt to Lower-than-Expected AI Performance

Customer experience (CX) vendors often overpromise and underdeliver on their artificial intelligence (AI) capabilities, according to the author of a recent study.

Utterance recognition rates and containment rates – key performance indicators for AI-based CX platforms – often fall woefully below business's expectations, according to John Triano, executive vice president of sales and business development at Remend and author of the Remend 2023 AI Self-Service Automation Report and Buyer’s Guide.

Remend's John Triano

Triano, a longtime sales and channel leader at EarthLink, Plum Voice, 8x8, Inference and Five9, spent six months interviewing providers and customers to chart the market. He said his conversations revealed a common theme of lower-than-expected AI performance.

"Some [vendors] were promising 65-70% containment rates. Some people were promising 80%; some people were promising 90%," Triano told Channel Futures. "And these companies were finding that they were lucky to get 25-30% containment rate, even after massaging it considerably."

What Are Utterance and Containment Rates?

Triano defines an utterance as "whatever a user communicates to the virtual assistant (VA)." A containment rate by extension is the percentage of times a virtual assistant can handle the user's problem without the help of a live, human agent.

Moreover, those customers have often elected to switch platforms when the first one underperforms. These products include large, integrated platforms from contact center and unified communications providers, and standalone AI tools.

Technology advisors who help businesses source these platforms see the same trend. And they believe they can help their customers navigate the myriad of vendor promises.

CXponent's Joe Rice

"This happens all the time," CXponent CEO Joe Rice told Channel Futures. "This is a gap that agents/partners need to fill ... to set expectations. I believe that vendors have real examples of 70% containment rates, but those are exceptions over what most customers can achieve right away. Or maybe they achieved 70% over a longer period of time."

Are vendors intentionally overplaying their capabilities? Are utterance and containment rates the best way to measure AI performance in CX? And how are rapid advancements in generative AI shaping platform churn?

Channel Futures spoke to Triano, Rice, Telarus CX leader Sam Nelson, CX Effect CEO Andrew Pryfogle and Rise Technology Advisors co-founder Eric Ludwig about these questions. Read their commentary in the eight images above.