CX Users Adapt to Lower-than-Expected AI Performance

Many customers want "plug-and-play" AI capabilities for their customer experience platforms, but they need to play an active role in contextualizing the technology they purchase.

James Anderson, Senior News Editor

January 25, 2024

8 Slides
Containment Rates
eamesBot/Shutterstock

Customer experience (CX) vendors often overpromise and underdeliver on their artificial intelligence (AI) capabilities, according to the author of a recent study.

Utterance recognition rates and containment rates – key performance indicators for AI-based CX platforms – often fall woefully below business's expectations, according to John Triano, executive vice president of sales and business development at Remend and author of the Remend 2023 AI Self-Service Automation Report and Buyer’s Guide.

john_triano_remend140x185.jpg

Triano, a longtime sales and channel leader at EarthLink, Plum Voice, 8x8, Inference and Five9, spent six months interviewing providers and customers to chart the market. He said his conversations revealed a common theme of lower-than-expected AI performance.

"Some [vendors] were promising 65-70% containment rates. Some people were promising 80%; some people were promising 90%," Triano told Channel Futures. "And these companies were finding that they were lucky to get 25-30% containment rate, even after massaging it considerably."

What Are Utterance and Containment Rates?

Triano defines an utterance as "whatever a user communicates to the virtual assistant (VA)." A containment rate by extension is the percentage of times a virtual assistant can handle the user's problem without the help of a live, human agent.

Moreover, those customers have often elected to switch platforms when the first one underperforms. These products include large, integrated platforms from contact center and unified communications providers, and standalone AI tools.

Technology advisors who help businesses source these platforms see the same trend. And they believe they can help their customers navigate the myriad of vendor promises.

JoeRice_CXponent123x185.png

"This happens all the time," CXponent CEO Joe Rice told Channel Futures. "This is a gap that agents/partners need to fill ... to set expectations. I believe that vendors have real examples of 70% containment rates, but those are exceptions over what most customers can achieve right away. Or maybe they achieved 70% over a longer period of time."

Are vendors intentionally overplaying their capabilities? Are utterance and containment rates the best way to measure AI performance in CX? And how are rapid advancements in generative AI shaping platform churn?

Channel Futures spoke to Triano, Rice, Telarus CX leader Sam Nelson, CX Effect CEO Andrew Pryfogle and Rise Technology Advisors co-founder Eric Ludwig about these questions. Read their commentary in the eight images above.

Read more about:

Agents

About the Author

James Anderson

Senior News Editor, Channel Futures

James Anderson is a senior news editor for Channel Futures. He interned with Informa while working toward his degree in journalism from Arizona State University, then joined the company after graduating. He writes about SD-WAN, telecom and cablecos, technology services distributors and carriers. He has served as a moderator for multiple panels at Channel Partners events.

Free Newsletters for the Channel
Register for Your Free Newsletter Now

You May Also Like