CX Users Adapt to Lower-than-Expected AI Performance
Many customers want "plug-and-play" AI capabilities for their customer experience platforms, but they need to play an active role in contextualizing the technology they purchase.
![Containment Rates Containment Rates](https://eu-images.contentstack.com/v3/assets/blt10e444bce2d36aa8/blt9ffa72664bbb5d25/6537d56949585af3353861b6/CX.jpg?width=700&auto=webp&quality=80&disable=upscale)
eamesBot/Shutterstock
If a vendor promises a 70% containment rate, that could mean many things, according to CXponent CEO Joe Rice. The channels through which customers reach out to a business vary, as do the nature of their requests.
"The baseline mix of existing interactions for every customer is unique, which means any vendor claims are without context for any individual buyer," Rice said. "Anyone who’s getting 70% containment probably means they’ve done nothing to improve that for years, or maybe there’s a new 'use case' or call type in their business that’s been a recent development."
Rice said changes in the digital experience – whether on mobile or a website – can impact self-service rates in the most significant way.
"So if a client has already taken the low hanging fruit to take calls away from the call center, those interactions are no longer the denominator in the pool of calls, which takes away a bunch of the 'containment' opportunity," he said.
For example, a customer may reach out to a business to ask for the status of their order. That could be the low-hanging fruit.
"If there’s an easy way to access that info digitally, there are way fewer calls to contain, even when the vendor could contain a really high percentage of 'Where are my order calls'?" he said.
In other words, if one business' contact center platform is getting a lower containment rate than another business,' that doesn't necessarily mean one is performing better than the other. If one business is starting with a more digitally sophisticated foundation, their room for containment rate improvement might be lower.
It's all about context.
Rise Technology Advisors co-founder Eric Ludwig said the size of the vendor's platform might impact the grandeur of its promises to the customer.
"I feel like the larger platform-based suppliers can be more aggressive with new or 'acquisition' clients as it’s greenfield for them versus optimizing client environments where it becomes a 'write-down' of existing revenue," Ludwig told Channel Futures.
And to some extent, that's the sales playbook for vendors across the technology landscape.
"While a stretch adjacency, we found similar outcomes for movement to next generation data networks (private-public), adoption of IP vs. TDM in the phone service space, some 'cloud' vs. on-premise environments as well," Ludwig said.
John Triano, executive vice president of sales and business development at Remend, said many customers have elected to shift their CX platform in light of lower than expected containment rates.
And that's to be expected, given the hefty investments they made in their platform, he said.
"When you see that you spent all this money and got a low containment rate, naturally the customer starts entertaining thoughts of, 'What should we do?' They start engaging with other companies, and the other companies are promising them those things as well," Triano said. "Then they [say], 'Well, let's move our technology to another company.'"
Businesses aren't just considering the money they spend on technology. They also view the poor containment rates in light of the money they're spending on human labor.
“You're at 50% of what you thought you were going to contain; you're still having to send to a live agent, and they're having to perform that work," he said.
Moreover, low utterance recognition rates mean businesses have less content with which to train their AI models, Triano said.
"They'll have a meeting and say, 'OK, here are all these utterances we saw. Here are the number of calls or transactions where we saw this. It equates to 2% of your overall volume.' Is that worth training that AI engine to handle that utterance?” he said.
Triano added that switching platforms takes less time than ever.
For example, he said platforms leveraging AI-based natural language processing (NLP) might have taken 8-12 weeks to deploy a year prior.
Now he's saying those deployments occur in a matter of seconds.
"It's all due to the generative capabilities and the speed at which those large language models are able to learn and tie in that data so quickly," Triano said. "It makes it easier for people to move."
Ludwig agreed with Rice that many customers need to identify the low-hanging fruit in their CX strategies.
"If there isn’t an IVA (intelligent virtual agent) or there is little IVR (interactive voice response), there are immediate opportunities," he said. "I think it’s important to take stock of a client’s engagement within the contact center, understand their journey from call center to contact center, and gauge their workstreams from all channels (web, digital, voice, email, etc.)"
Although businesses might feel more empowered than ever to switch platforms, the best solution might already be sitting in their hands.
"Remember, even a 10% containment rate is a massive savings to a company," CX Effect founder and CEO Andrew Pryfogle said. "A 300-person contact center that pays an average of $40,000 per year, per agent would save over $1.2 million with just a 10% containment rate, not counting the additional gains by driving improved efficiencies with their teams. While AI is driving huge improvements, it’s only going to get better as the models continue to get smarter.”
If a model has the potential to exceed 50% containment rates, that feat will neither occur immediately nor automatically, Pryfogle said.
"Enterprises do need to temper their expectations a bit. AI has tremendous potential to automate so much around the contact center and the digital front door. But it still requires a lot of human intervention to fine tune the models," he said.
Samantha Nelson, vice president of CX for Telarus, echoed the sentiment that businesses need to consider seriously their unique context when selecting a solution.
"Several CX providers have described themselves as AI-based, considering the myriad of AI applications that serve as a powerful catalyst for innovation. However, the true measure of success is going to lie in aligning these AI solutions seamlessly with business objectives," Nelson told Channel Futures. "So while vendors have different AI approaches, it’s important for solutions to strike a synergy that truly transforms customer interactions while propelling organizations towards sustainable growth. There is no one-size-fits-all in the AI space as it relates to CX technology.”
Rice encouraged businesses to evaluate their own "pie chart" of customer calls and customer intents.
In one slice, "easy wins" contain a high number of use cases, such as checking the status of an order. IVR and digital platforms can easily take those problems off the human agent's plate. In a second category, the contact center either encounters inconsistent use cases or a low volume of good use cases. In the third category, certain complex calls "shouldn’t even count as something that can be contained."
"Once you identify those, the client also really needs to be clear on how AI and self-services will be powered by the client data," Rice said. "Where does it rest (systems), what data can we count on to answer customer questions, and then how will we measure effectiveness and the needs for improvement after the initial install?"
What can vendors do to better empower platform users?
Triano encouraged a four-fold plan.
First, a constant evolution of the product set.
"Make sure that you're staying up to date with your competitors, because if you fall behind you're not going to be successful," he said.
He encouraged vendors to provide a plethora of collateral for the partners who help customers procure these platforms.
Similarly, his third encouragement is to push more partner education and training.
"As a service provider, you shouldn't expect the partner just to be a lead source for you. You need to empower them to learn more about your product so they can be a better ambassador of your product," Triano said.
Fourth, providers need to give realistic expectations. If they pitch their product as a plug-and-play solution that will automatically give the customer high containment rates, they're misleading them.
"You want to educate the customer and make them understand that they too have a role in this to play," Triano said. "Utimately, the customer is the expert in your business. They're not the experts in AI, but they are the experts in their business. And they have a role to play in these deployments being successful."
What can vendors do to better empower platform users?
Triano encouraged a four-fold plan.
First, a constant evolution of the product set.
"Make sure that you're staying up to date with your competitors, because if you fall behind you're not going to be successful," he said.
He encouraged vendors to provide a plethora of collateral for the partners who help customers procure these platforms.
Similarly, his third encouragement is to push more partner education and training.
"As a service provider, you shouldn't expect the partner just to be a lead source for you. You need to empower them to learn more about your product so they can be a better ambassador of your product," Triano said.
Fourth, providers need to give realistic expectations. If they pitch their product as a plug-and-play solution that will automatically give the customer high containment rates, they're misleading them.
"You want to educate the customer and make them understand that they too have a role in this to play," Triano said. "Utimately, the customer is the expert in your business. They're not the experts in AI, but they are the experts in their business. And they have a role to play in these deployments being successful."
Customer experience (CX) vendors often overpromise and underdeliver on their artificial intelligence (AI) capabilities, according to the author of a recent study.
Utterance recognition rates and containment rates – key performance indicators for AI-based CX platforms – often fall woefully below business's expectations, according to John Triano, executive vice president of sales and business development at Remend and author of the Remend 2023 AI Self-Service Automation Report and Buyer’s Guide.
![john_triano_remend140x185.jpg john_triano_remend140x185.jpg](https://eu-images.contentstack.com/v3/assets/blt10e444bce2d36aa8/bltd107a8a2666af1da/65b2932a64cf57040aaf91c3/john_triano.jpg?width=700&auto=webp&quality=80&disable=upscale)
Remend's John Triano
Triano, a longtime sales and channel leader at EarthLink, Plum Voice, 8x8, Inference and Five9, spent six months interviewing providers and customers to chart the market. He said his conversations revealed a common theme of lower-than-expected AI performance.
"Some [vendors] were promising 65-70% containment rates. Some people were promising 80%; some people were promising 90%," Triano told Channel Futures. "And these companies were finding that they were lucky to get 25-30% containment rate, even after massaging it considerably."
What Are Utterance and Containment Rates?
Triano defines an utterance as "whatever a user communicates to the virtual assistant (VA)." A containment rate by extension is the percentage of times a virtual assistant can handle the user's problem without the help of a live, human agent.
Moreover, those customers have often elected to switch platforms when the first one underperforms. These products include large, integrated platforms from contact center and unified communications providers, and standalone AI tools.
Technology advisors who help businesses source these platforms see the same trend. And they believe they can help their customers navigate the myriad of vendor promises.
![JoeRice_CXponent123x185.png JoeRice_CXponent123x185.png](https://eu-images.contentstack.com/v3/assets/blt10e444bce2d36aa8/blta1ba59bc4e28d9f5/65b293e4fb5ad7040a994aaf/JoeRice.png?width=700&auto=webp&quality=80&disable=upscale)
CXponent's Joe Rice
"This happens all the time," CXponent CEO Joe Rice told Channel Futures. "This is a gap that agents/partners need to fill ... to set expectations. I believe that vendors have real examples of 70% containment rates, but those are exceptions over what most customers can achieve right away. Or maybe they achieved 70% over a longer period of time."
Are vendors intentionally overplaying their capabilities? Are utterance and containment rates the best way to measure AI performance in CX? And how are rapid advancements in generative AI shaping platform churn?
Channel Futures spoke to Triano, Rice, Telarus CX leader Sam Nelson, CX Effect CEO Andrew Pryfogle and Rise Technology Advisors co-founder Eric Ludwig about these questions. Read their commentary in the eight images above.
Read more about:
AgentsAbout the Author(s)
You May Also Like