Large language models (LLM) and generative AI technologies are the latest must-have technologies that vendors are incorporating into CX platforms and underlying tools. Since the general availability of OpenAI’s ChatGPT late last year, the CX community has been seeking ways to harness the power and convenience of the technology, while reducing the likelihood of Microsoft Tay-like missteps.
The demand for utilizing LLMs is overwhelming, partially due to the “wow” factor of LLMs, but concerns remain about the potential for LLMs to return wildly incorrect responses to prompts. “Everybody wants to say that they’re using GPT3 in some way or GPT4 even now,” says Chris Ezekiel, CEO of Creative Virtual, a provider of a conversational AI platform. He notes that organizations are rightfully wary of hallucination, where an AI model generates content that is nonsensical or not supported by the training data.
LLMs are created by training a neural network, which is an extremely complicated type of mathematical function involving millions of numbers that convert an input, such as a sequence of letters, in a prompt, into an output, which is the system’s prediction for the next letter. Through each subsequent round of model training, an algorithm adjusts these numbers to try to improve its guesses, using a mathematical technique known as backpropagation. The process of tuning these internal numbers to improve predictions is what it means for a neural network to “learn.” As such, what a neural network generates are not actually letters but probabilities, which is why a query typed multiple times into a LLM prompt will generate a different answer each time. LLMs are also prone to inventing facts and reasoning incorrectly. Researchers do not yet fully understand how these models generate language, and they struggle to steer their behavior.
Creative Virtual is using a technique described as vector matching to ensure that queries or content that is being processed by LLM engines are only matched to preapproved content within the knowledge base. According to Ezekiel, question-and-answer content is vectorized, or turned into a number with 1,536 dimensions, which correspond to different attributes, which are then used for matching content already included in the knowledge base. This ensures that the risk of inappropriate, incomplete, or simply false information being surfaced is eliminated. The technique is called nearest neighbor matching, and Creative Virtual can not only just match up the nearest match, but also the next nearest neighbors as well, and then provide additional responses that have been vetted and approved.
“What we’re actually doing there is always giving the answer that’s already been signed off,” Ezekiel says, noting that right now, ChatGPT is only being used to match similar content. “The great benefit it has over existing neural networks that do something similar, say like a Google Dialogflow that most people are familiar with, is it doesn’t require any training. The model is already trained and we’re just using its ability to give proximity of language, sentences, and questions to what’s already in the knowledge base. It is called nearest neighbor matching them through these vectors.”
Ezekiel says that Creative Virtual already has one customer in Australia using the technology in a commercial deployment. He adds that while both platform vendors and users are concerned about LLM hallucination, vector matching is a “a zero-risk method for people to get into large language models, and getting some of the big benefits without taking the risk.”
With respect to incorporating LLMs and generative AI into its conversational AI technology, Creative Virtual is taking a measured approach. The company is building native support for LLMs into its conversational AI engine via an API, but initially is only using generative models to suggest specific actions or responses that an agent can take based on that intent.
“What we are seeing is some customers wanting to deploy the generative part in contact centers first,” Ezekiel says. In this scenario, LLM technology can be configured to listen in to ongoing calls between agents and customers, or read live chats or texts, and then suggest possible responses or answers to agents, using a generative approach.
“Obviously there’s less risk there to deploy a generative approach to those answers being given [directly to customers], because you would expect the human in the loop would be able to recognize something that wasn’t quite right before giving it out,” Ezekiel says.
He adds that Creative Virtual eventually will roll out other use cases for LLMs as these models improve, including summarization, clustering/analytics, and goal-driven dialogs, which allow virtual agents to clarify issues, sell, handle objects, and negotiate.
Ultimately, the challenge for both vendors and end-users is to ensure that the rush to deploy LLM and generative AI does not overwhelm the primary mission of any CX function, which is to support customers with the correct and most relevant information possible. While generative AI tech can be deployed today, taking a measured, careful approach that eliminates the risk is often the most prudent approach.
Author Information
Keith has over 25 years of experience in research, marketing, and consulting-based fields.
He has authored in-depth reports and market forecast studies covering artificial intelligence, biometrics, data analytics, robotics, high performance computing, and quantum computing, with a specific focus on the use of these technologies within large enterprise organizations and SMBs. He has also established strong working relationships with the international technology vendor community and is a frequent speaker at industry conferences and events.
In his career as a financial and technology journalist he has written for national and trade publications, including BusinessWeek, CNBC.com, Investment Dealers’ Digest, The Red Herring, The Communications of the ACM, and Mobile Computing & Communications, among others.
He is a member of the Association of Independent Information Professionals (AIIP).
Keith holds dual Bachelor of Arts degrees in Magazine Journalism and Sociology from Syracuse University.
