How atSpoke Leverages Innovations Behind GPT-3

A few years ago, when I started working in applying machine learning to text, models could not come close to human performance. Achieving this seemed a remote possibility at the time. Today, that remote possibility is much closer to reality with the arrival of new language models such as GPT-3.

GPT-3 is the latest iteration of deep learning models created by OpenAI. With just a brief prompt from a user, GPT-3 can generate text that’s impressively human-like. It can perform intelligent tasks like writing code based on a short description and answering questions about a passage.

The Transformative Underpinnings

What are the underpinnings of this major milestone in artificial intelligence? GPT-3, as well as its predecessors GPT-2 and GPT, and the BERT family of models are powered by two innovations: Transformer-based architecture and unsupervised pre-training.

Transformer-based neural networks vastly improve the efficiency of how models capture complex relationships between words. The boost in efficiency enables deeper neural networks that could do a much better job of “learning” how language works. First used to improve machine translation (a task at which AI is now achieving close to human accuracy), the Transformer architecture is the foundation for the revolutionary, powerful text models we are seeing today.

Unsupervised pre-training, the second innovation underpinning cutting-edge AI, provides models with an out-of-the-box “understanding” of language. In the pre-training process, BERT and GPT family models “learn” how language works by picking up statistical patterns from massive amounts of training text without any human supervision. This enables them to learn custom language tasks with considerably less training data. Because of unsupervised pre-training, GPT-3 can learn how to write code for website interfaces with only a handful of examples.

Thanks to these dual innovations, Transformer-based models for language are now the central focus in AI research and a major lever in AI applications across industries.

atSpoke’s Neural-Network-Powered Language Models

As a machine learning engineer at atSpoke, I’m excited to integrate the latest advancements in neural-network-powered language models into our service desk solution, distinctive for its robust NLP capabilities. atSpoke’s human-in-the-loop system makes it easy for us to try out the latest models with real user data and build solutions quickly with user feedback.

Below are some of my favorite atSpoke features that are already enabled by the same neural-network-powered language models as GPT-3.

Question Answering

Got a question for your help desk? atSpoke doesn’t stop at search but can actually pinpoint the answer to your question in a knowledge base article with our powerful question answering model. Our state-of-the-art question answering model surfaces information faster without needing to open the article for the answer.

Semantic Search

Have you ever used search on a website and had it turn up no results because you didn’t know the exact right word? Or gotten results for the wrong meaning of a word? You don’t have to worry about that with atSpoke because our models are pre-trained with a strong understanding of language like GPT-3. When you send a request to atSpoke, it understands that a laptop is a type of computer but that a kitchen fork isn’t the same as a Github fork. Or, in the case below, atSpoke understands that “dog” and “animal” mean “pet.”

Fast Learning

Using our massive dataset of helpdesk messages, our models have been fine-tuned to already be familiar with the language in your requests. With only a few examples, atSpoke quickly learns which team to triage tickets to or which agent has the best answer for your question.

Translation-On-The-Fly

Requests in one language and knowledge base articles in another? Or vice versa? No problem. atSpoke is powered by multilingual models and the latest in machine translation.