The Importance of Tone in AI: A Missed Opportunity
Prioritize tone in AI communication for emotional depth and understanding.
Embrace tone-based AI for richer human-machine interactions. Dive into Anders Hvelplund’s analysis and transform your understanding of emotional communication in the digital age.
Artificial intelligence (AI) has made much progress in recent years. This development has been largely driven by the rise of transformer architecture, a neural network that performs very well on sequential data, such as text, phone calls, videos, etc. Most recently, ChatGPT by OpenAI has stunned the world with its ability to provide coherent responses to even the most specific queries – and in a conversational style that seems all too human. Similarly, we have seen AI becoming visually creative with the advances of tools such as Dall-E and Midjourney. You simply provide a text-based prompt, and these tools will generate fantastic art – no matter how specific or unusual your prompt is.
These advances are dependent on the written word. ChatGPT and Dall-E require written prompts to generate their outputs – and in the case of ChatGPT, they deliver their outputs as written words. This is not coincidental: Almost all the attention in natural language processing (NLP) research has been on words.
The Communicative Value of Tone
Undoubtedly, this narrow focus on text-based AI has led to tremendous development in the field. However, it also means that a critical aspect of human communication has been ignored: tone. Tone, which refers to a message’s emotional or attitudinal quality, is a vital element of human communication. It conveys the speaker’s intentions, feelings, and personality and can significantly impact how the listener receives a message. Consider the following sentence: “I never said she stole the money.” Depending on which word is emphasized, the sentence’s meaning can change significantly.
This lack of attention has led to misunderstandings and inappropriate responses from existing AI solutions – for instance, virtual assistants like Siri and Alexa. We should not be surprised. According to Mehrabian’s model, on the topics of feelings and attitudes, tone accounts for 38% of communication, while body language accounts for 55% and words only 7%. Recent studies by Yale’s Michael Krauss have challenged Mehrabian’s model, finding that body language may be less important than tone. As former FBI hostage negotiator and author of Never Split The Difference, Christopher Voss, says: “Body language and tone of voice – not words – are our most powerful assessment tools.” AI models that do not focus on tone are thus missing the most critical piece in understanding the emotional contents of human conversation.
See More: Ineffective Communication Can Cost Businesses
The Business Case Is a Human Case
The neglect of tone in mainstream AI is not for lack of opportunity. Specifically, we need to take advantage of opportunities to understand sentiment and well-being better, prevent the advance of cognitive diseases, and create emotionally intelligent automation.
For example, contact centers are spending billions of dollars to measure customer sentiment and the well-being of their agents but may need to receive more value for it. Traditionally, they have relied on surveys for these purposes. However, surveys suffer from low response rates and bias. If they look to AI to solve the problem, the current text-based AI solutions have inherent limitations in accurately detecting sentiment and well-being as described above; it is uncommon for customers to directly express their satisfaction with the service they received. However, paying attention to their tone of voice can provide a strong indication.
Moreover, contact centers provide a great example that tone is a relevant measurement of customer satisfaction and a way to improve it. As Alfred Wolfsohn said, “The voice is the muscle of the soul.” Customers will appreciate being met by agents with friendly and engaged tones. However, this need is not serviced by the text-based tools currently in use, which are constrained to teaching agents to follow scripts and use the right words. Moreover, mastering your tone is a universally helpful muscle to build. Therefore, this benefit is also highly relevant outside the call center – it matters in every human interaction.
There is also a significant opportunity to enhance the emotional intelligence of automation initiatives, such as voice assistants. Many studies have shown that humans often need to be more satisfied with the current capabilities of chatbots. Attempting to make chatbots more human-like can actually increase their unpopularity. This may be caused by frustrations that these seemingly human-like bots lack emotional intelligence and, therefore, come across as inconsiderate, insincere, or just downright cold. Incorporating tone technology into automation efforts could be part of the next phase to lift voice assistants out of their current period of stagnation.
Finally, tone can detect a person’s well-being and health in addition to its crucial role in communications. A 2021 survey of the academic literature in this area concluded that speech analytics, in particular acoustic and rhythmic features, is a promising tool for diagnosing the advent of early cognitive decline and Alzheimer’s disease, with “most studies recording diagnostic accuracy of over 88% for Alzheimer’s and 80% for mild cognitive impairment.” As we can see, tone analysis can be incredibly effective in predicting the onset of these diseases. With the increasing prevalence of wearable technology, the time is ripe to implement such analysis on a larger scale.
See More: How to Use AI While Preserving The Human Element
Responsibly Scaling Tone AI
Regarding implementation, tone-based AI has great benefits as it is easier to scale and has better data privacy properties than its text-based counterparts. Unlike text-based approaches, it is not specific to certain languages and scales much better over multiple languages and accents. This means that users of the technology can broaden their potential audience. Similarly, they will not necessarily be required to train models specifically for their contexts, generating vast implementation savings compared to text-based models.
Moreover, tone analysis has superior privacy properties compared to other forms of AI, which is increasingly important today, where data privacy, rightly, is a significant concern for individuals, businesses, and regulators. While AI systems based on visuals or text operate on some of our most personal data (social security numbers, account numbers, etc.), tone analysis focuses on less sensitive data such as pitch and speaking rate. In addition, tone models tend to be smaller than text-based models and can, therefore, be implemented locally on devices such as smartphones, computers, wearables, or robots. This allows individuals and companies to keep their most critical data on their premises and avoid sharing it with third parties, ensuring that what happens on your device truly stays on your device.
Breaking the Tone Barrier
One reason that tone analysis has yet to have a more significant impact is that until recently, models struggled to perform consistently across different conditions (such as accents, background noise, and voice quality). However, the transformer-based architecture that has been successful in text-based AI can also be applied to tone-based AI, resulting in models as impressive as those of ChatGPT and Dall-E. For example, audEERING, a European leader in voice AI, used this architecture to achieve the holy grail in voice AI: accurate prediction of valence or the emotional value (positive/negative) of an experience. Moreover, they demonstrated that these models can now be used across languages and accents in any audio scenario and are ready for the real world. These models are already being implemented in the contact center context.
As we enter 2023, there is thus a big opportunity to focus more on tone-based AI – not only for the research community but also in the world of business and practical affairs. And as text-based AI tools continue to develop and enter the mainstream, it will become increasingly clear that a significant dimension of communication – perhaps the most important – is being entirely sidelined. It’s time to finally give tone the attention it deserves and unlock tremendous value.
How can tone-based communication revolutionize AI interactions? Let us know on Facebook, X, and LinkedIn. We’d love to hear from you!
Image Sorce: Shutterstock
MORE ON CONVERSATIONAL AI
