The Personalized AI Assistant Is at Your Service
Find out what an AI-powered augmented reality assistant can do for you beyond answering questions and making lists!
Kelly Peng, CEO and CTO of Kura Technologies, writes about how AR overlays and intelligent responses empower personalized AI assistants for users. AI redefines how we interact by simplifying mundane tasks to enhance immersive experiences.
In 2022, Jessica Lis of Emarketer estimated that there will be over 145 million voice assistant users in the United States by the end of 2023. Big tech companies have taken full advantage of the fact that most people engaging with voice assistants (about 88%) use them on their smartphones, adding features that allow you to set a timer or play a song from across the room simply by asking for it.
However, as convenient as these voice assistants are, they are nowhere near as capable as a true AI-powered virtual assistant could be. It’s been estimated that over 80% of information processed by the brain is derived from visual cues, and a voice-only assistant leaves a lot of interactive potential on the table.
For example, asking Siri to show you how to change your car’s oil isn’t going to do the trick; you’d have to go to YouTube or find a website tutorial to get the answer. Thanks to multimodal AI and the growing capabilities of augmented reality, that won’t be the case for much longer.
Multimodal AI and AR: The Personalized Assistant
The next generation of virtual assistants will feel familiar, like Siri and Alexa, but capable of giving and receiving information in multiple ways. With augmented reality’s ability to produce visual overlays on top of the real world, you see through a pair of AR glasses, we’ve reached the point where AI-powered assistants can display images, text, video, and other information to the wearer on demand in real-time.
1. Vision and display
Instead of asking an old-school assistant like Siri how to change the oil in your car and then clicking through an article on your phone or skipping through a YouTube video to find the information you need, an AR-based AI assistant can walk you through the process as though they are standing in the room with you.
Modern AR glasses equipped with eye tracking and outward-facing cameras can leverage AI object recognition to point to people, places, or objects in the real world using the digital overlay.
In the oil-changing example, the user could be walked through the maintenance process as the assistant highlights individual engine components to show the user what needs to be removed or replaced. And, because each individual makes and model of a car is slightly different, the information provided can be customized based on what the AI is “seeing” through the camera on the glasses.
An oil change is just one of countless examples, and you can easily imagine an AI-powered augmented reality assistant teaching you the step-by-step process of baking a loaf of bread, pruning a plant, or setting up a new computer, among many other use cases.
2. Generative properties
As many users know, an answer to one question sometimes leads to additional questions. The personalized assistant can provide in-depth answers to those follow-ups or show additional information — like context or visual examples — to clarify its point, adding to the same conversation.
Personalized assistants can also use multimodal AI to become partners in brainstorming ideas. Have you ever asked your smartphone an open-ended question like, “What can I get my mom for my birthday?”
If you have, you know that the response is typically a link to a website you may or may not have ever heard of. A more intelligent AI assistant digests a much larger volume of information to provide you with answers that may be more recent and recommended by multiple people rather than just one product list found via a web search.
See More: Breaking New Ground: A Dive Into Multimodal Generative AI
Personalized Assistant Use Cases
Multimodal AI with augmented reality is the absolute cutting edge of these emerging technologies, and there are several pressing use cases for truly digital personal assistants that will change the ways users do everything from business to travel to face-to-face interaction:
- Manufacturing: Personalized assistants work in multimodal AI to pull up the information needed, then generate step-by-step training guides, standard operating procedures, and other documentation in the AR display. Personalized assistants can also generate scenarios based on the user’s preferences for more in-depth training on rarely needed or often forgotten tasks.
- Tourism: From live translations of one language to another, location mapping when actively on a hike or similar adventure, to restaurant recommendations, personalized assistants engage with the real world around users to enhance already immersive experiences within AR.
- Business: Personalized assistants can act as designated note-takers, bring people from all over the globe into a virtual meeting room, and manage multiple agendas and projects simultaneously.
- Retail: For users who want to know what sneakers someone is wearing and how to buy them, personalized assistants can leverage AR and multimodal AI to image those shoes (or a purse, coat, hat, etc.) and find exactly where and how they can be purchased.
Safeguarding Users While Enhancing Their Lives
As exciting as these developments are, caution is always wise regarding implementation. Like all smart devices, certain protocols could be harmful if a user’s data is breached.
With AR, the stakes could be higher, as hackers who breach the personalized assistant could gain access to what the user is doing, their location, preferences, and other private information.
Some important consumer protection notes:
- Be aware of what you’re asking of your personalized assistant: Anything you input into or create with any multimodal AI tool — including a personalized assistant — could be used to refine the AI further. It pays to exercise an abundance of caution when considering what you want to ask your assistant for help.
- Explore the settings: Multimodal AI comes with security and privacy controls consumers can use to moderate and modulate their experience and ensure they’re comfortable with and consent to using their data and information.
- Be mindful of reality: A reining concern around AR data privacy and security is a hacker’s ability to alter user perception or reality through fake signs or displays. While AR can be a powerful tool to augment the real world, users should always be aware of their real-world surroundings and diligently monitor for differences between what their screen shows and what is around them.
- Understand your content sources: The personalized assistant learns and generates content and information from the world around users. What those assistants have been fed, there is a certain question of unreliability. Data manipulation, spoofing, and sniffing can all contribute to creating unreliable content — so be judicious when evaluating the information multimodal AI provides you.
It’s long been reported that AR and multimodal AI are the future of work, retail, manufacturing, and more – and personalized assistants are great examples of what’s possible with the smart implementation of these technologies. It’s naïve to expect voice assistants to fall out of fashion anytime soon.
As personalized assistant integrations become increasingly common, it’s exciting to consider how Siri, Alexa, Cortana, and their counterparts could be augmented with this emerging, exciting new tech.
How is your workplace incorporating augmented reality tools? What safety protocols have been used to protect the data? Let us know on Facebook, X, and LinkedIn. We’d love to hear from you!
Image Source: Shutterstock
MORE ON AI ASSISTANT
