While accelerometers and gyroscopes are generally how smartwatches detect hand movements, a newly-released patent hints that Apple may be considering scanning veins in your wrist via the Apple Watch to detect non-touch gestures in the future.
First spotted by AppleInsider, the patent describes a scenario in which a device with one or more sensors to “capture one or more images of the user’s hand” and then use those images to figure out the positions of the veins in that user’s hand. From those “vein poses”, you could theoretically interpret gestures or finger positions.
The patent was originally filed in August 2018, but these days, no-touch gestures seem like a pretty good idea given the global pandemic. And as explained in the patent’s background, no-touch gestures could actually be useful in terms of accessibility as there might be times when a user can’t speak or may not want or be able to use touch-based inputs to control a device. (Say, you’re cooking and your hands are covered in messy gunk.)
Current Apple Watches might be capable of this sort of thing already. In addition to the typical green light used by most wrist-based wearables, Apple Watch’s optical heart sensor can also use infrared, and both of these sensors are referenced in the patent. If you’re unfamiliar with how the watch currently uses these sensors, in a nutshell, they’re used to detect how much blood is flowing through your wrist and therefore, calculate heart-related metrics. So it’s not entirely farfetched to think the Apple Watch’s optical sensor could be used to scan veins as well. That said, it might not be accurate enough on its own, as the patent does suggest using additional sensors embedded in the strap or even a camera.
On top of your typical gesture-related tech, this could be useful for AR and VR. Apple’s interest in AR is an open secret at this point, and using the Apple Watch for gesture control would be one way of making an Apple AR headset or pair of smart glasses easier to adopt for skeptical consumers. Most AR headsets and smart glasses do include some sort of physical touch control in the form of gloves, or discreet ring-like loops. However, they take a bit of getting used to and are a far cry from the hand-waving you’d expect from sci-fi films like Minority Report.
The document mostly describes this tech in relation to a wrist-based wearable, but there’s no reason it couldn’t work with phones too. The LG G8, for instance, lets you unlock the phone by scanning the veins in your hand via an IR sensor. All you have to do is hover your hand over the phone’s camera. That said, we’ve tried this feature and in practice, it’s a bit more unreliable and difficult to use than you’d like.
As always, patents offer a glimpse into what a company might be exploring, but they shouldn’t be taken as inevitabilities. Apple is constantly filing dozens of patents, many of which never see the light of day in an actual product. Then again, some do. In any case, this particular patent is one of the cooler ones—if only because it teases at a potential wearable AR ecosystem.