Friday, November 22, 2024

A closer look at Apple’s Vision Pro keyboard and other controls

Date:

An Apple developer session provided an in-depth look at the many ways users will (eventually) control the new Vision Pro headset, including a virtual keyboard you can type on in the air. Thanks come to us Spatial Input Design Sessionin which two members of Apple’s design team guide potential developers through best practices for designing apps for the new platform.

Apple seems to be keen for users to interact primarily with the headset by simply looking at UI elements and making small hand gestures with their arms resting on their lap. But in his developer session, Apple designer Israel Pastrana Vicente admits that “some tasks are better suited for direct interaction,” which could include reaching for and touching user interface elements (a feature Apple refers to as “direct touch”). There is also support for using physical keyboards and gamepads or game controllers.

Some tasks are better suited for direct interaction.

So let’s talk about the Vision Pro’s virtual keyboard. Apple designer Eugene Krivoruchko explains that it’s important to provide plenty of visual and audio feedback as you use it, to compensate for the “missing tactile information” associated with touching the reading tip. “While the finger is above the keyboard, the buttons display a hovering state and a highlight that gets brighter the closer you get to the button surface,” Krivoruchko explains. “It provides an affinity signal and helps guide the finger to the target. At the moment of contact, the state change is fast and responsive, and is accompanied by a matching spatial sound effect.”

See also  Android 13 virtualization allows the Pixel 6 to run on Windows 11 and Linux distributions

There’s also support for voice input, with the same developer session noting that focusing your eyes on the microphone icon in the search field will trigger Speak to Search. This will most likely pull audio data from a file Six microphones Built into Vision Pro.

Direct touch can also be used to interact with other system elements. There’s the ability to tap and swipe as if you were interacting with a touch screen, and Apple’s demo shows the wearer making a mid-air stylus motion to write a word and draw a heart shape in Markup. Although the primary interaction is done via the user’s hand, Krivoruchko shows how it also uses eye tracking to augment gestures. “You can control the brush cursor with your hand, similar to a mouse cursor, but then if you look to the other side of the canvas and click, the cursor jumps there to land right where you are looking. This creates a sense of precision and helps cover the large canvas quickly,” says the designer. .

We still have plenty of questions about how Apple’s expensive new Vision Pro headset will work in practice (especially its ability to pair with motion controllers from other manufacturers), but between our hands-on experience and developer sessions like this one, the experience is starting to come into focus.

POPULAR

RELATED ARTICLES

How Climate Change Affects Turtle Nesting Sites: What You Need to Know

Climate change is an ever-growing concern, and its effects...

Putin, a member of the International Criminal Court, is set to travel to Mongolia despite an arrest warrant against him

Despite an arrest warrant from the International Criminal Court,...

Japan Typhoon: Millions ordered to evacuate as one of strongest typhoons in decades hits Japan

What's the latest?Posted at 12:48 BST12:48 GMTImage source ReutersTyphoon...