Designing for Hololens
But what is Hololens like to work with as a designer?
Compared to 2D apps wearables require a completely different UX and UI vernacular, both to overcome their limitations and make the most of their possibilities. Because of this we’ll be sharing our experiences in designing for this exciting and unexplored field these past months.
A little background first: we are no stranger to neither AR nor wearables, having spent years crafting numerous augmented reality apps for a range of customers. These apps were mainly done for tablets and phones, but also for most of the last-gen wearables like the Vuzix M100, Epson Moverio BT200 or the first Meta. Things such as a wearable warehousing tool, or a gesture-controlled AR sculpting app — a whole host of apps ranging from enterprise tools to entertainment. Here at TWNKLS HQ we are currently taking the next step by developing a range of apps for Hololens, trying to get the most out of the Developer Kits.
Below you’ll find my three main impressions of designing for Hololens, laced with some links that offer some background.
Know what to show
Make no mistake: wearables still have a long way to go in order to fulfil their promise. But in a world where most AR functionalities only exist in After Effects it was great to see that Microsoft’s product delivers. It’s simply very impressive.
Physically it is well made and feels durable. It’s self-contained, easy to setup for your head and has an array of sensors to provide an excellent tracking stability. This ensures the Hololens has a situational awareness just like you do, and it offers a spatial sensation with steady holograms and 3D sound.
However there are some peculiarities that greatly influence what you can and can’t show:
- Its additive display makes it impossible to show blacks, or even shadows onto the real world (as explained thoroughly here). Especially when you’re dealing with messy backgrounds (like a factory floor for example) bright colors are necessary to keep up contrast in varying light situations. So no black text on a white background.
- The tracking is good but not perfect. As standard it doesn’t respond that quickly to changing rooms (like standing in a crowd of moving people for example) and generates a quite rough mesh. This is most apparent in ambient occlusion: when you place a hologram behind a real world object the hologram is cut off quite coarsely. Both of these things can be adjusted for accuracy, at the expense of processing power.
However very shiny objects or glass simply do not track.
Want to show a black spider that crawls over your window? Tough luck.
- Make sure you don’t get cut off. A narrow field of view (FOV) means your holograms will get trimmed along the edges. This is especially true vertically. This makes it quite hard to keep an overview when you’re moving around a lot, and it’s common to have to search around for your hologram due to this. Whilst this can be infuriating, you can also use it to your advantage. By intentionally spawning people and things outside of the FOV the game Fragments cleverly uses this trait to generate suspense.
Objects also get cut off when you come too close to them, which on the upside gives an instant x-ray vision of whatever you’re looking at.
Summing up you’re best off avoiding dark shiny objects that move around a lot, but if you show something more static and luminous you’ll be sure to wow.
Make your UX spatial
Traditionally spatial UX has always been a bit trying for most users.
People have the idea that interacting with a spatial UI means waving your hands around like Tony Stark. Whilst admittedly those gestures are cool if you’re playing a game, they get old when you’re having to use Holo for an 8-hour workday. It just gets tiring.
In essence Hololens is designed around two main gestures: bloom and airtap. Since bloom is reserved for getting back to the main menu you’re pretty much left with the airtap. That’s it. These gestures are combined with the aim provided by your gaze to create a navigation that is very much like using a mouse on a screen.
Decoupling aiming and selecting also makes for a much less tiring interaction, which is a very interesting approach. Just like you could trick your Wiimote into making a perfect tennis swing with a mere flick of the wrist the Holo lens lets you interact without the typical fatigue associated with 3D UI.
So to interact with your world you need to observe the following:
- Feedback is king. Due to a lack of this the combination of gaze and airtap tends to be tricky. To provide a bit of grip your aim is signified with a dot, which also shows whether your hand is recognized. This is important because the Hololens can recognize hand gestures significantly beyond its FOV. Microsoft also gives you a clicker which offers a more positive feedback but does occupy your hand. Lastly you can use voice controls, although its lack of detail and susceptibility to environmental sound does not make it the preferred option.
- Always keep your interfaces spatial. Don’t be tempted use a UI that’s fixed to your gaze, because you’ll have no way to select its elements. You are not Arnold Schwarzenegger and this is not Terminator.
- In terms of UI these input methods mean your buttons should be large and spaced widely apart. Have your gaze snap to nearby buttons to avoid narrowly missing them. The key here is to offer a satisfying and easy UX whilst taking into account the inaccuracies of waving your arms around in front of some glasses.
- Make sure your elements are near the center of your field of view to avoid them being cut off by the FOV. In the very likely event you do lose your window make sure you can find it back. Using an arrow to guide you back or perhaps using sound is a good way to get you back in the action.
Design with a 3D mindset
As designers we love our tools. Sketch, Invision, even just having exactly the right fineliner — tools matter. Different jobs require different tools. Hololens is no different. You learn by a copious amount of mistakes what works best when designing for a truly 3D experience. Here’s what works best.
- Designing for a 3D experience is less graphic design and more product design. Your holograms have volume, they reflect light differently as you walk around them and they derive their sense of scale from their surroundings. So work with that. In doing so you’ll create something that works as well in-app as it does on paper.
- Keep an eye on color and contrast. After all that your hologram’s colors will blend with whatever is behind it, and that might not always work that well. A good test to see if your colors and contrast hold up in real life is to put it in Photoshop using a busy background image, with the interface bundled in a group that is at 85% opacity and set to Lighten.
- Now you know how to work Photoshop make sure you use it as little as possible. Go to 3D as early in the process as you can. Make models out of foam or clay and interact with them as you would with Hololens. Touch it, shove it, tape it to the wall and get a feel for it. If your office is covered in off-cuts and foam dust at the end of the day: that’s a sign you’re on the right path!
Hololens is a huge step towards that elusive wearable you can wear all day and augment your world with. It opens a whole range of real-world applications from maintenance to training or simply sitting on the couch.
And there’s a lot more to come: Meta 2 will be hitting developers soon with its amazing FOV, and Magic Leap still promises to be an AR experience to blow us all away. In terms of UX the step to 3D will be one of the biggest opportunities in design since the first touchscreen. So let’s get experimenting!
Useful UI/UX resources