Reality

Microsoft insists on calling AR and VR ‘Mixed’ Reality. Maybe we should too.

Microsoft insists on calling AR and VR ‘Mixed’ Reality. Maybe we should too.

Back in 2015, when the the world was just beginning to pay serious attention to virtual reality, Microsoft surprised everyone by announcing HoloLens. Instead of surrounding you with virtual imagery – like Oculus and every other VR company – HoloLens brought the digital world into the real. To date, it remains one of the coolest things Microsoft has ever made.

But along with HoloLens came Mixed Reality, a term that seemed to confuse pretty much everyone not working in Redmond. ‘Virtual’ and ‘augmented’ reality were the established lingo, and HoloLens seemed to be just a fancy form of the latter. ‘Mixed’ reality reeked of marketing buzzword, it sounded lame, and so I avoided it as much as possible.

Now I’m starting to budge. In the last two years – culminating in this week’s Build conference – Microsoft has laid out a foundation for its grand scheme to fundamentally change how we interact with our devices. Knowing why Microsoft so stubbornly adheres to mixed reality is crucial to understanding how it plans to get there.

HoloLens was just the start.

Mixed Reality includes AR, VR, and beyond

Over the past few months, I’ve had several discussions about mixed reality with Greg Sullivan, Director of Communications for Windows and Devices at Microsoft. In each, he’s repeatedly drilled into my head the idea that mixed reality is a spectrum – one that encompasses VR, AR, and everything in-between. In fact, the term has academic origins that long predates modern virtual experiences.

So no, you’re not wrong to call HoloLens an AR headset, it’s just that AR is but one part of the mixed reality spectrum. The term also encompasses a wealth of other device categories:

  • Fully immersive VR headsets like Rift, Vive, and Microsoft’s offerings coming later this year.
  • Camera-based AR, like Snapchat’s funky masks or games like Pokemon Go.
  • Augmented virtuality experiences, whereby real-world elements are brought into a virtual experience.
  • Potential future devices, that can do the best of both worlds, changing between being opaque like Rift, or transparent like HoloLens.

While HoloLens happens to be an AR device, Microsoft would be doing itself a disservice to limit its scope to that bit of spectrum. HoloLens was the first device in the Windows Mixed Reality platform, which seeks to enable immersive experiences in all the above form factors.

But why go with HoloLens first? After all, VR headsets seemed to have more immediate consumer applications for things like gaming, and are a lot less expensive to produce than HoloLens.

According to Sullivan, Microsoft wanted to start by solving the more difficult problem. VR headsets have the luxury of obscuring the real world. You can create more interesting experiences with some environmental interaction, but VR can be fun even while your feet are stationary.

HoloLens, on the other hand, required an understanding of the environment around the user. It was a deeper problem to solve, but one that would reap rewards; Sullivan says Microsoft used what it learned from HoloLens to create affordable VR headsets that beat the competition to the punch with inside-out tracking.

That means that, unlike Oculus and Vive, Microsoft’s partner VR headsets can map your movements in the real world without the need for messy external sensors. I’ve spent a fair amount with the…

A trio of visionaries will explore the future of augmented reality

Above: Tom Cruise in Minority Report inspired lots of tech companies.

I’m very excited about the trio of technology and entertainment visionaries that are the latest speakers for our upcoming GamesBeat Summit 2017: How games, sci-fi, and tech create real-world magic. They include John Hanke, CEO of Niantic Labs and co-creator of Pokémon Go; Ralph Osterhout, CEO of Osterhout Design Group, maker of augmented reality glasses; and John Underkoffler, CEO of Oblong Industries, maker of the Mezzanine collaboration software and science advisor for the film Minority Report.

GamesBeat Summit 2017 will take place on May 1-2 at the historic Claremont resort hotel in Berkeley, Calif., just a short distance from San Francisco. You can secure your seat here. Register today and receive 20 percent off current ticket prices. Use the code Deantak.

I will moderate the session on the future of augmented reality, games, and new technologies. Our visionaries will have a conversation across the seams of science fiction, real-world technology, and games.

Above: John Underkoffler, CEO of Oblong Industries, was science advisor on Minority Report.

Underkoffler has been trying to make the vision of the 2002 film Minority Report, where actor Tom Cruise uses “data gloves” and gestures to control a transparent computer, into a reality. He founded Oblong in 2006, and launched Mezzanine for enterprise collaborators in 2012. Oblong’s technological and design trajectories build on fifteen years of foundational work at the MIT Media Lab, where Underkoffler was responsible for innovations in real-time computer graphics systems, optical and electronic holography, large-scale visualization techniques, and the I/O Bulb and Luminous Room systems.

He has also been science advisor to films including The Hulk, Aeon Flux, and Iron Man. He serves on the National Advisory Council of Cranbrook Academy in Bloomfield Hills, MI, and on the Board of Directors of the E14 Fund in Cambridge, Mass., and of the Sequoyah School in Pasadena, Calif. He is the recipient of the 2015 Cooper Hewitt National Design Award and holds a PhD from the Massachusetts Institute of Technology. Underkoffler will also be giving a solo talk at our summit on May 1.

Above: John Hanke, CEO of Niantic Labs.

Hanke has been running Niantic and riding the rocket ship of Pokémon Go, which has more than 65 million monthly active users and has generated more than $1 billion in revenue in less than a year. His company is dedicated to building “real world” mobile gaming experiences that foster exploration, exercise and social interaction. Niantic has developed and released two games, Ingress and global phenomenon Pokémon Go. The company was originally founded as a start up within Google to explore the creation of new kinds of entertainment at the intersection of location, social, and emerging mobile devices. Niantic was spun out as an independent company in 2015 with backing from investors such as Google, Nintendo and The Pokemon Company.

Earlier in his career, John was a founder of Archetype Interactive, creators of one of the very first online massively multiplayer games, Meridian59, and then went on to co-found Keyhole, a company acquired by Google for the technology that lead to the creation of Google Earth. He was vice president of Google’s Geo division (overseeing projects including Maps and Street View) for seven years before founding Niantic.

Above: Ralph Osterhout, CEO of Osterhout Design Group

Osterhout is a developer, designer and…

HP moves into VR and AR with investment in Venture Reality Fund

Hewlett-Packard is putting on its headsets. One of the world’s oldest technology companies is investing in the fledgling virtual reality market by becoming an investor in The Venture Reality Fund.

The exact amount wasn’t disclosed. But HP Tech Ventures, the new corporate venture arm of HP, has joined as an investor The VR Fund, which has become one of the most active investors in VR, augmented reality, and mixed reality startups. It is HP’s first move into VR investments.

The VR Fund has invested in a number of early-stage startups developing infrastructure, tools, platforms, content and apps for the mixed reality ecosystem.

The VR Fund will provide HP with early access to leading AR/VR/MR technologies with commercial applications in HP’s target markets including office, retail, healthcare, manufacturing and education. The VR Fund’s…