AR & VR - The Upcoming Future

Every up-to-date Windows 10 PC now comes with the built-in AR and VR platform and every major PC maker like Dell HP and Asus built a headset for it. Facebook hosted what was arguably the world's largest VR event ever and give us a glimpse at two upcoming Oculus headsets. Magic leap a company that has never even released a single product. But we think it might one day we least something AR related just collected new investments putting it at a total of two billion dollars. 

And the relatively new company called Pi max VR just finished a successful Kickstarter campaign for a VR headset with two 4k screens and 200° a field of view. Companies are clearly investing incredible sums of money even though consumers don't seem to be too interested. And the internet is full of videos and articles claiming that ER and VR are dead. So there is a clear fundamental mismatch is AR really the future or is it just a fad.

let's first run through what is available with VR and AR today. VR by definition requires covering up your full vision with something virtual. So it is basically always experienced through a headset mobile VR. Where users slide their phones into headsets like with gear VR and daydream are relatively cheap and offer a basic experience. That stand-alone headsets have really just become a thing with oculus releasing the goal recently. Which basically runs the components and the OS of an Android phone slightly customized.

But in the headset itself, no phone needed. And finally, we have the high-end stuff the headsets that are hooked up to powerful machines to provide reasonably high-quality images for tracking and so on. PlayStation VR is an affordable entry into this category and on the PC from Oculus. An HTC vive were the first two players all three of these require a powerful machine and external cameras for positional tracking. The newest competitor is the Windows mixed reality platform that now ships by default on Windows 10 pcs. And all major PC makers made a headset for it like my dell visor for example. And mixed reality headsets also bring some major improvements to the mix.

They actually do inside-out tracking with these little cameras on the handsets. So there's no need for external cameras and they also run on lower-class hardware like regular laptops for example. That's the VR spectrum today and to put things into context. Let's also talk about sales figures there's no reliable source for them online. But here are our best estimate market researchers claim that Oculus and vive have sold around half a million units each sony confirmed. That it has sold over 1 million PS VR headsets by June 2018. And Samsung confirmed that sold 5 million headsets by January.

Data is a little scarce and these numbers aren't exactly huge. But it's a start for a technology that started reaching consumers just two years ago. And with Google Facebook Samsung Sony Microsoft and all major PC makers having pretty expensive bets and VR.

We can reasonably expect much more growth in the future unlike VR which for now is trapped in headsets er doesn't have to be. Because every time we add digital things on top of real things in real time we can call it a yawn. So if you've ever used snapchat filters or played pokemon go then congratulations you have technically used it before currently.

There are really two ways of achieving AR. First one is snapchat Method.The snapchats method where we open up a device with the camera and the screen we capture a video feed. And then we just add something on top of the video in real time. So this is kind of half AR because we're not adding things on top of actual reality. But rather a proxy of it and man that everyone copies snapchat with this, not just other social media services. But on snapchat but also Apple Google and Microsoft as well those three have added of Mantid reality platforms right into their operating systems. So the phones and even computers were analyzed what their cameras see for depth and lighting and stuff.

They will create depth maps and at any app place digital objects onto them that stick to surfaces have shadows. And just generally try to behave as real objects would. So that's the snapchat method but there's also what I'll call the hololens method. Which is way more advanced the hololens projects little floating hologram like things on top of actual reality the screen itself here is see-through. So users don't have to see reality through the camera and the screen of their phone. They just see reality with the projections on top.

So this is really futuristic but for now, it is restricted to basically only the Microsoft hololens. And even though this is an impressive headset. It is mostly just meant for developers as it is very expensive and has a few major issues like a very narrow field of view. So that's our for now adding things on top of camera feeds seems relatively trivial. And hundreds of millions of people use it and enjoy it but adding things on top of actual reality is limited and expensive for now holo land sales figures according to its commercial lead are. And I'm quoting this in thousands not hundreds of thousands.

This only took AR in its current form about two to three years. So that's where we are today most big tech companies are pumping out VR headsets and platforms even though consumer interest in them seems relatively low. And AR has also seen lots of investment and unlike VR even some pretty widespread adoption. But only because it was tied in with the free services. And the devices we already used anyway not because people were actually willing to pay for it. In other words, companies are investing a ton of money despite not being able to make money back right now. Which can only mean one thing they expect massive returns in the future, how massive Mark Zuckerberg announced that his latest oculus conference. That he's hoping to see 1 billion users of VR devices.

He conveniently omitted a timeline for that but still, that's about 200 million more users than all of Instagram has for now. Microsoft CEO Satya Nadella speculates that AR and VR or as his company likes to call it a mixed reality will be one of the three technologies that will shape our future the most.

If you were wondering the others are artificial intelligence and quantum computing they're regular stuff. So mixed reality is in the top three with those Tim Cook the CEO of Apple when recently asked in an interview -what technology he is the most excited about said, it was a are like out of all technologies he chose AR as number one. He said he sees AR being used by consumers by every business. He knows anything about in education and everywhere else. So three of the world's most important tech CEOs are telling us that some combination of AR and VR will basically be the next big computing area.

And yet consumers it seems to be pessimistic think about why that could be I kept coming back to this. The Gartner hype cycle it's a model Gartner has developed for understanding the spread of radically new technologies. And it states that as a technology first appears people get excited companies to tell them that everything will be possible. And the future is now the hype gets real and expectations get inflated. But when first units start shipping out and the experience doesn't live up to those expectations. Then consumers very quickly give up in a category and become disillusioned the first generation of products. Usually doesn't look anything like the future.

They were promised bitter journalists and self-declared YouTubers like yours truly declared the death of an industry. And consumers decide it's time to move on. But if the technology has real merit, companies will keep hammering at it until they fix the early problems. And eventually, people start seeing its benefits again. And according to Gartner, this is where AR and VR currently are on the curve. They're not too far off the first wave of VR and AR was definitely overhyped. And many users give up on a category that's what we're seeing right now. There's also a clear renewed interest from companies to start fixing some of the inherent issues of the first generation.

And if they do there might be a resurgence, so what the biggest players are up to for the future. Facebook seems to have two important projects for the near future.

First- They, of course, want to expand their AR cameras in all of their social media smartphone apps.

Second- It is projected Santa Cruz which will be Facebook's first fully standalone headset. That will do inside out tracking with four cameras on the headset itself. This is planned for 2018 and if you think about it. It's kind of the VR form-factor dream no cables no external cameras no reliance on a phone or a PC just full VR freedom.

Facebook really seems to hope that this category will become its main platform going forward. And it's not hard to see why sure it's a cool form factor. But maybe, more importantly, it also moves Facebook away from the competing Google and Microsoft platforms. Which it hates to rely on even if oculus OS for now just seems to be a custom Android build Santa Cruz is a first step towards. Facebook building standalone VR headsets running on an OS Facebook can't control.

So that's the near future they have also announced that their latest developer conference. That they think in the far future AR glasses that just project things on top of the real world. Hololens style will eventually replace smartphones and TVs Microsoft thinks. And talks about AR and VR as two ends of its mixed reality spectrum. People don't realize just how much most of Microsoft's newest creations support this platform. So It's called Universal Windows platform apps which are modern apps built for Windows 10 like Skype or the Twitter app. And don't just work on PCs and Windows phones but also in mixed reality.

So as Windows 10 on the PCs gets more apps. And does mixed reality Microsoft's latest design language called fluent design adds translucency lights shadows and a more material feel. Which sure looks pretty on a PC but really starts to make sense in a 3d environment where apps appear as physical objects. That need physical properties.

Microsoft's new focus on pain 3d and building a vast library of user-generated 3d objects might feel a little silly on a PC. But instantly make sense for you've guessed it a mixed reality. So the direction of all the existing development projects already works towards strengthening this new platform. But let me also make some predictions as the Xbox is becoming more and more just a regular PC. We can be fairly sure that it will soon support the Windows mixed reality platform as well. And that would be a big deal Microsoft's other big project at the moment called Windows core previously known under the codename Andromeda OS. That aims to strip Windows 10 down to its core and to make all the extras on top of it modular.

So it can be adapted to run on anything from giant servers to tiny wearables or a standalone mixed reality headset running Windows 10. For example, just imagine any Hardware maker could just take the stripped-down version of the OS and all the sensor technology and just make AR and VR headsets with it. That sounds pretty cool.

The last Prediction I have for Microsoft comes from this sentence. Now mixed reality will span from PCs to bones or Paik headsets see through headsets in the near future headsets that can do both. So I can't tell you what exactly Microsoft meant by that. But here's a basic guess we already have cameras on the Front's of our headsets and screens inside it seems pretty obvious. That sooner or later the user could just be shown what the cameras see. So that way headsets could technically switch between VR and AR.

And there are so many more predictions I could make but one thing is clear. that Microsoft is all-in on mixed reality their weakness though is that they don't have a viable smartphone strategy. Which conveniently brings us to Google and Apple. The core strengths of both are that they own the most important device category out there.

Smartphones are the most important platform for both AR and VR right now. Mobile has the most users and these two companies are leveraging their positions. But what I'm not sure it's working for them as well as one could expect given their dominance. I mean Facebook and snapchat the two most important users of smartphone AR. Both use their own AR algorithms as far as I can tell and not those from Google's AR core or Apple's AR kit and Sampson. The most important Android phone maker in the world has sold way more oculus powered gear VR headsets than Google has sold daydreams on top of that.

I also haven't seen them talk about any of the more advanced stuff either like inside our tracking or moonshot projects like the hololens. And they could totally be working on those things in secret. But all I can see from them, for now, is that they are focusing on mobile AR and VR. And having big challenges establishing themselves as the dominant platforms on their own platforms. So anyway from all of this I can conclude seven big trends.

First, inside-out tracking will become the new standard for VR headsets. It already works reasonably.

Second, there will be a big push for standalone headsets in the near future starting with the oculus go.

Third as standalone headsets don't have to rely on Windows or Android. We'll have a new era of platform Wars new platforms can emerge like oculus OS which I think will move further and further away from the standard Android.

And really any company with a few good ideas and enough money could just build a new platform magic leap is rumoured to build a pair of holo land style AR classes. That I imagine would run on a whole new platform steam or HTC could come up with something of their own or Amazon could do with AR and VR. With a few cool products spend endless amounts of money on them and voila platform Wars forth. The boundaries between AR and VR will get blurred VR headsets with cameras on the front will do both at the same time. While AR and VR are currently mainly for gaming.

We will soon start seeing other applications to Windows mixed reality especially. Lets you use desktop programs like Microsoft Office for example. And two oculus is moving in this direction too, as a matter of fact, I typed out and researched most of the script for the video in OneNote. And while I totally know that only a geek would ever do that in 2017 everything worked surprisingly well. And also companies are working on social via AR to Facebook has spaces. And Microsoft has alt space VR which are social VR platforms or you can hang out and collaborate with other people. So gaming it might lead the way but other categories will become relevant as well.

The next point is that people will start realizing that not everything in AR and VR has to be 3d just because we moved from static paper to dynamic screens doesn't mean that we replaced all static texts and images with video in the same way. We will realize that 2d apps and content are just fine in these new environments as well. And we'll live right along with the 3d stuff and my last prediction is that AR and VR won't be mainstream for at least five more years to come. Which is why it's okay if people don't care about it now and by the way that's what people in the industry are predicting Facebook.

For example clearly put AR and VR on their ten-year roadmap as far into the future as they could why? Because many breakthroughs have to be made before they can go mainstream just to name a few who will need more computing for better visuals better input methods. Because our current controllers are really only designed for gaming as well as better lenses with wider fields of view.

That don't fold up and distort close to your face and can change between focal lengths. For example, these are big challenges but as head of Oculus research, my Culebra said the question with these problems is not if we will solve them but rather when we will solve them.

So they will happen but until then AR and VR will complement our existing devices not replace them. Those are my predictions and five to ten years will have compact standalone headsets with good performance. And inside-out tracking that can switch between AR and VR.

Let you not just game but have social experiences and get real work done and further in the future. We will have proper see-through glasses that can project virtually anything on top of reality. It is true the AR and VR industry is eerily quiet now but I do think there is a big storm coming up on the horizon. And we'll just have to wait for it.