Working with Mixed Reality, Mixing AR and VR

Mixed Reality is a term for technology that mixes the daily, physical reality, with something virtual. Lately, this term has been implemented in the form of Augmented Reality (AR), which augments something on top of a certain object, that can be seen using external display, thus making it an augmented one. On the other hand, over the past 2 years, development of VR has been accelerating swiftly, mainly because of the rise of Oculus Rift, a Head-Mounted Display (HMD) that enables its user to experience a Virtual Reality (VR) World.

Continue reading →

99 Names

99 Names is a web VR experience that exhibits 99 Names of Allah in an immersive 3D space. This is my first foray in both Virtual Reality application and WebGL programming through ThreeJS. It’s a simple application, where user can access the web page using their phone or desktop/laptop browser and instantly they can feel the experience, where they’re surrounded by rotating circles showing the 99 names of Allah.

99 Names

The barebone of the project is completed using Web VR Boilerplate, where it ensures that everyone can get a grip of the experience, whether they’re on a desktop, or smartphone, with or without a virtual reality head-mounted display such as Oculus Rift or Google Cardboard. All within the web browser, no application to install. I think that this is a good way to evangelize VR, since at this point, the VR world really needs a whole lotta application to expose its features and shortcomings.

I had so much fun making it. The boilerplate makes it really easy to develop VR experience, so I can focus on the main content, which was all made using ThreeJS. Though I’ve read about it a lot in the past (it’s part of my to-learn-list for about 3 years now, haha), but this is actually the first time thoroughly learning about it. I can say that the best way of learning programming language/library is by making things with it. So far, I’ve learned a lot about 3D pipeline. Which makes me wonder, why didn’t I do this years ago?

However, from the interaction design point of view, I realize that catering VR experience to both kind of platform (desktop or smartphone) is tricky. For example, in smartphone based VR, the user input is limited. Not all phone can use the magnetic input from Google Cardboard, a thing that hopefully will be rectified by Google Cardboard 2. I’m not sure about the other HMD, maybe you, dear reader have another data?

While on the other hand, I can make plethora of input in the desktop version, since the user can use keyboard or mouse or joystick, or other devices to give input to the application. A thing that obviously won’t be mapped precisely in the smartphone counterpart. I did ran into the vreticle library which will help me make a gaze input system for VR, but I still founded some trouble implementing it.

Therefore, at this point, this experience is a passive one, no user input is involved. But I do hope to complete it with one at some point.

99 Names can be accessed at Play with it and let me know what you think.

Meanwhile, here are some screenshots of the steps I did in making it




It’s A Good Time to Develop VR Content

Ever since the head of my department bought two Oculus Rift HMDs and lent one of them to me, I started to dabble a lot into this field. Some months before it, I managed to get a Google Cardboard, which was a good introduction to VR, but I didn’t really able to develop further interest, partially because I had quite a massive simulation sickness for playing with it in a short time (though, this is probably mistake in my side, since I have an acute vertigo to begin with).

Now, Oculus Rift DK2 is actually a very good piece of hardware. I can use it comfortably for 20 minutes using my 2012 non-retina MBP. It also comes with a rapidly developed software suites for developers, meaning it’s actually pretty fast and easy if you want to start making contents for it. The ecosystem is lively, plus there’s a lot of options to pick from should you want to start your adventure here.

Indeed, generally speaking, it’s a good time to develop for VR. As I looked around for games and apps demos for the Rift, I founded out that there’s not much released by big studios/companies, but there are plenty to download or buy from small indie developers. This is another proof that developers who bought it, aren’t afraid of showing whatever they had, even if it’s just a single level wandering around demo type of thing. And this is very important.

This new wave of VR devices still has plenty to crack to make it ready for public use. Several issues, including the haunting simulation sickness are still there. I haven’t come across any research paper that tried to answer it, but each people who made contents for VR has different approach to reduce it. This is why, the more demos available, the more solutions appear.

Also, think about the UI of the content. For starter, this isn’t a flat screen 2D monitor, it’s a full blown 3D experience right from the beginning. Sticking widgets and buttons in the top corner just won’t cut it. How would you answer it? Of course by showing your solution via a demo app.

Those are just two of many issues that I found interetsing in VR contents. Obviously many technical issues will spring out, such as efficient use of polygon to reduce judder (which will also reduce simulation sickness) or how to make the contents less resource intensive, so people don’t have to own a high end gaming PC such as one suggested by Oculus recently, etc.

Those are issues that will be there in a long time. Meanwhile, you can choose your own path in development. Hardware wise, Oculus isn’t the only player in town. For PC-based VR, we will have:

  1. Open Source VR with hardware made by Razer (coming soon July ’15)
  2. Valve and HTC’s Vive, which were claimed by many to be better than Oculus Rift
  3. and the upcoming FOVE which is still running its crowdfunding campaing in Kicksarter.
  4. Last but not least, Oculus Rift Consumer Model, coming soon in Q1 2016

Also, Sony has its own Morpheus which will run exclusively on PS4. This is a good contender, since it can offer a VR experience for millions of existing PS4 players who inclined to get a new PC.

On the other hand, for mobile based VR, there are already a handful of hardware to choose from:

  1. Zeiss VR One
  2. Avegant Glyph
  3. Google Cardboard
  4. Oculus Gear VR
  5. Durovis Dive

With that in mind, you would suspect that software wise, developing contents for them, should provide you with many options as well. You’re right. For developing game, both Unreal Engine and Unity (both are popular game engines) offers support for Oculus Rift and I can’t see why they won’t do the same for others, when available. I tried both to make quick VR prototype and yes, they’re very friendly and provides you with tools to rapidly develop VR contents and polish it along the way.

If web is your thing then afraid not, as both Firefox and Chrome also aims for the VR platform as well. They have the capabilities to deliver the VR content. For programming them, you have the bulletproof Three JS to do it. Also, there’s a work in progress JS API for VR named WebVR, which sounds and looks very promising.

I can sense this VR era would end up quite a lot like the mobile smartphone boom several years ago. The availability of apps really usher the era, partially because of the democratization of developer tools. Every one can make one and every one has the same chance to proof themselves while giving solutions to the platform.

Now I just need to make one myself.