Month: March 2022

thumbnail for vr and innovation blog

VR and Immersion

AA logo white
White Avatar Academy Logo

 

Article by: Henry Gaudet

Publishing Date: 28th March 2022

VR is on its way to becoming mainstream, but until it does, until we all intuitively understand its language – its shorthand and conventions – the way we do with the language of film – it can be hard to grasp what exactly Virtual Reality brings to the table. Sure, it looks great, but what is it… for?

There’s a vague desire to connect it to existing media, to think of it like video games, or like 3D movies, or some sort of hybrid. And in so doing, we mentally pigeonhole it with those other media, assuming it has similar benefits and limitations, that it’s essentially them, but with a twist.

But those comparisons don’t really capture what VR does or what it can do. And so, for those not yet familiar with the technology, we’d like to offer some insight into its potential, starting with the actual experience of being in a virtual environment. 

While movies and TV provide a window into a new reality, VR drops you right in the middle. There is no screen. There is no frame. Everything around you on all sides belongs to this digital reality. Even you – yes you – are digital! Hold your hand in front of your face and what you see is not your physical hand, but a digital double, one that can interact with this world.

The world you find yourself in may be simple or detailed, stylised or realistic, but it is novel and (assuming the designers have done their job well!) engaging. It invites exploration and experimentation. And it is as close to distraction-free as we’re likely to find today. 

Just about every form of media out there comes with a risk of distraction. Most of us have at some point scrolled on our phones while “watching” TV, giving neither our full attention. With VR, that’s not really an option. When you put on your headset, the virtual environment fills your entire field of view. There is no glancing at your phone or checking the clock on the wall without removing that headset.

To be fair, this level of immersion comes with some limits and considerations. The headset leaves you effectively blinded to the physical world, so you will need to ensure that you wear it in a safe environment. 

If you move in the VR world, make sure you have the space to move in the physical. Even for seated experiences, it’s important to make sure that the area around you is clear so that a gesture in the virtual world doesn’t knock over your very physical coffee cup!

The result is a more focused experience, one where multitasking is simply not an option, and one that contrasts sharply with our attention-craving modern world. 

This feature particularly stands out in VR training exercises. Paired with the interactive nature of VR, this makes for a course that holds attention and produces rapid learning. 

White Avatar Academy Logo
AA is an Oculus ISV Partner
Address

Hand-Tracking Developments

White Avatar Academy Logo

 

Article by: Jordan Murphy

Publishing Date: 21st March 2022

Virtual Reality has pushed the boundaries in the immersive field since it started, and one of the newest features of this is Hand-Tracking. Hand-Tracking for the Oculus Quest is still in the beta version at the current time; however, Facebook have revealed their new iteration that they are working on. Their high-fidelity hand-tracking is on Facebook’s research page, a publication called Constraining Dense Hand Surface Tracking with Elasticity

Hand-Tracking works by using the inside-out cameras that are a part of the Oculus Quest’s hardware. The headset detects the position and orientation of the user’s hands and tracks the configuration of the users’ fingers. Once the headset detects them, the computer vision algorithms are used to track the movement and orientation of the user’s hands. 

Facebook Reality Labs

Facebook Reality Labs (FRL) journey to create more natural devices started with the Touch controllers. The Touch controllers for the Oculus Quest are comfortable to hold and contain sophisticated sensors. They deliver a life like hand presence and make the most basic interactions in VR feel like the user is doing that action. As close to the real thing that the Touch controllers can get, they cannot replicate the expressiveness of hand signs or the act of typing. 

On the 1st of December 2020, FRL published a paper with an accompanying video that showed their new research and technology allowing them to track the human hand in real time. With Facebook’s interest in VR through Oculus, it is not surprising to see them working on a better VR environment that is closer to human interaction. The efforts made to create the hand-tracking system for the Oculus Quest had an issue with hand overlap, which is the core of the goal of FRL’s latest research. The research focused on solving the big problem of losing tracking when hands overlap, touch, or fingers are self-occluded (Hexus, 2020).

Self-contact and hand-tracking technology

Many of the gestures and actions that people make with their hands involve self-contact and occlusion: shaking hands, making fists, or interlacing their fingers while thinking or idle. These uses illustrate why hand-tracking technology today is not designed to treat the extreme amounts of self-contact and self-occlusion exhibited by the common hand gestures.

FRL’s paper states that by extending recent advances in vision-based tracking and physically based animation, they can present the first algorithm capable of tracking high-fidelity hand deformations through highly self-contacting and self-occluding hand gestures. In the paper, they describe that by constraining a vision-based tracking algorithm with a physically based deformable model, they have obtained an algorithm that is robust enough for use in self interactions and massive self-occlusion that are displayed by the common hand gestures, allowing them to have two hands interacting with each other and some of the most difficult gestures that the human hands can make. 

Despite the improvements that this paper has proven to make on the already publicly available hand-tracking technology, the method that FRL has developed has its limitations. It is computationally expensive. The method is also limited by the ability to capture high frequency folds and wrinkles. In Figure 4 of the paper, they recorded that sometimes the captured images do not portray the wrinkles on the tracked mesh. These are issues that can be worked out in future iterations, but this research shows a promising future for hand-tracking in the immersive industry.

Conclusion

At Mersus Technologies, we use Hand-Tracking on our Avatar Academy Platform. This allows us to create as close to real world experiences as possible which benefits the learner in the transfer of the knowledge and skill they gain from Avatar Academy to the real world procedures. 

White Avatar Academy Logo
AA is an Oculus ISV Partner
Female vr ar xr developers

The women who have made innovations in Virtual Reality

White Avatar Academy Logo

 

Article by: Stephen McQuaide

Publishing Date: 8th March 2022

When we think of people in the immersive tech industry, who comes to mind? It’s easy to look at the people endorsing the product, the people companies use to help market the product, but often times, we don’t think about the individuals who are innovating the immersive landscape, those working first-hand on the technology that makes this possible. It is important to spotlight these people, to bring attention to them, their hard work and accomplishments. To celebrate International Women’s Day, let’s look at four female figures who have set new standards for the industry through new technological developments as well as finding new and creative ways to implement VR in other settings.

Liv Erickson

To start with, there is Liv Erickson, who is currently a Senior Manager at Mozilla. She has extensive experience in software development for XR, 3D and metaverse technologies that contribute to the evolution of graphics and cloud computing. On top of that, she also has experience in management, strategic planning and collaborating with various teams.

Throughout her 12-year career, she has also worked for Microsoft and Amazon to develop and manage developments of various projects. In that timeframe, she has created several open-source VR and AR applications, a site that visualises Excel charts in 3D, co-developed the Digital Afterlife Project which addresses the challenges involving user data after the user themselves pass away and much more. She is currently working on Mozilla Hubs, a 3D collaboration platform inside a customisable space which works on desktop, mobile, VR and even on a browser.

Check out her other accomplishments through her LinkedIn.

Nicole Lazzaro

Nicole Lazzaro founded XEODesign, Inc., a firm that consults with clients on how to increase engagement with play. They identify ways to improve engagement in gaming and develop an understanding of players and what motivates them in order to improve the experience. She has almost 30 years of experience in designing the player experience and virtual reality is no exception.

She has developed an XR experience called Follow the White Rabbit, a mind-bending VR puzzle adventure immersing the user into a magical spectacle. She also developed the first iPhone game to utilise the accelerometer that measures the acceleration of motion, which would pave the way for games like Doodle Jump. Other efforts include creating the 4 Keys to Fun, a model that insights on the important elements required to make a game fun and interesting and consulting with the Obama White House and the US State Department on how to use games to improve the state of our world.

Check out her other accomplishments through her LinkedIn.

Rosie Summers

Rosie Summers is a 3D animator at XR Games. Despite her career only being active in the last 5 years, she has made strides as an animator for VR projects. These include helping to bring worlds to life with titles such as Angry Birds Movie 2 VR: Under Pressure and Zombieland VR and most notably, carving out her own niche as a Virtual Reality Artist.

What comes with creating art in VR is the performance. She essentially draws an image while in the digital world through the use of motion controls, allowing her dynamic hand movements to paint the picture. As she paints, the development of the painting is shown to the audience on a screen, so they can see how it is being made in real-time. Because of this, her paintings translate into a live spectacle. These live paintings have been showcased for clients such as Facebook, National Football Museum, the BBC and Riot Games. She has attended a variety of festivals and workshops throughout the UK to spread the power of this medium to the wider audience.

Check out her own website to get an improved look at her skills and portfolio.

Yuka Kojima

Yuka Kojima is the CEO and co-founder of FOVE, who made the first eye tracking VR headset. This means that the technology is able to capture the user’s eye movement in an accurate way, with one site even stating it as ‘Iron Man meets Oculus’. It will undoubtedly pioneer the technology and create the potential for even more immersive experiences.

In 2017, she made it into the top 100 list of most powerful women according to Forbes Japan and even made the front cover of the magazine. Before her success, she had set up a Kickstarter Campaign back in 2014 for the headset. She also previously worked at Sony Computer Entertainment Japan before leaving to branch out into VR.

Check out the FOVE website to discover more about her company.

Conclusion

Mersus Technologies have been producing VR experiences for the past 5 years, and are currently further developing our Avatar Academy platform.  Several aspects of Avatar Academy require serious research and development of new systems, and our female staff members are integral to this.  

Women are represented at every level of the Mersus team. We have female immersive developers, creative developers, concept designers, middle and senior management.

Mersus Technologies strives to create an environment in which all members of our community should expect to be able to thrive, be respected, and have a real opportunity to participate in and contribute to the company’s activities so that they can achieve their fullest potential. We are always seeking to increase the diversity of our team.  

We understand that embracing diversity makes our workforce more innovative, resilient, and high-performing.  We truly believe that teams that are as diverse as possible make better apps, and women play a crucial role in what is a traditionally male-dominated technology sector.

On International Women’s Day 2022, we look forward to a future where women are better represented across all technological fields. We in Mersus are doing our part.

Through Avatar Academy the women on our team play an integral role in setting new standards for the Immersive industry. The woman to keep an eye on for 2022 in the Immersive space are Palak, Fiona Moran, Polly Wong, and Brenda Mannion.

AA is an Oculus ISV Partner