INTRODUCTION

In our last article, we introduced the exciting features of our platform’s meeting room and character customization. If you missed it, you can catch up here. Today, we are thrilled to unveil a groundbreaking addition to our web experience: The WebXR World. Let’s dive into this immersive new feature and explore what it has to offer!

Link to the demo: https://udl-3d-web-multiplayer.onrender.com/

FEATURES

In this version, users can enter VR mode by simply pressing the VR button on the left side of your screen, and step into a virtual environment. This feature is compatible with mobile devices that support VR, making it accessible wherever you are.

Users can also join the meeting room inside VR mode, just click the call button below the TV to connect with everyone present. This feature allows you to listen to conversations, join discussions, and stay engaged with the meeting activities, all from the comfort of your VR setup.

HOW IT WORKS

Explore the WebXR

In our quest to provide cutting-edge virtual reality experiences directly through your web browser, we’ve integrated WebXR technology into our platform using the powerful react-xr library. This technical deep dive will explain how we’ve implemented this feature to seamlessly bridge the gap between our React application and the WebXR API, enabling a fully immersive VR mode with just a click.

React-xr is a specialized library that extends React’s capabilities to support WebXR, which is a web standard for enabling augmented reality (AR) and virtual reality (VR) experiences in web applications. By using react-xr, we effectively bridge our React application with the WebXR API.

Set up VR in Canvas

Controller-Based Navigation and Interaction

In the VR mode, traditional keyboard and mouse controls are replaced by VR controllers, providing a more immersive and intuitive experience. Here’s how we handle navigation and interactions using react-xr:

Using the useController Hook: react-xr provides a useController hook that allows us to receive input from VR controllers. This hook makes it easy to handle various controller inputs, such as button presses, joystick movements, and trigger actions.
Free Look and Head Tracking: In VR, users can freely look around using their headsets. We synchronize the head tracking data from the headset to the user's avatar, ensuring that everyone in the VR environment can see where others are looking. This is achieved by continuously updating the avatar's orientation based on the headset's orientation.

Hook being used to receive data input from controller

Creating an Immersive UI in VR Mode

Transitioning into VR mode poses unique challenges, particularly when it comes to maintaining a functional and interactive user interface (UI). Traditional HTML and CSS-based UIs become ineffective in VR environments, as these technologies are not designed to render within 3D space. To ensure seamless interaction and connectivity within our VR World, we have implemented a new UI system using the three-mesh-ui library. This article will delve into how we create and manage UI elements in VR mode to keep users connected and in control.

Why Traditional UIs Fail in VR Mode

When entering VR mode, the web browser's rendering of HTML and CSS elements no longer applies. These elements are designed for 2D screens and operate within the Document Object Model (DOM) of a web page. When entering VR mode, the browser switches from rendering the page's DOM to rendering the WebXR scene. This context switch means that standard HTML and CSS elements are no longer drawn because the browser focuses on the 3D canvas provided by WebXR.

Introducing Three-Mesh-UI for VR UI

To overcome this limitation, we use three-mesh-ui, a library specifically designed for creating UIs within a 3D space. Three-mesh-ui allows us to build interactive UI elements that render correctly in VR, ensuring users can navigate and interact seamlessly.

How Three-Mesh-UI Works

Three-mesh-ui creates UI elements as meshes within the 3D scene. These UI meshes are integrated directly into the VR environment, allowing them to be rendered and interacted with just like any other 3D object.

Create call button by using three-mesh-ui

CONCLUSION

The introduction of our WebXR marks a significant leap forward in enhancing your virtual interaction and collaboration experiences. With seamless access, a lifelike meeting environment, and effortless connectivity, our WebXR is designed to revolutionize the way you engage with colleagues and clients. Dive into this new immersive realm and discover a whole new level of productivity and engagement.

REFERENCES

https://immersiveweb.dev/

https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API

https://github.com/pmndrs/react-xr

https://github.com/felixmariotto/three-mesh-ui

Topic WebXR

Virtual Office: Integrating WebXR for Immersive Meetings