Metaverse? Not sure, but WebXR? Hell yes!

One of the core technologies of the future Internet, that we are already using for applications beyond entertainment.

You may have heard these last weeks about the Metaverse. My own definition of it, convoluted from the heterogeneous definitions I’ve found around, is that “the Metaverse is a means of manufacturing immersive digital spaces for human activities integrating social networks and virtual economies through a blend of virtual and physical spaces”. The hope (and hype!) is that current technologies for augmented reality (AR), virtual reality (VR), and high-speed Internet will allow for a seamless flow of our lives at the edge of physical and virtual worlds. Possibly even unconscious in the farther future?

I feel all this is a bit creepy but at the same time stands as an opportunity for the further development of mankind, perhaps even a new technological revolution. If used correctly of course, like with all technologies. What is surely certain is that the development of the Metaverse (or whatever comes out in our attempt to build it if it fails) implies a burst in the range of actual applications of tools like VR and AR to our lives and a motivation for the development of novel technologies. Let’s give here a look into this future, by building on the present.

WebXR

At the level of hardware and software, a key component of the Immersive Internet (my short definition of Metaverse, although a bit far from the definition above) is the WebXR API, a group of standards to support the rendering of 3D scenes in AR or VR in various kinds of web-connected devices. By “various kinds of web-connected devices” I mean all hardware spanning from high-end AR/VR immersive devices like Facebook’s Oculus or Microsoft’s Hololens, to less immersive but easier-to-engage devices like tablets and smartphones. The point is that the WebXR standard facilitates cross-platform programming as it parses the different kinds of user input controls of the different devices (say touch in a smartphone, mouse in a computer, hand recognition on Oculus, etc.) and their output display capabilities (flat screens, immersive 3D, etc.) into unified handlers for programmers. Thus, essentially the same source code can run correctly on the different devices to render the AR or VR experiences.

While the X in WebXR leaves place for Augmented and Virtual realities (and actually also for other buzzwords like Mixed reality, used more by Microsoft), it is more important to stress the Web part of WebXR. The point is that WebXR-based applications consist exclusively of code for web programming, that is the same code used to build every web page: HTML, CSS, and JavaScript. The high degree of standardization of the browser development communities, defined by W3C, ensures coordination and wide cross-device compatibility. On top, the Web in WebXR makes it easier for programmers to develop content. And not only expert programmers but also novices and amateurs. Even I, an amateur programmer with little time to dedicate to it, can build some basic AR and VR programs or clone and edit the code written by the professional WebXR expert working in my research group. Moreover, one month ago we had at EPFL (Center for Intelligent Systems CIS EPFL , EPFL Extension School, EPFL Entrepreneur Club and EPFL Tech4Impact) high school students visiting our lab who could engage with the source code of our latest WebXR apps for molecular visualization themselves, making some small but useful edits. Because it’s so easy to get cool things done with client-side web programming!

Last, an obvious advantage of programming for web browsers is that since the programs stand right as regular web pages, you don’t need to install nor distribute them. Users just access a regular page and voilà, they are working or playing with your new app. In fact, I present later on in this article some examples that you can just go and use right now on your computer, smartphone, tablet or Oculus!

Connectivity and integration with other technologies

Of course, a key aspect of the Metaverse is online connectivity. Not surprisingly, Facebook is one of the stronger supporters pushing for the Metaverse. So much, that they just changed their name to Meta!

But beyond the inherent need for online connectivity to support social network media, which still needs to be developed for the Metaverse, the mere fact of all this happening in browsers enables today some really cool applications. For example, one can make use of speech recognition engines that operate in the cloud to easily add speech-based commands to the web app. This is particularly useful for immersive devices, where users need to have their hands free to interact with virtual objects so grabbing a mouse or typing on a keyboard poses a limitation.

Another important use of internet connectivity is that of easily allowing collaborative sessions where multiple users experience the same immersive session. Say teacher and students in a class; artists, performers, and audience in a theatre; or a group of coworkers discussing work. With web-based coding the whole thing is already online, so connecting two or more sessions should be easy. And it is! In the simplest form, using direct browser-to-browser connections through for example WebRTC you can share immersive sessions even without any intermediary server other than one required just to establish the initial connection. Moreover, the WebXR standard allows users with different kinds of devices to all share the same session (perhaps with different levels of immersion, but they can at least share views). Of course that’s at a simple level, from which you can build onto bigger projects where the experience is centralized in a server to which all users connect -but my point here was to keep it simple, that’s why I stress direct browser-to-browser links.

Actual applications

WebXR is being increasingly used, but to be honest I still see it struggling to reach real-world applications. The building of the Metaverse is in progress; in fact, it has just begun. As more developers move into it, hardware improves, costs go down, and more money is invested in developing content and improving hardware, we will for sure see more and more people using VR for applications of actual utility and not just a mere curiosity on which they play two or three games, view a 360° video, and that’s it.

Example 1: cryptocurrency market in VR

I first take the chance to show you here this article where I integrated WebXR and API calls to a crypto information hub to provide an immersive view of bitcoin price in real time. Much like a stock market center for cryptocurrencies in VR, that I’m sure the Metaverse will someday have:

Example 2: chemistry and biology in VR

Disciplines that deal with abstract objects and concepts, or with real-world objects and concepts that are too small or too big to visualize, could expect the strongest impact from Metaverse-related technologies especially VR and AR. We have already seen this on low-end but zero-cost augmented reality:

Especially with Google-provided 3D models for in-smartphone webAR to cover biology and moleculARweb to cover chemistry and its connection to biology:

But the Metaverse asks for more immersivity. We are in our lab now moving to slightly more expensive yet affordable VR devices that support WebXR, like Facebook’s (rather Meta’s) Oculus Quest 2, and beginning to develop molecular visualization tools for this device. See here one working example developed in our lab by WebXR engineer Fabio Cortes, part of a larger program we developed for science communication activities during the inauguration of Switzerland’s biggest research center on electron microscopy.

The example above, whose code you can inspect like for any other webpage, uses WebXR to display the (experimentally determined) X-ray structure of human carbonic anhydrase, a human enzyme, bound to a sulfonamide (a group of molecules of clinical use). By moving around extremely immersively and controlling the orientation and zoom with its hands, researchers can much more efficiently explore the structure and -as used in this case- teach about protein chemistry and how clinical drugs act on our bodies.

The hope is to soon allow multiple users to view and work on the same molecular system at the same time, by sharing the VR session, and to incorporate a treatment of the physics and chemistry so as to have not just static 3D views but also enable interactive simulations. With such simulations, scientists will be able to interactively try how a clinical drug candidate binds to a protein, or how a mutation on a protein could affect its activity, among hundreds of other applications, some of which we don’t even imagine today. Right now we are in our lab working on these and other fronts, attempting to develop multi-user experiences where various users can act on the same molecule and also implementing very simple physics to describe how atoms move. It’s a long way from the dream tools for work and teaching in science, but in my opinion, with WebXR we are on the right track. We recently commented about all this in this letter in Nature Computational Science:

Closing remarks

The Metaverse and the software and hardware developments associated to it promise, in my opinion, a revolution on how we teach, learn, entertain, and even work. Right now I identify two main limitations, that hopefully will be addressed and sorted out as we build the Metaverse:

On one side, content. Even the games available today for VR devices are rather simple, and the amazement remains at the stunning quality of the 3D views and the fidelity of hand tracking. But the content itself, is today very limited. I think, or assume, that more investments and people working on developing the Metaverse will change this. In particular, I believe we need to couple WebXR programmers with virtual artists and experts in various disciplines (say from the hard sciences like chemistry, biology, and physics, to economics, finances, etc.) to ensure that quality content is developed. Otherwise, it will all remain in a curiosity and one more kind-of-failed attempt at going 3D (as happened in the past with anaglyphs, fiducial marker-based AR, early haptic devices, etc.)

On the other hand, costs. In particular for teaching, where I see large potential for these technologies, these devices are still expensive… Even rich countries cannot afford say one Oculus Quest 2 per student, at least as of late 2021. While it is remarkable that costs have gone down from 2–4k USD to just a few hundred USD, this is still not low enough for many families even in developed countries. I do hope that prices will go down to at least the order of 100 CHF. That said, it is very important to bear in mind the WebXR standard allows running the same content not only on a high-end device like the Oculus or MS’s HoloLens, but also on consumer devices like smartphones and tablets. Although much less immersive and lacking hand tracking, with these devices users can at least see the content. With current smartphones and without incurring high costs, the problem of immersivity might be tackled by WebXR’s capability to render VR views for smartphones inserted inside cardboard goggles, and the limitation of hand tracking might be resolved with machine learning tools that achieve this simply through the device’s webcam. For example, this article shows how to track hands with MediaPipe in the browser:

And these science outreach articles use in-smartphone WebXR to teach about the structures of viruses:

Let’s hope for the best possible future for the Metaverse, or at least for the technologies that will be developed along the way. The future of multiuser immersive experiences is getting closer and promises revolutionary applications to real-world work, education, and entertainment.

I am a nature, science, technology, programming, and DIY enthusiast. Biotechnologist and chemist, in the wet lab and with computers. I write about everything that lies within my broad sphere of interests -the @lucianosphere. Check out my lists for more stories. Become a Medium member to access all its content and subscribe to get my new stories by email (original affiliate links of the platform for which I get small revenues without special costs to you).

For inquiries of all types, contact me here. For small jobs (on programming, data analysis, cryptocurrencies, biotech + bioinf project evaluations, science outreach + communication, molecular data analysis and design, molecular graphics, photography, moleculARweb tutorials, science teaching and tutoring, etc.) check my services page here.