HackMD
Multi-apps
tags: devlog m3
So much of the VR industry, mobile computing, and PC applications have been stuck inside of a 2D, windowed frame, and closed context.
Exokit
- https://exokit.org/
- https://github.com/exokitxr/exokit
- https://github.com/exokitxr/exokit-web
- https://github.com/msub2/aframe-exokit-avatars
Exokit is a project that enables developers to build XR experiences using the same code that runs on the web.
https://www.youtube.com/channel/UC87Q7_5ooY8FSLwOec52ZPQ
Exokit Web is a Javascript library for composing multiple WebXR applications in a regular web page. It works in all web browsers, VR and AR headsets, with no dependencies. The engine is written on top of Node and emulates a web browser, providing native hooks for WebGL, WebXR, WebAudio, and other APIs used in immersive experiences.
Preview of the site https://exokit.org/
Exokit Web can load sites made with any WebXR engine – THREE.js, A-Frame, Babylon.js, JanusWeb, Unity.
XR Iframes (xr-iframes) are like iframes but with extra spatial attributes like position, scale, orientation. Here's an overview of them: https://docs.exokit.org/development-guide/xr-iframe-overview/
The concept is very similar to what Croquet has implemented, even alike in the code.
Things can start to look quite chaotic since these weren't really designed with the knowledge that someday they might be loaded in as overlays inside other apps.
Watch on Youtube: https://www.youtube.com/watch?v=COhmahoC2Mg
Metachromium
- https://metachromium.com/
- https://store.steampowered.com/app/685110/Metachromium/
- https://twitter.com/webmixedreality/status/1306094079369977857 - Release thread
Metachromium adds AR to your VR.
It is a version of Chromium that layers WebXR on top of SteamVR.
Tame an AR pet, scan QR codes, and more.
Any WebXR app works with it, except it doesn't take over the scene. Content is composited (blended) together between the base reality and WebXR.
Why?
You should be able to control what you see. You should be able to pull out a Game Boy, a phone, an avatar, wherever you are.
That's impossible in every XR system today. Metachromium lets you run any WebXR content anywhere.
Free Association in the Metaverse - Avaer @Exokit - M3 (Mar 28, 2020 )
Watch video on Youtube: https://www.youtube.com/watch?v=PXdUJIcRBkk
Tech demo: buy avatars in VRChat market to boost mining in Minecraft, to build a CryptoVoxels parcel on your No Man's Sky planet. Blow off steam chopping grass in Ocarina and chill with Pikachu and a friend on Beach.
Made with WebXR – Avaer from Exokit talks about connecting virtual worlds with WebXR layers.
- WebXR apps that work with:
- VRChat
- No Man's Sky
- Minecraft
- Noclip.website
- Half Life: Alyx (?!)
- Your browser
- Oculus Quest
- Basically everything
- Tokenized items that do things!
- Tokenized avatars!
- Metaverse inventory
- Shared rooms with HUD
- ETH trading interface
- ERC1155 trading interface
- 3D token minting platform
(Demo) View on Youtube: https://www.youtube.com/watch?v=q71-bP6ZBQQ
| From VRChat: the avatar and sword are WebXR holograms | No Mans Sky: holograms work in any VR app | How the inventory system looks, those are 3D glTF NFTs |
- Overlay viewport
- Streaming webrtc connection from browser
- Can add camera a distance from head to facetime other avatars on HUD
Similar scene in Ready Player One, seen halfway in this video https://www.youtube.com/watch?v=rVmmG9HgTac
Overlays can help establish identity between worlds, cross-communicate, inspect items, translate languages, and more.
- metachromium (run websites as AR in VR)
- poly, use browser to shape / create and put em on chain
- meta assembly - party system
Seamless Portals
- Part 1: https://gist.github.com/madjin/ff580a7bbe6cc077cc87c3971801b99fImage Not Showing Possible Reasons
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
- Part 2: https://hackmd.io/@XR/crosslink2Image Not Showing Possible Reasons
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
https://nwn.blogs.com/nwn/2022/01/metaverse-portals-neosvr-gorilla-tag-anyland-portal.html
Even in native app development land the web glues things together in backends and frontends. These portals are basically WebXR apps that cleverly disguise when other virtual worlds are loading in the background. With Metachromium running alongside SteamVR, Koolala was able to demonstrate seamless portal interlinking between:
- Native -> Native
- Native -> WebXR
- WebXR -> WebXR
Native -> Native
The elevator is not part of NeosVR and is not modding the NeosVR client - it's an overlay app made with WebXR that is running alongside SteamVR. There is a special fork of the open source web browser Chromium that is dubbed Metachromium, developed by a single engineer (@webmixedreality), that enabled this capability to author and run WebXR apps like AR in VR.
Native -> WebXR
Koolala was able to demonstrate a diverse range of portal designs within his demo, such as this seamless portal connecting between the native app Anyland to a WebXR site in Mozilla Hubs. It could be possible to add a portal gun to this as well, similar to Rick and Morty or the game Portal.
WebXR -> Native
To complete the trip Koolala then opens a final WebXR portal app linking from Mozilla Hubs (web) into Gorilla Tag (native), an indie game with millions of players. The transition from a hubs robot avatar to gorilla is practically seamless.
Webaverse
Github: https://github.com/webaverse
Docs: https://docs.webaverse.com/
Watch on Youtube: https://www.youtube.com/watch?v=L4YUeIRjUxg
Note: These are ancient screenshots of Webaverse from 2019-2020
Metachromium helped prove that it's possible to glue the metaverse together through the power of interoperable open standards. The problem was that maintaining a browser fork is an enormous task for a solo dev, especially when they are not being paid to do so.
Note: Couple people even rejected the idea of adding a $1 price tag to the app on Steam
The Plan
The idea seeds of Webaverse emerged around the chicken and egg problem of how to generate awesome content for this web metaverse vision in a sustainable manner way. There needed to be a way to close the value loop for creators to make cool apps which NFTs helped solved, and for the platforms to continue adding features / support.
Another aspect of Webaverse pertained to the composition of overlay apps. Nobody was designing their programs to be composable with one another which lead to a cacaphony when attempting such. So Avaer worked on the ability to geofence apps as a way to load / traverse between them.
The Street also emerged as a discovery engine for apps and experiences. Instead of navigating a Website by scrolling up and down, you could navigate The Street by walking forwards and backwards. Land could be owned, so as to decentralize the means of discovery (of which is often the centralized chokepoint in so many places). Below is a top down view of the web UI for browsing, see https://hackmd.io/@XR/maps for context.
Croquet
- https://croquet.io/docs/
- https://github.com/croquet/microverse/blob/main/docs/QuickStart.md
- https://croquet.io/blog/april2022/croquet-launch/index.html
Croquet is a software system for creating multiuser digital experiences on the web. It lets you build real-time multiuser apps without writing a single line of server code, or deploying or managing any servers.
Second, different worlds may use totally different software stacks and rendering frameworks. There will not be a single standard for world interoperability, but at the same time, you do want to be able to view and traverse from one world to the next.
The multiapp approach allows for seamless portals in a browser based virtual world scenario. Croquet is making use of iframes that switch layers in order to achieve this effect.
 |
 |
 |
|---|
Blog post about portals: https://davidasmith.medium.com/multiplane-portals-for-the-metaverse-a4b7af913e8e
Microverse: https://croquet.io/blog/april2022/microverse/
Most of the applications that we use on our PC, mobile phones, or tablets are sandboxed within a native app that’s functionally like a virtual machine where you have a 2D window portal into the app. With the future of spatial computing, then that sandboxed 2D frame disappears, and applications become objects with behaviors that are interactively combined within a shared virtual space.
PlutoVR
The OpenXR and WebXR standards are enabling distribution channels of immersive content through apps like Metachromium and Aardvark, and the OpenXR overlay extension allows for this more modular AR gadget content to be used within the context of existing VR applications run on Steam. Voices of VR
Jump to timestamp: https://www.youtube.com/watch?v=sXl9ol3g7EY&t=133s
Brief overview of Multi-WebXR Launching in MetaChromium
Watch on Youtube: https://www.youtube.com/watch?v=uuN3KnUIJgw
All of these technologies are utilizing the Overlay extension of OpenXR in order to overlay augmented layers of reality on top of VR experiences, and they’re working together in way that will facilitate emergent behaviors that break out the normal 2D frames of our existing computing paradigm. When you run an application on mobile on your computer, then there’s usually only one application that’s in focus for any given moment. You can context-switch between apps, or copy and paste, but our computing paradigm has been all happening within the context of these 2D frames and windows.
The promise of spatial computing is that we’ll be able to break out of this 2D frame, and create a spatial context that allows for apps to directly interact with each other. This will originally happen through lighting, opacity shifts, or occlusion between 3D object, but eventually there will be more complicated interactions, collisions, and emergent behaviors that are discovered between these apps.
XSOverlay
XSOverlay is a Desktop Overlay application for OpenVR. It aims to be as intuitive as possible, minimalist and out of the way, but fully functional. XSOverlay conforms to your needs, and will adjust itself as such!
 |
 |
 |
|---|---|---|
| You can open up multi-monitor displays across apps | The UI can be attached to your wrist, like an AR watch | Imagine working in any VRchat world as your office |
OVRToolkit
LIV
aardvark
Aardvark is a framework for building augmented reality "gadgets" that run on top of virtual reality experiences.
Gadgets are constructed using a bunch of custom React components (from the @aardvarkxr/aardvark-react package) and run in the Aardvark application. Gadgets use these components to show interactive models, 2D UI, or other stuff that will draw on top of any VR applications you run. You can attach these gadgets to your hands and bring them with you in your favorite VR apps.
Watch on Youtube http://www.youtube.com/watch?v=pux6RbySUMU
Tutorial: https://aardvarkxr.github.io/aardvark/watch_tutorial/
Portable Farm
A 3D idle farming game for VR that uses a portal to allow users to see and interact with the scene presented inside of it, while inside of any other VR game
Portable Emulator
Portable Emulator is the second iteration on a new type of SteamVR overlay that displays 3D objects and scenes within VR, allowing users to interact with objects on another dimension, on top of their normal VR experience.
This game requires Retroarch to function, instructions will be provided in video form to make using this app as easy as possible.
M3 org
Something many of these awesome projects have in common is that most of the devs have met each other at a Metaverse Makers Mixer event.
Exokit / PlutoVR devs meeting up and testing Metachromium: https://www.youtube.com/watch?v=WrVh_d09yCE
Watch on Youtube: https://www.youtube.com/watch?v=WrVh_d09yCE
Pluto: https://www.youtube.com/watch?v=sXl9ol3g7EY&t=148s
M3 Mixer with Aardvark dev:
Playlist: https://www.youtube.com/playlist?list=PLFvnZ7So4-FVO54dSXDTU2bKyOz7Z__Ye
