XR fragments - HackMD

XR fragments ============ Introducing 4D URL's using Protocol-Agnostic-Syntax (PAS?) to discover, link, fallback, navigate & query 3D. # UPDATE 10-02-2023 * See [THE MILESTONES in this document HERE](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#Milestones) * See the XR fragments spec (draft) [here](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#Appendix-XR-fragments-draft) * See [the updated USECASES here](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#XR-fragments) <br><br> ![](https://i.imgur.com/ZVwt5eJ.png) ## Problem / Background The 'metaverse'-concept is opted as a solution for networked 3D content. However, recent shiny attempts can easily be labeled as 3D/VR **walled 3D gardens**. There aren't public protocols/standards for 3D content-linking, only fileformats. By enriching 3D assets with HTML-like attributes, we can build webrings & connected worlds **without** gatekeepers. Some quotes: > "We are easily fooled into believing things are not that bad when we see a few big vendors joining up to serve something tasty. " - Michiel Leenaars > *"The idea of the metaverse, can be "a honeypot trap for 'architecture astronauts.'" Those are the programmers and designers who "want to only look at things from the very highest levels," he said, while skipping the "nuts and bolts details" of how these things actually work."* ~ John Carmack (creator of DOOM and Oculus VR) > "Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better." - Dijkstra # A network of glTF files I have interviewed several WebXR creators: ![](https://i.imgur.com/LyxumnW.png) > **glTF**: an open JSON-based format for 3d objects,textures & animation, a successor of the XML-based VRML. Now imagine the following glTF-files network-idea: * Add attributes (which link to other [glTF](https://www.khronos.org/gltf/)-URI's) inside glTF-files using your favorite 3D software * export your 3D scene as [glTF](https://www.khronos.org/gltf/)-file (including attributes) * host it somewhere as `mydomain.com/index.gltf` * surf to `mydomain.com` in an webXR-website or browser (wolvic) which supports XR URI fragment's * `mydomain.com/index.gltf` will be requested & rendered * surf to `otherdomain.com/#spawn=1,5,0` to teleport the user to xyz coord 1,5,0 (inside otherdomain.com/index.gltf e.g.) ![](https://i.imgur.com/tDFAIvd.png) This allows a public navigate-able **read-only** 3D content-layer on top of HTTP, in a no-code and discoverable way. There **is** already **plenty** infrastructure to use (HTTP(s)/IPFS), we don't need to create new infrastructure. > This project will focus on a gltf-webxr-demo, but ideally the XR URI fragments can be supported natively by browsers (like wolvic). ## Solution: XR fragments * like [media fragments](https://www.w3.org/TR/media-frags) introduce XR fragments for 3d navigation/manipulation (dynamic hinting) > by doing so, WebXR devs won't have to write the same application-code over and over again, and **remote** XR assets can integrate **better** with eachother: ## XR fragment spec: See the XR fragments spec (draft) [here](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#Appendix-XR-fragments-draft) ## Example usecases See [the updated USECASES here](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#XR-fragments) ### Possibilities * Linking worlds together using URI's * Dynamically generating worlds (since `glTF` == `JSON`) using serverside scripting language * build your own WebXR viewer using HTML/JS * mirror `glTF`-URI's on IPFS or archive.org * XR URI fragment's are gltf-agnostic (FBX/usdz or future fileformats will benefit from it too) * allow clients to reduce cpu/gpu load by cherry-picking / excluding content from gltf-files * add XR URI fragment support to SEARXR (easy 'webxr ring browser' test-environment) ### Eventual possibilities * Native XR fragments support in opensource [wolvic](https://wolvic.com/en/) browser * consolidation of proven fragments/attributes in a KHR_XR_FRAGMENT or similar extension for the [glXF draft-format](https://github.com/KhronosGroup/glXF) > But first, the priority is battleproofing/testing the fragments/attributes, before consolidating them into lower or higher level standards. # Background During the development of (NLNet's funded) [SearXR](https://searxr.me), I realized: > 'Surfing' the 3D web is hard using headsets, albeit impossible. > The browser exits fullscreen/WebXR mode when clicking HTML anchor-tags, and nobody has yet suggested URI-to-3D-navigation, **webmasters as trustkeepers** and/or using communication standards like irc/matrix. My initial hope was the [W3C immersive navigation proposal](https://github.com/immersive-web/navigation). But after 3 year following/being engaged with it, it seems immersive **crossorigin** navigation will not be on the table soon in the ways that many were hoping. Therefore, more opensource serverside XR-silo's can be expected. From a corporate view, this is convenient, because this deadlock gives centralized solutions (walled 3D gardens) an advantage (in the last 3 years) . **WebXR rings** liberate itself from this deadlock: it goes back to the simple idea of **read-only** **webrings** in which **webmasters can control which content to host or link to**. The idea is that a good web**master** (not platforms) and **good** content allow for destination which are worthy to remember, revisit and to trust. More about me: * [github](https://github.com/coderofsalvation) * [linkedin](https://www.linkedin.com/in/leonvankammen) # Historical efforts * **FBX** (as closed source as it can get, no PBR=-support like gltf) * **VRML** (too early, hardware not powerful enough, superseeded by glTF) * **HTML** (does not allow immersive navigation, see background below) * **FB** is building a **walled 3D garden** called 'horizon' * **MS** is building a **walled 3D garden** called 'altspaceVR' * [sketchfab.com](https://sketchfab.com) turned walled garden (shop) * [glXF draft-format](https://github.com/KhronosGroup/glXF) has various limitations, most importantly being tied to glTF only * many startups & nonprofits are building open/closed-source walled garden's, by requiring specialized servers (*) > \* = an opensource walled 3D garden is a an **opensource blackbox** made of revolutionary technologies (unlike http+glTF) which only a handful of people understand on a technical level. This makes adoption harder. The common issues: * tunnelvision/focus on building 1 platform * hardware cannot render too much 3d at the same time (VRML) * over-planned ecosystems-in-a-box (like yahoo/angelfire) * obsessing over 'one seamless world' made of multiple files * obsessing over 'owning parcels', NFT's, marketplaces etc. * china-like planned ecosystem (instead of organic ecosystem like www) * decentralized efforts can easily introduce high technical barriers to entry (blockchains,IPFS), at least compared to simple **http+gltF** webhosting. > lowhanging fruit: provide public **read-only** navigation for an open format (`.gltf`) which can be accessed online by native games/apps **AND** the web. # Budget & Resources There are currently no funding sources or code for this. I would like to request **10k** euros (gross) to fund, which are spreaded across the milestones below, including communication, reporting, promo/travel activites. Optionally, another **10k** could be used to create a team/foundation around HyperglTF to better integrate with it's NGI/user-operated internet-surroundings. # Ecosystem, Actors & Outcomes Inputs: * OpenSource WebXR developers who are looking for simple way to link,navigate & query public 3D content * glTF / game content-creators * HTML/GLTF hosting (free) * HTML5 app on 5apps- or gitlab- page (=free) Actors: * Leon (projectlead) * NLNet (stakeholder) * People with WebXR devices (mobile/desktop/AR/VR) Outcomes: * website with protocol RFC * git repo with: * example documentation / reference * link to XR URI-supported 'browser' (https://searxr.me) * XR query-parser in many languages: python3 / php7 / java / lua / c++ / c# * empower existing WebXR applications * query-parser as `THREE.js` library available on git/npm * query-parser as `AFRAME.js` library available on git/npm # Challenges * use of [history API](https://html.spec.whatwg.org/multipage/browsing-the-web.html#the-history-interface) could limit/create possibilities * limiting the v1 spec to something which stays simple to implement for 3d content creators * is there already a human-readable space-separated query-format for entities/groups/presets? * how to promote a `foo.com` => `foo.com/index.gltf` redirect (client- or server-side? or both?) and use [history API](https://html.spec.whatwg.org/multipage/browsing-the-web.html#the-history-interface) (not) for that? * how to invite feedback while also communicating principles: # BOM * glTF fileformat, license: Apache / royalty-free * Haxe(script) compiler license: GPL2/MIT # Milestones See [THE MILESTONES in this document](https://hackmd.io/@JO6OXLnCRkKjtdtIEBdZag/Hkb40BGpo#Milestones) ps. each milestone is estimated at 666.66 euro (no pun intended)