Interoperable Media Object GLBs

This is a pack of GLB files meant to have media item's meta data dynamically assigned to them during runtime. Like shortcut icons, but 3D objects instead of just icons.

In addition to loading the base GLB of an arcade cabinet (for example) from the asset pack, the platform must apply arbitrary meta data to the object to take it from a generic model to an objectl that visually represents the media it references.

The final step is to make the media object interactive, so when activated it displays its content in-world, with the option to launch it in a new window instead.

Meta Data Fields

• title The title of the media.
• file The URL to go to when the shortcut is launched in a new window (or to dispaly on the in-world screen when active in some cases.)
• preview The URL to show on the in-world screen when active. (If none is provided, the "file" field is used instead.)
• screen The direct-image URL to use on dynamic screen surfaces.
• marquee The direct-image URL to use on dynamic marquee surfaces.

Interop Target Platforms

This asset pack sets out to be interoperable with as much as possible, but has the following priorities.

Primary Interop Targets

• PlayCanvas 3D Home Space Kit

Secondary Interop Targets

• Decentraland + any other platform w/ GLB support.
• Hyperfy
• Webaverse

Interop Functionality

There are 3 main points of functional interop desired with this asset pack.

1. ASSET COMPATIBILITY Native art asset compatibility via GLB support.
2. DYNAMICALLY VISUAL Dynamically load textures from URLs specified in arbitrary meta data that can be applied to the art asset to make an object visually look like the media it represents.
3. DYNAMICALLY INTERACTIVE Allow users to activate the visual object & see it displayed on the in-world screen via Dynamic Web Tabs.

Primary Interop Functionality

• Bind an arbitrary set of meta data to media objects as they are loaded into a world to logically drive their look & interactive behavior.
• Assign up to 2 images to the various Dynamic Material Channels via potentially CORS-disabled direct image URLs. (May require CDN such as Cloudinary to support CORS-disabled URLs.)
• Display the URL defined in the meta data on an in-world web tab when the user activates the object.
• Destroy the in-world web tab when the user deactivates the object.
• Provide a way for the user to launch the defined URL in a new window, in addition to being able to view it in-world. (Important for many reasons, but a big one is because platforms that must use iframes for displaying content in-world will encounter many media objects who's meta data points to a website that doesn't allow itself to be loaded inside of an iframe.)

Secondary Interop Functionality

• Support more than 1 active media object at a time.
• Support animated GIF/APNG/MP4 usage on texture channels when appropriate.
• Support in-world playback of the media defined in the meta data beyond just iframes, such as render-to-texture video support, PDF viewer support, or any other way the in-world preview when activating the object could be enhanced.

Open Source Development Project

• The asset pack's functionality is being prototyped & tested in the open source AArcade: PlayCanvas project.
• However, the asset pack itself will just be a collection of GLBs distributed through a CC0 licensed GitHub repo.
• TBD if other developers will create functional prototypes in Hyperfy & Webaverse.

Model List

These are the models to be ported over from AArcade: Source into the interop asset in this phase:

Previews	Cabinets	Previews	Cabinets
	wall_pad_w		lappy
	poster		flatscreentv
	sprite_tall		cdcase
	big_movie_wallmount		wood_cabinet
	slim
	shaft
	fatty
	table

Visual Mesh Guidelines

• Assume most official GLB extensions are supported as defined.
• Use meters as the unit of measurement for best practice, but not strictly required as unit of measurement is accounted for elsewhere.
• Put model origin points at the logical snap position when raycast spawning.
• TBD Assume no structured animation system & no structured naming convention for camera or light attachment points.
• TBD on targeting material variations via KHR_materials_variants.
• TBD on targeting mesh node variations (such as Source engine's Body Groups.)

Collision Mesh Guidelines (TBD)

There are several unofficial glTF-compatible collision specifications to choose from. Let us consider the systems used in the targeted platforms.

Hyperfy

• Specification: https://docs.hyperfy.io/3d-models#how-do-i-add-colliders
• Uses mesh node-naming convention to define mesh nodes as visual-only, collision-only, or both.
• Uses both "_collider" (collision-only) and "_xcollider" (both) naming convention. Mesh nodes are visual-only if neither.
• Collision meshes must each be convex hulls.

DCL

• Specification: https://docs.decentraland.org/3d-modeling/colliders/
• Uses mesh node-naming convention, same as Hyperfy.
• Uses only the "_collider" (collision-only) naming convention. Or visual-only otherwise.
• TBD if collision meshes must each be convex hulls.

There's an incentive to implement the name scheme in a platform that ingests Decentraland glTFs to build a third party client utilizing the content base of Genesis City.

Webaverse

• TBD where to find Webaverse glTF bundled collision specification.
• TBD if mesh node-naming conventions are supported.
• Both convex & non-convex collision meshes seem to be supported, based on the code found here.

Oncyber

• https://docs.oncyber.io/architects/getting-started
• https://oncyber.io/uploader

Yet to be confirmed these collider implementations function the same on both platforms despite following similar name schemes

Oncyber	Hyperfy
https://docs.oncyber.io/architects/getting-started	https://docs.hyperfy.io/3d-models#how-do-i-add-colliders

Issues

• Creating a model that employs the collision naming convention from any of these platforms makes it looks / functions broken in pretty much everything else
  • https://github.com/omigroup/gltf-extensions/discussions/102#discussioncomment-3428295

Here's how it looks in Windows model viewer

Some platforms only support convex collision hulls while others support full trimeshes

Somnium Space understands to hide the collision only nodes, recognizes the name enough:

• Hide the meshes that are collision only through naming convention
  • Somnium Space automatically hides any node that has _collider or _xcollider
  • Doesn't look broken, but doesn't work right either, good middleground
  • Could be a glTF extension
• Doesn't know enough to make that the collisions, have to check box to make visual mesh into collisions, the collider nodes are useless when bundled with the same glb
  • Only useful when exported as separate collision mesh
  • Use separate collision mesh glb called *_col.glb

---

Dynamic Web Tab Guidelines

Dynamic web tabs simply refers to displaying the URL referenced by the meta data on the objects in-world screen while the user has it activated.

The position, rotation, & size of the screen are defined in the hierarchy of the GLB via a special naming convention used on a bone.

• Assume target platforms only support planar in-world web tabs.
• Position a single bone with the proper naming convention (defined below.)
• Target interop platforms will create an in-world web tab there while the user has the object activated.

Bone Naming Convention

dynscreen_in_16.628_13.349

• Starts with dynscreen_.
• Followed by either in, m, or cm to indicate the unit of measurement.
• Followed by another _.
• Then the width of the desired web tab surface, with fractional sub-units.
• Followed by yet another _.
• Finally concluded with the height of the desired web surface, with fractional sub-units.

Dynamic Material Channels Guidelines

• There are only 2 dynamic image channels: screen and marquee.
• The arbitrary meta data assigned to the object will have matching fields, each containing a direct URL to a potentially CORS-disabled image file.
• Target platforms will load the images defined in the meta data & assign them to any materials with channels referencing them using the proper naming convention in their texture asset name (defined below.)

Texture Naming Convention

TBD Determine if glFT retains the naming of texture assets used in PBR material slots (such as the base color map) to be able to power the dynamic material system. Assuming it does, naming convention is as follows.

• DynScreen to be replaced by the meta data's screen image.
• DynMarquee to be replaced by the meta data's marquee image.

Document originally created by SM Sith Lord on 9/2/2022

---

Notes

Oncyber / Hyperfy name schemes conversation in OMI group
Sith likes Ash's collision system the most, but prob best to choose based on most powerful forces in the ecosystem

https://github.com/omigroup/gltf-extensions/discussions/102#discussioncomment-3428295

If you’re creating a 3D model with many mesh shapes that each need their collider, there are some handy add-ons that can help. For Blender you can use Simple Renaming Panel to automatically add a _collider suffix to all mesh names in a group.
https://github.com/Weisl/simple_renaming_panel
Source: https://docs.decentraland.org/3d-modeling/colliders/#best-practices-with-colliders

Avaer's Thoughts

leaning towards the godot style, but there should definitely be a distinction between concave and convex (and possibly even more, like box and capsule)
the engines all support the same set basically
I'll open an issue for the team to make this specifiable, it's really simple to do
currently you set it on the physics component property but I've heard it asked several times from model authors to be able to do it by name in blender
https://github.com/webaverse/totum/blob/main/type_templates/glb.js#L198

it's also worth looking into USD approach: https://blogs.nvidia.com/blog/2021/08/10/usd-physics-extension/
now that it's supported in THREE.js for example
and the basis of omniverse

glXF

https://github.com/KhronosGroup/glXF/blob/main/specification/2.0/README.md
https://github.com/KhronosGroup/glXF/issues/1