# Interoperable Media Object GLBs
This is a pack of GLB files meant to have media item's meta data dynamically assigned to them during runtime. Like shortcut icons, but 3D objects instead of just icons.

In addition to loading the base GLB of an arcade cabinet (for example) from the asset pack, the platform must apply arbitrary meta data to the object to take it from a generic model to an objectl that visually represents the media it references.

The final step is to make the media object interactive, so when activated it displays its content in-world, with the option to launch it in a new window instead.

# Meta Data Fields
- ```title``` The title of the media.
- ```file``` The URL to go to when the shortcut is launched in a new window (or to dispaly on the in-world screen when active in some cases.)
- ```preview``` The URL to show on the in-world screen when active. (If none is provided, the "file" field is used instead.)
- ```screen``` The direct-image URL to use on dynamic screen surfaces.
- ```marquee``` The direct-image URL to use on dynamic marquee surfaces.

# Interop Target Platforms
This asset pack sets out to be interoperable with as much as possible, but has the following priorities.

## Primary Interop Targets
- PlayCanvas 3D Home Space Kit

## Secondary Interop Targets
- Decentraland + any other platform w/ GLB support.
- Hyperfy
- Webaverse

# Interop Functionality
There are 3 main points of functional interop desired with this asset pack.

1. **ASSET COMPATIBILITY** Native art asset compatibility via GLB support.
1. **DYNAMICALLY VISUAL** Dynamically load textures from URLs specified in arbitrary meta data that can be applied to the art asset to make an object visually look like the media it represents.
1. **DYNAMICALLY INTERACTIVE** Allow users to activate the visual object & see it displayed on the in-world screen via **Dynamic Web Tabs**.

## Primary Interop Functionality
- Bind an arbitrary set of meta data to media objects as they are loaded into a world to logically drive their look & interactive behavior.
- Assign up to 2 images to the various **Dynamic Material Channels** via potentially CORS-disabled direct image URLs. (May require CDN such as [Cloudinary](https://cloudinary.com/) to support CORS-disabled URLs.)
- Display the URL defined in the meta data on an in-world web tab when the user activates the object.
- Destroy the in-world web tab when the user deactivates the object.
- Provide a way for the user to launch the defined URL in a new window, in addition to being able to view it in-world. (Important for many reasons, but a big one is because platforms that must use **iframes** for displaying content in-world will encounter many media objects who's meta data points to a website that doesn't allow itself to be loaded inside of an iframe.)

## Secondary Interop Functionality
- Support more than 1 active media object at a time.
- Support animated GIF/APNG/MP4 usage on texture channels when appropriate.
- Support in-world playback of the media defined in the meta data beyond just iframes, such as render-to-texture video support, PDF viewer support, or any other way the in-world preview when activating the object could be enhanced.

# Open Source Development Project
- The asset pack's functionality is being prototyped & tested in the open source [AArcade: PlayCanvas project](https://playcanvas.com/project/892980/overview/aarcade-playcanvas).
- However, the asset pack itself will just be a collection of GLBs distributed through a CC0 licensed GitHub repo.
- **TBD** if other developers will create functional prototypes in Hyperfy & Webaverse.

# Model List


These are the models to be ported over from AArcade: Source into the interop asset in this phase:

| Previews | Cabinets | Previews | Cabinets |
|:------------------------------------:| ------------------- |:--------:| -------- |
|  | wall_pad_w |  | lappy |
|  | poster |  | flatscreentv |
|  | sprite_tall |  | cdcase |
|  | big_movie_wallmount |  | wood_cabinet |
|  | slim | | |
|  | shaft | | |
|  | fatty | | |
|  | table | | |

# Visual Mesh Guidelines
- Assume most [official GLB extensions](https://github.com/KhronosGroup/glTF/tree/main/extensions) are supported as defined.
- Use **meters** as the unit of measurement for best practice, but not strictly required as unit of measurement is accounted for elsewhere.
- Put model origin points at the logical snap position when raycast spawning.
- **TBD** Assume no structured animation system & no structured naming convention for camera or light attachment points.
- **TBD** on targeting material variations via [KHR_materials_variants](https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Khronos/KHR_materials_variants/README.md).
- **TBD** on targeting mesh node variations (such as [Source engine's Body Groups](https://developer.valvesoftware.com/wiki/$bodygroup).)

# Collision Mesh Guidelines (**TBD**)
There are several unofficial glTF-compatible collision specifications to choose from. Let us consider the systems used in the targeted platforms.

## Hyperfy
- Specification: https://docs.hyperfy.io/3d-models#how-do-i-add-colliders
- Uses mesh node-naming convention to define mesh nodes as visual-only, collision-only, or both.
- Uses both "_collider" (collision-only) and "_xcollider" (both) naming convention. Mesh nodes are visual-only if neither.
- Collision meshes must each be convex hulls.

## DCL
- Specification: https://docs.decentraland.org/3d-modeling/colliders/
- Uses mesh node-naming convention, same as Hyperfy.
- Uses **only** the "_collider" (collision-only) naming convention. Or visual-only otherwise.
- **TBD** if collision meshes must each be convex hulls.


There's an incentive to implement the name scheme in a platform that ingests Decentraland glTFs to build a third party client utilizing the content base of Genesis City.

## Webaverse
- **TBD** where to find Webaverse glTF bundled collision specification.
- **TBD** if mesh node-naming conventions are supported.
- Both convex & non-convex collision meshes seem to be supported, [based on the code found here](https://github.com/webaverse/totum/blob/main/type_templates/glb.js#L198).

## Oncyber

- https://docs.oncyber.io/architects/getting-started
- https://oncyber.io/uploader

Yet to be confirmed these collider implementations function the same on both platforms despite following similar name schemes

| Oncyber | Hyperfy |
|:------------------------------------:|:------------------------------------:|
|  |  |
| https://docs.oncyber.io/architects/getting-started | https://docs.hyperfy.io/3d-models#how-do-i-add-colliders |

### Issues

- Creating a model that employs the collision naming convention from any of these platforms makes it looks / functions broken in pretty much *everything else*
- https://github.com/omigroup/gltf-extensions/discussions/102#discussioncomment-3428295

Here's how it looks in Windows model viewer

Some platforms only support convex collision hulls while others support full trimeshes


Somnium Space understands to hide the collision only nodes, recognizes the name enough:
- Hide the meshes that are collision only through naming convention
- Somnium Space automatically hides any node that has `_collider` or `_xcollider`
- Doesn't look broken, but doesn't work right either, good middleground
- Could be a glTF extension
- Doesn't know enough to make that the collisions, have to check box to make visual mesh into collisions, the collider nodes are useless when bundled with the same glb
- Only useful when exported as separate collision mesh
- Use separate collision mesh glb called `*_col.glb`

---

# Dynamic Web Tab Guidelines
Dynamic web tabs simply refers to displaying the URL referenced by the meta data on the objects in-world screen while the user has it activated.

The position, rotation, & size of the screen are defined in the hierarchy of the GLB via a special naming convention used on a bone.

- Assume target platforms only support planar in-world web tabs.
- Position a single bone with the proper naming convention (defined below.)
- Target interop platforms will create an in-world web tab there while the user has the object activated.

## Bone Naming Convention

- Starts with ```dynscreen_```.
- Followed by either ```in```, ```m```, or ```cm``` to indicate the **unit of measurement**.
- Followed by another ```_```.
- Then the **width** of the desired web tab surface, with fractional sub-units.
- Followed by yet another ```_```.
- Finally concluded with the **height** of the desired web surface, with fractional sub-units.

# Dynamic Material Channels Guidelines
- There are only 2 dynamic image channels: ```screen``` and ```marquee```.
- The arbitrary meta data assigned to the object will have matching fields, each containing a direct URL to a potentially CORS-disabled image file.
- Target platforms will load the images defined in the meta data & assign them to any materials with channels referencing them using the proper naming convention in their texture asset name (defined below.)

## Texture Naming Convention
***TBD*** Determine if glFT retains the naming of texture assets used in PBR material slots (such as the base color map) to be able to power the dynamic material system.  Assuming it does, naming convention is as follows.

- ```DynScreen``` to be replaced by the meta data's ```screen``` image.
- ```DynMarquee``` to be replaced by the meta data's ```marquee``` image.

*Document originally created by SM Sith Lord on 9/2/2022*


---

## Notes



Oncyber / Hyperfy name schemes conversation in OMI group
Sith likes Ash's collision system the most, but prob best to choose based on most powerful forces in the ecosystem

https://github.com/omigroup/gltf-extensions/discussions/102#discussioncomment-3428295



> If you’re creating a 3D model with many mesh shapes that each need their collider, there are some handy add-ons that can help. For Blender you can use Simple Renaming Panel to automatically add a _collider suffix to all mesh names in a group.
https://github.com/Weisl/simple_renaming_panel
Source: https://docs.decentraland.org/3d-modeling/colliders/#best-practices-with-colliders

### Avaer's Thoughts

> leaning towards the godot style, but there should definitely be a distinction between concave and convex (and possibly even more, like box and capsule)
the engines all support the same set basically
I'll open an issue for the team to make this specifiable, it's really simple to do
currently you set it on the physics component property but I've heard it asked several times from model authors to be able to do it by name in blender 
https://github.com/webaverse/totum/blob/main/type_templates/glb.js#L198

> it's also worth looking into USD approach: https://blogs.nvidia.com/blog/2021/08/10/usd-physics-extension/
now that it's supported in THREE.js for example
and the basis of omniverse


### glXF

https://github.com/KhronosGroup/glXF/blob/main/specification/2.0/README.md
https://github.com/KhronosGroup/glXF/issues/1