HackMDKTX + VRM
Business Plan Idea
- Webaverse holders get access to:
- KTX / Draco compression for glTF / VRM
- Anata holders get
- KTX / Dracom compress
- Access to LoRA training
- 1 voucher for avatar tailoring services
Samples
Scilly
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Original: https://cdn.discordapp.com/attachments/895395385156173834/1186758232774021340/scilly.vrm?ex=659469bb&is=6581f4bb&hm=0596291fa0df78e40f0ae22153074247f5f1d968921b705e93778712bd360c39&
After: https://cdn.discordapp.com/attachments/895395385156173834/1186758239958868029/scilly_merged4_unity3.vrm?ex=659469bd&is=6581f4bd&hm=9c5a77e89a04892f9681c2d1b783d6c66ce2a1db371d09b5222b76c975a005f6&
Meshopt:
VRoid sample
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Anata sample
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Steps I took
Software Used
- https://github.com/m3-org/CharacterStudio
- https://github.com/KhronosGroup/3D-Formats-Guidelines/blob/main/subpages/KTXArtistGuide_glTF-Compressor.md
- https://github.com/vrm-c/UniVRM/
- https://github.com/atteneder/KtxUnity
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- (Optional) Process VRM through Character Studio to get more optimized version that has 1 draw call
- Rename VRM to GLB
- Process through https://github.com/KhronosGroup/3D-Formats-Guidelines/blob/main/subpages/KTXArtistGuide_glTF-Compressor.md
Comparing Image Encoding
UASTC
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
ETC1S
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Blendshapes broke, had to fix in Unity with https://github.com/atteneder/KtxUnity
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
After fixing, the VRM is now 2.1 MB
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- Confirm it worked by unpacking GLB to glTF using gltf-pipeline: https://github.com/CesiumGS/gltf-pipeline
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Currently there's a bug with character studio. I had to import to Unity in order to convert to VRM 1.0 and then export as VRM 0 after modifying the materials back to mtoon for VRM0.
- Go to where textures are stored, ensure that the mimeType and uri point to the ktx2 textures and that the
KHR_texture_basisuextension is being used
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
My shapekeys broke when I processed this sample, and had to use Unity in order to fix it. Remember to export with sparse accessors checked to preserve them.
What does this mean?
VRM was designed for interoperability across games and vtubing apps that are typically 1-4 players. Over the last year we've seen a major adoption of VRM on platforms, particularly on the web, wanting to support many custom VRM avatars in one scene.
Optimizing a VRM avatar takes a lot of skill and time since there are a lot of gotcha things that can break results. Although VRM is based on glTF, we can't use many ready-made tools like glTF-transform because they're destructive as well.
Example, I optimized this VRM with character studio then brought it into blender and re-exported it using Saturday06 VRM plugin. Without doing any changes to the file, it almost doubled my file size from 9.3 MB to 16.3 MB.
Original file: ~15 MB
Optimized with Character Studio: 8.2 MB
+ KTX compression: 6.4 MB
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Challenges
KTX not supported in blender by default
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
KTX not supported in Unity by default, needs additional dependency
Not all virtual world platforms support it, but more would be inclined to with easier tooling and proven benefits for the stakeholders. One case study can make all the difference.
Meshopt
Original file: https://cdn.discordapp.com/attachments/895395264213442571/1190393740163022928/225.vrm?ex=65a1a390&is=658f2e90&hm=af3a32ea7f49f51d7e04dbec510795c588686a8a6738ef2a30a53f1b6402b659&
Meshopt: https://cdn.discordapp.com/attachments/895395264213442571/1190393469789798520/225_ktx19.vrm?ex=65a1a34f&is=658f2e4f&hm=063d4baa5f621559de6640288ed5580194bc135d35e88aacd78c8ffb11f32ad2&
From 4.8 MB -> 500 KB, morph targets kept intact!!
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Stuck on this one error
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
VRM result doesn't work in hyperfy / oncyber. Shows as valid VRM and glTF though. JSON is valid.
Maybe its an order of operations?
Error in gltf sample viewer
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Error in gltf report
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Links
- https://github.com/zeux/meshoptimizer/pull/199
- https://github.com/zeux/meshoptimizer
- https://vrm-validator.fern.solutions/
Implicated Benefits
Creating a web-based tool that allows users to optimize their VRM avatars with reduced file size and GPU footprint while maintaining visual quality is likely to have several positive implications for users, developers, and the virtual reality (VR) community at large. Here are some potential implications:
-
Improved Performance:
- Smaller file sizes and GPU footprints mean that VR experiences will run more smoothly, especially on devices with limited resources. This can lead to improved frame rates and overall performance, contributing to a better user experience.
-
Increased Accessibility:
- Optimized avatars are more likely to work well on a broader range of VR hardware, making VR experiences more accessible to users with various devices and specifications.
-
Faster Loading Times:
- Smaller file sizes result in faster loading times, which is crucial for VR experiences where latency can impact immersion. Users can enter VR environments more quickly, enhancing the overall experience.
-
Bandwidth Savings:
- Reduced file sizes can lead to lower bandwidth requirements when downloading or sharing avatars. This is particularly beneficial for users with limited internet connectivity or for platforms that may have bandwidth constraints.
-
Compatibility Across Platforms:
- Avatars optimized with your tool are more likely to be compatible with a wider range of VR platforms and applications, allowing users to seamlessly use their avatars across different environments.
-
Community Collaboration:
- The availability of such a tool could foster collaboration within the VR community. Users might share tips and best practices for avatar optimization, leading to a more knowledgeable and engaged user base.
-
Cost Savings for Developers:
- Developers hosting VR experiences may benefit from reduced server costs due to smaller file sizes. Additionally, optimized avatars could result in lower data transfer costs for users.
-
Encouraging VR Adoption:
- A tool that makes VR more accessible and enhances performance could contribute to broader VR adoption. Users may be more inclined to explore and participate in VR communities if they have a positive and optimized experience.
-
Reduced Environmental Impact:
- Smaller file sizes contribute to reduced data transfer and storage requirements, potentially lowering the environmental impact associated with data centers and internet infrastructure.
-
Customization and Personalization:
- Users may be more inclined to experiment with different avatars and customize their VR experience if optimization is straightforward. This could lead to a more vibrant and diverse virtual world.
However, it's essential to consider potential challenges, such as ensuring the tool is user-friendly, addressing concerns about data security and privacy, and providing proper documentation and support for users. Additionally, ongoing updates and improvements to the tool will likely be necessary to accommodate evolving VR standards and technologies.
