---
title: Avatar LOD
description: Technical concepts to scale the number of simultaneous users per virtual worlds by an order of magnitude.
image: https://i.imgur.com/FwYuprK.jpg
robots: index, follow
lang: en
dir: ltr
breaks: true
disqus: xrdevlog
---

# Avatar LOD

{%hackmd theme-dark %}

###### tags: `meebitsdao` `devlog`

- :house: [**Back to xrdevlog home**](https://xrdevlog.com)
- :arrow_right: [**Part 1: Avatars**](https://hackmd.io/@XR/avatars#Spritesheets)
- :arrow_right: [**Part 3: NFT 3D avatars**](https://hackmd.io/@XR/nftavatars)

**See part 1: https://hackmd.io/@XR/avatars#Spritesheets**

Many virtual worlds struggle to scale beyond 60-100 concurrent users per server instance, especially those that support UGC and custom avatars. The bottle neck is usually that people are not the best at optimizing their own avatars.

I believe I may have a scaling solution for virtual worlds to be able to support the number of custom avatars in a single world an order of magnitude higher than what's possible today.

The concept utilizes popular gamedev optimization techniques typically used for environments. This time the further away an avatar is from another player, the less information they have to send - including details about the avatar itself.

However, one of the bottlenecks for UGC platforms is that people are not good at optimizing their own avatars. Even some of the most popular avatar builders like [ready player me](https://readyplayer.me/) and [VRoid studio](https://vroid.com/en/studio) aren't all that optimized enough.

I propose we build a system which takes any VRM file, animates it with a walk cycle (to start with), and generates a sprite sheet out of it. The likeness of a users avatar from any angle will be visually preserved while optimizing down to 2 triangles and 1 texture material.

## Ecosystem Background

VRChat is the world's biggest social VR platform with over 1M custom avatars uploaded to it. Their solution to the performance problem is by implementing a rating system and replacing avatars that exceed a user's settings with a grey robot or 'fallback avatar'.

https://medium.com/vrchat/vrchat-on-quest-avatar-performance-woes-1a3cc228a442

Here's an awesome video that overviews the vrchat fallback avatar system.

<iframe width="100%" height="315" src="https://www.youtube.com/embed/Bm0lkuN_T6g" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

If you're on an underpowered system you probably have seen many of these grey robots. It sucks a lot of the charm from the experience.

How VRChat would look on a low end machine.

Recently VRChat pushed an update to utilize the fallback avatars more for performance settings. See the announcement on [twitter](https://twitter.com/vrchat/status/1438268399197597702).


Fallback avatars are the backup avatars you pick per slot for people on low powered systems or Oculus Quest. It's a bit jarring to have different realities where you might appear completely different to the person you're talking to.

Altspace is another highly popular social VR platform. During a large gathering at the 2020 virtual burning man event I noticed something interesting, some of the avatars from a distance were replaced with billboarded images that changed into full 3D when I walked up to them.


Cryptovoxels also implemented a similar sysem in 2020. The avatars looked like plain Cryptovoxels avatars. They enabled low end machines to be able to render 100s of avatars which was useful during large events.

https://twitter.com/cryptovoxels/status/1242944487422214145

The issue with them is that it was very hard to tell who was who when we all became the same sprite. Typically Cryptovoxels avatars are highly expressive with custom wearables.


I think many people would like their fallback avatars to look like their main, only more optimized. What if there's a way to automate such?

> Note: [Webaverse](https://webaverse.com) has an automated pipeline going from VRM -> 2D sprite sheet

---

### Level of detail

When you're standing far away from another avatar or object it doesn't matter if its in full 3D or a cardboard cutout it will look just the same from your perspective.

Notes: https://hackmd.io/@XR/avatars#LOD-Avatars

Game devs use this as an optimization technique in order to render vast immersive environments as far as the eye can see. What if we simply apply the same logic to avatars?

> 
https://twitter.com/dankvr/status/1437145019698368512

When it comes to the metaverse, we can use it to enable virality and the existence of crowds in virtual spaces by reducing how much data our computers download and send each other about avatars in the scene that are further away from the camera.

https://hadean.com/blog/mmorpgs-part-1-its-time-to-rethink-game-architectures/

---

## 1. VOX -> VRM


When Meebits dropped, I wanted to demonstrate the power of VRM file format to thousands of collectors in the community. The manual method required 3 dependencies and like 12 steps, but it was enough to inspire more innovation.

Instructions: https://madjin.github.io/docs/docs/create/import-meebit

<iframe src="https://player.vimeo.com/video/609972600?h=5bd34d566b" width="100%" height="360" frameborder="0" allow="autoplay; fullscreen; picture-in-picture" allowfullscreen></iframe>

MeebitsDAO saw the video and helped streamline the process with [meebits-blender-utils](https://github.com/MeebitsDAO/meebits-blender-utils). With this tool it's possible to seamlessly convert every meebit to glb, VRM, fbx, and obj. Better yet, the tool was written in such a way that it would work for any t-posed humanoid voxel art.

Github: https://github.com/MeebitsDAO/meebits-blender-utils

See full video: https://www.youtube.com/watch?v=aPv0Gnx0JQg

## 2. Creating the Sprite Sheet

**What is a sprite sheet?**

> A sprite sheet is an image that consists of several smaller images (sprites) and/or animations. Combining the small images in one big image improves the game performance, reduces the memory usage and speeds up the startup and loading time of the game. [source](https://www.codeandweb.com/what-is-a-sprite-sheet#:~:text=A%20sprite%20sheet%20is%20an,loading%20time%20of%20the%20game.)


> Note: Sprite sheets can be generated with other animations as well. 

Few things are needed for this process:

- Avatar file
- Walk animation
- 4-8+ camera positions


It might seem redundant to generate a sprite sheet of a meebit since meebits are super optimized and low poly already. I figured since MeebitsDAO did such a great job with the blender-utils and helped red pill a wide community on open file formats like VRM we could continue the positive trend.

**Steps**

First need to upload obj, mtl, and texture zipped to mixamo and select Walking animation with 'in place' checked. Then download the DAE file with the skin.


For the proof of concept I used https://www.webglstudio.org/ with the spritesheets plugin. Detailed instructions are on https://hackmd.io/@XR/avatars#Convert-VRM-to-2D-spritesheet


How-to video: https://www.youtube.com/watch?v=2tid3_06X_U


This workflow kinda sucks, it's hard to position camera directly in the center.



### Blender pipeline

I believe that the blender workflow is the way to go, as it was with the meebits blender utils. If we can import a humanoid VRM file, animate a walk cycle on it, and then export a spritesheet, it would be a huge step forward.

- https://blendermarket.com/products/spritehandler
- https://blendermarket.com/products/get-sheet-done

**Get Sheet Done**

This plugin generates grid-based spritesheets right inside Blender from images or when rendering an animation. Things needed to produce a sprite sheet using this tool:

- Cameras all setup with correct angle and distance
- VRM avatar file
- Ability to set walk animation on arbitrary VRM
- Export with transparency


### Webaverse Pipeline

- https://github.com/avaer/avatar-asset-pipeline#create-level-of-details-lod
- https://github.com/webaverse/app/blob/master/avatar-spriter.js

VRM -> Spritesheet pipeline just needs an export option (clip taken 10-13-21)



To test: go to settings -> graphics -> avatar quality -> low


3D wearables can equip onto 2D sprite avatars, which look pretty cool. From a distance it sorta even ceases to appear as a 2D avatar when a 3D sword is on your back.


This can save a lot of bandwidth and make everything run smoother for people since custom avatars average 10-20mb each and if you have a world with up to 40 people it adds up quickly.


Perhaps just you and your party or those within radius of your avatar are in 3D while everyone else is 2D in order to boost performance.

#### More Links

- https://opengameart.org/forumtopic/how-to-make-a-2d-isometric-spritesheet-from-a-3d-model-using-blender
- https://github.com/theloneplant/blender-spritesheets
- https://github.com/johnferley/Game-Sprite-Creator
- https://www.youtube.com/watch?v=-zpORxZF4FE
- https://www.youtube.com/watch?v=txh4CgzQmpg
- https://assetstore.unity.com/packages/tools/animation/sprite-sheet-creator-57044
- https://assetstore.unity.com/packages/tools/sprite-management/sprite-baking-studio-3d-to-2d-31247
- https://ngc-games.itch.io/sprite-sheet-generator

### VRChat avatar setup

Guide: https://vrcat.club/threads/retro-sprite-shader-v2.2222/

In a new Unity project I imported [retrospritesV2.unitypackage](https://drive.google.com/file/d/1bIWSjK-4uYNb1x1qcUmsNzJ9c2OxqhlX/view) and VRChat avatar SDK 3.0 then followed the instructions in the vrcat.club thread. After I made the Texture Array I just replaced the texture in the sample file from the project.


Here's how it looks when I hit play and then flip to the Scene tab.


Afterwords I just need to add an avatar scene descriptor to the gameobject then upload to VRChat as an avatar. For some reason I was halfway in the ground, obviously I need to do more work on this demo.


Important thing is that the shader effect works! Sprite avatars could be used as fallback avatars in the future, they are incredibly optimized for it (2 polygon only).


---

## Distributed Architecture

> Note: I'm not an expert in large scale multiplayer server architecture, feedback is welcome.

In order to handle so many simultaneous user connections you want to do stuff like **lower frequency of updates for avatars that are further away and lower quality versions of the assets that get sent/displayed**, hence avatar LOD.

There was some insightful knowledge dropped in this hacker news thread: https://news.ycombinator.com/item?id=21093320

<iframe width="560" height="315" src="https://www.youtube.com/embed/QeZtqoydXpc" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

#### P2P Mesh

Full mesh topology means every client is connected to every other client. Everytime a person joins, they open connections to everyone else. Bandwidth and CPU/GPU consumption increases linearly.


It's good for party systems, such as squadding up with your friends like in Fortnite, but inefficient at scale. Even on a fast network connection and beefy computer the performance quality starts degrading around ~10 audio-only participants.

https://dev.to/lazerwalker/why-video-chat-is-a-hard-technical-problem-43gj

#### Octrees

An Octree is one technique to used to speed up rendering (or collision detection) by reducing the size of the [Potentially Visible Set](https://en.wikipedia.org/wiki/Potentially_visible_set) of geometry that is sent to the rendering pipeline.

> As I was programming my 3D browser shooter I was particularly unhappy the way I handle bullet / ship collisions. So I started looking for a better solution, say hello to Spatial Partitioning! Instead of checking every bullet against every ship (O(n^2)) you can instead split up the world in partitions with the help of an octree. Every node in this tree represents a 3D space and each space allows a maximum number of entities. If the number of entities exceed the maximum, it splits the space into 8 smaller ones. It keeps doing this recursively. Now, for every bullet you can query the tree for other ships in the same space (node) instead of checking EVERY ship in the world. This makes this new method way more efficient (O(log n)) when handling a lot of entities.
>
Link: https://github.com/nickyvanurk/3d-spatial-partitioning

#### More Links

- https://documentation.improbable.io/spatialos-overview/docs/workers-and-load-balancing
- http://rp1.com/
- https://en.wikipedia.org/wiki/Croquet_Project
- https://croquet.io/docs/croquet/

---

## Possible future integrations

### VRChat

Easy process for users to make lofi sprite versions of their main avatars to use as the fallback. Benefit is that they both will share a resemblence.

### Atlantis0x

https://twitter.com/atlantis0x

It's like a crypto-native Gather Town, still in very early stage development.


> All of our game assets, even textures, are stored securely on-chain + loaded directly from @IPFS
>
> On top of that, we built a museum for viewing + buying NFT collections on @rarible

### Gather Town

https://twitter.com/gather_town

It's like pokemon meets zoom.

They don't seem like they have custom avatar support or are working on web3. Maybe they'll see what Atlantis0x is doing and offer some level of support? I saw a dev in open metaverse interoperability discords like M3 and OMI.

### Treeverse

https://play.treeverse.net/

Browser based MMORPG inspired by Old School Runescape that's currently in private alpha. Will have play to earn features, private homes, and more.


The sprite avatars here are more detailed than others in resolution and actions. This is what the avatar configurator looks like when you login Treeverse for the first time:


It's still too early to tell if/when custom sprite avatars are released. Perhaps it can be done per collection (meebits, bored apes, etc.)

### Webaverse

https://twitter.com/webaverse

Doom sprite support would not be that difficult to add in. There's a process for loading in 3rd party NFTs and generating map tiles as well: https://github.com/search?q=org%3Awebaverse+map+tile&type=code

**Notes**


- https://ianbelcher.me/tech-blog/building-a-webxr-version-of-doom#sprites
- https://gamedev.stackexchange.com/questions/115323/doom-style-2-5d-movement-animations
- https://threejs.org/docs/#api/en/materials/SpriteMaterial

### JanusWeb

https://twitter.com/janusxr

Sprite avatar support would likely not be that difficult to add in. Bai has been working on loading Doom from wad files inside JanusWeb if you check his youtube channel.

<iframe width="100%" height="315" src="https://www.youtube.com/embed/SzFd_CqcC3Q" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

### Cryptovoxels

https://twitter.com/cryptovoxels

Had sprite avatar support in the past, removed in later version, perhaps can be added back in?

<iframe width="100%" height="315" src="https://www.youtube.com/embed/GeRf7O0QjT8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

---

### LoFi Metaverse

Internet maps: https://hackmd.io/@xr/maps

What if you could play Cryptovoxels or Substrata like you play Gather Town or Atlantis0x? Perhaps the top down renders originally intended as map tiles can double as the lofi experience of virtual worlds.


Simple MVP could be adding in multiplayer sprite avatars to a top down ortho render of the map.


Top down ortho view taken in [Substrata](https://substrata.info) which can load in Cryptovoxels data as well. Imagine adding live data to the map.


### MUGEN

M.U.G.E.N is a 2D fighting game engine that supports user made content

https://arstechnica.com/gaming/2020/08/how-the-mugen-community-built-the-ultimate-fighting-game-crossover/

Watch live: https://www.twitch.tv/saltybet

Kung Fu Man is the very first character. The sprite sheet is 277kb.

https://mugen.fandom.com/wiki/Kung_Fu_Man

<iframe width="560" height="315" src="https://www.youtube.com/embed/9PCeVDNQT0k" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

https://mugenguild.com/forum/topics/super-smash-bros-ultimate-mod-mugen-1-1-188188.0.html

---

## Advanced Sprites



Watch on youtube: https://www.youtube.com/watch?v=HsOKwUwL1bE
{%youtube HsOKwUwL1bE %}



---

## Notes

- https://hackmd.io/@XR/avatars#Spritesheets
- https://www.spriters-resource.com/
- https://twitter.com/dankvr/status/1437145036983001091
- Godot engine
- https://github.com/godotengine/godot/pull/52544#issuecomment-927967193
- https://github.com/wojtekpil/Godot-Octahedral-Impostors

Awesome game blending pixel art and 3D: https://twitter.com/pixelated_milk

#### Protogen

https://kaelygon.gumroad.com/l/sprite-proto

Heavily modified version of Error's [vrchat sprite shader](https://vrcat.club/threads/retro-sprite-shader-v2.2222/) with tons of advanced features added. This avatar is for sale, would be good to get one for research perhaps!

<iframe width="100%" height="315" src="https://www.youtube.com/embed/S7fgPsUn7zQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

#### VRchat Sprite Generator

What if there was a method of generating a spritesheet from one’s avatar as easy as stepping into a booth and getting a scan like those body scanners at conventions?

<iframe width="100%" height="315" src="https://www.youtube.com/embed/6e-Hhz0vQe8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

This sprite machine is located at The Sandbox Dance Lounge by Splinks (pictured below): https://vrchat.com/home/launch?worldId=wrld_aea5c1d3-7abc-4287-acf8-200279c12700

To export the sprite sheet one would just need a hotkey to full screen the render. This setup doesn't capture any animations but perhaps a future iteration might.

#### NeosVR Sprite Generator

Asked in the NeosVR discord if anyone is doing anything interesting with sprite avatars. It seems like a few people are, and that there's a gadget that can create a sprite sheet of an avatars head. Perhaps it can be modified for the whole body.

https://www.landontownsend.com/single-post/super-imposter-shader-my-journey-to-make-a-high-quality-imposter-shader

https://twitter.com/RustedMoss/status/1556693718551388161