HackMDAnata Challenges
Kiiba
We have faced quite some challenges, the biggest one in my opinion being how badly some of the assets have been named. We're really struggling with that when it comes to the hair models. We have modelled all the hair parts individually; that means front hair, left hair, right hair, back hair, pony tail and ahoge. The hairs on opensea are named by color; eg. Mid Black. That says nothing about what type of hair it is, other than the color. That means that we individually have to check every piece of hair in the collection, make sure that it matches with the 2D illustration. This costs A LOT of time, especially since there are many many variations of hair. We cannot generate these hairs and the name only tells us more about the color, and even that is wrong. There are hairs with 'red' in the name that turned out to be completely different colors like green and yellow.
The whole occurrence also happens with several other assets like the clothing and wearables, but it's less common as they do describe the outfits better. On top of that, many assets have unique textures. That means that yes, we may have 3D meshes that we can reuse, but we need an artist to make that unique texture. Some are as simple as a recolor, but some are complicated patterns that can be difficult to recreate, as we do not always have the original files for these patterns.
Another thing we struggle with is the fast that 2D concepts don't always work as well in 3D (expectation vs. reality). One big example is hats. In an illustration you can easily make the hair disappear underneath a hat, as it simply covers the illustrated part underneath, in 3D it's not that easy. You will have hair mesh sticking through the hat. There are two possible solutions that i can personally think of; you remove hair so the hat fits perfectly, but removing the hat means the hair will look weird, as some parts have been removed. Or secondly being that we scale the hat to be bigger or put it in a different position/angle.
Boom
Some challanges we faced during production was having two art teams for male and female, armature and rigging merging, heavy sanitation of files, sanitation of metadata and traits, creating a system of feedback and iterations, creating devOps workflow for managing massive projects with version control, creating a platform / web based utility with robust features that were custom made for optimization, and production streamlining.
Meme
Alright!, this are a small summary of what we've gone through in code section 🙂
We needed to find a way to automate loading nft data files and exporting a usable vrm file, but alot of traits from the nft data shared the same id, yet they looked different, so we also needed to develop a way to load different models that had the same ID. Batch exporting was created to fix this with the ability to choose loading directly from a manifest or from attributes of specific manifest files
We had (and still have) VRM merging issues: being over 2k different models, some of them in some cases have different features, in most cases this doesnt affect the result, but in cases were it does, we needed to find out what was the differnce, and add code to manage correctly this specific cases to work correctly.
We needed to make the character studio easier for testing and quick previewing, so we needed to allow artists/users to drag and drop any VRM models / json attributes / textures and animations to preview results without going through the whole process of creating a manifest file, this improved testing times
An optimization tool was also made in case artists needed to export models without modificationa, fine tune, and merge result into an optimized character
The next improvements/updates were made to improve usability and optimizations for characters exported from character studio, we faced issues while developing these, but the end result made a better product:
- Merging spring bones from the different traits without repeating bones
- Ability to define colision bones and export them in final vrm file
- Auto atlas texturing and material conversion (standard to mtoon)
- Merging visemes from different traits
- GLTF techniques (sparse accessors) To reduce file size.
- Load traits by nft attributes
- Manifest append to load set of traits and load new wearables into character
- Screenshot taker for quick previewing and vrm thumbnail image
- Improved mesh autoculling to avoid app crashing and letting user turn on hidden faces.
Alsara
Would like to share a brief review of the Anata QA process which I've been involved in and Jins work and contributions.
A big shortcoming in the assembly process was that some of the Anata metadata traits did not correspond with their art assets, meaning each 3D character needed to be manually reviewed in a QA process to ensure they were being exported with the correct trait models. For some anatas, existing 3D trait assets didn't exist and so would have to be built from scratch. This issue was tackled by Jin by creating a QA pipeline that a number of artists could be brought into, which also meant new users could test Character Studio and report back any bugs they encountered with the program. Mis-matched metadata has been particularly common for the hair traits, where a whole number of different hair styles and colors could fall under a metadata trait name like "Mid Blue". Understandably, for a program like Character Studio which depends on metadata naming for its assembly process, this meant that each Anata would have to be manually reviewed during the assembly process because the metadata no longer served as a true descriptor that the program could depend on.
Another moment is that artists have to be shown how to use Character Studio and have to be inducted into the review process, for this Jin has been instructing artists one on one and writing documentation for Character Studio, which is a time consuming task and usually means a call between the artist and Jin to go over finer details. Through the assembly QA process, artists require direct instruction and need to have their work reviewed to ensure the worked on VRM models are exported and done correctly, which means that Jin has to complete the additional step of manually reviewing each artists work and flagging any issues back to them. Jin also has to manage artist tasks on Dework and artist payroll, and in general the management overhead he undertakes is immense, as nobody else is involved in managing the Anata male team. Briefly, there was also an absence of artists available to work on the QA process during the winter holiday period, who are now back, helping speed up the assembly process. Jins project management work is only compounded with his additional responsibilities in communicating with the Anata team, the female assembly team, managing the Character Studio devs, writing documentation and github issues, recording internal video tutorials for artists, and more. The artists and devs on the male team appreciate the work he has been putting in and I personally feel like the project would take much longer to complete was it not for the dedication and overtime he has put in.
