# VRM Estimates

## Abstract

Since we have most of the 3D models done, I wanted to know for each Anata VRM what the sum of the parts are in terms of triangles, draw calls, and file sizes. Worked on some scripts to check the metadata of Anata, and match against the renamed versions the glbs are based on along with all of their stats, for each of the attributes.

These measurements can help us project performance budgets for the various traits and make some early decisions as we get deeper into the QA process.

## Steps

There were multiple files that contained information about the assets for male / female / shared in `metadata/stats` folder for $body_viz.json (or csv). First order of business was combining into one which I modified `scripts/visualize_stats.py` for with -m flag. There's a small bug currently which doesn't output a valid json, but it's easy to clean up.

Then I modified the data structure of master_viz.json to which the "Original" name became a parent object for each node. I used python for accomplishing such:

import json

# Load the JSON data
with open('master_viz.json', 'r') as json_file:
    data = json.load(json_file)

# Create a new data structure with "Original" as a parent node
new_data = {
    "Original": [item for item in data]
}

# Save the modified JSON to a file
with open('master_viz2.json', 'w') as json_file:
    json.dump(new_data, json_file, indent=4)

So now master_viz2.json looks like this:

{
    "Abstract Vision Brace + Cursed Brace": [
        {
            "Category": "BRACE",
            "Name": "Abstract_Vision_Brace_plus_Cursed_Brace",
            "Size": 1058.57,
            "Images": 2,
            "Draw calls": 2,
            "Triangles": 28,
            "File Path": "files/male/BRACE/Abstract_Vision_Brace_plus_Cursed_Brace/Abstract_Vision_Brace_plus_Cursed_Brace.glb"
        }
    ],
    "Abstract Visions of Flame Brace": [
        {
            "Category": "BRACE",
            "Name": "Abstract_Visions_of_Flame_Brace",
            "Size": 3140.28,
            "Images": 4,
            "Draw calls": 4,
            "Triangles": 10237,
            "File Path": "files/male/BRACE/Abstract_Visions_of_Flame_Brace/Abstract_Visions_of_Flame_Brace.glb"
        }
    ],
...

It makes it easier to grab the child objects when searching for the parent which both the original Anata metadata and the master JSON containing all the relevant stats for glb models have. For example:

`$ jq '."Katana Trio"[]' master_viz2.json`

yields us this:

{
  "Category": "WEAPON",
  "Name": "Katana_Trio",
  "Size": 867.37,
  "Images": 1,
  "Draw calls": 1,
  "Triangles": 5562,
  "File Path": "files/shared/WEAPON/Katana_Trio/Katana_Trio.glb"
}

Now here are steps I repeated for male / female for getting stats about each future VRM:

1. From repo cd to `anata/vrm/female`
2. `$ mkdir out`
3. `$ for i in */*.json; do cat "$i" | jq -r '[.attributes[]] | "[" + (map(tostring) | join(",")) + "]"' > out/$(basename $i); done`

I wanted to make these JSON files match the master_viz2.json more closely, so I searched and replaced the trait_type values to the name scheme I have across the rest of the project. For future reference, it's best to be as consistent as possible from the source in terms of name schemes.

#!/bin/bash

# Define the search and replace mappings
declare -A mappings=(
  ["Body"]="BODY"
  ["Brace"]="BRACE"
  ["Clips and Kanzashi"]="CLIPS_AND_KANZASHI"
  ["Clothing"]="CLOTHING"
  ["Earring"]="EARRING"
  ["Eyes"]="EYES"
  ["Face Other"]="FACE_OTHER"
  ["Glasses"]="GLASSES"
  ["Hair"]="HAIR"
  ["Hair Accessory Other"]="HAIR_ACCESSORY_OTHER"
  ["Halos"]="HALOS"
  ["Hats"]="HATS"
  ["Head"]="HEAD"
  ["Head Accessory Other"]="HEAD_ACCESSORY_OTHER"
  ["Masks"]="MASKS"
  ["Neck"]="NECK"
  ["Ribbons and Bows"]="RIBBONS_AND_BOWS"
  ["Set"]="SET"
  ["Sigil"]="SIGIL"
  ["Special Other"]="SPECIAL_OTHER"
  ["Tail"]="TAIL"
  ["Tattoo"]="TATTOO"
  ["Type"]="TYPE"
  ["Weapon"]="WEAPON"
  ["Weapon Brace"]="WEAPON_BRACE"
  ["Wings"]="WINGS"
)

# Iterate over JSON files and perform search and replace
for file in *.json; do
  # Backup the original file
  cp "$file" "$file.bak"

  # Perform search and replace using jq
  for search in "${!mappings[@]}"; do
    replace="${mappings[$search]}"
    jq --arg search "$search" --arg replace "$replace" '.[] |= if .trait_type == $search then (.trait_type |= $replace) else . end' "$file" > "$file.tmp" && mv "$file.tmp" "$file"
  done
done

echo "Search and replace completed."

This might make it easier to gather stats on categories later. Finally, we create a new JSON object that contains info about all the assets we have per metadata for each Anata. This is saved as `scripts/estimate_vrm.sh`:

#!/bin/bash

# Directory containing the JSON files
json_dir="vrm/female/out"

# Path to the master_viz2.json file
master_file="metadata/stats/master_viz2.json"

# Create the output directory if it doesn't exist
output_dir="${json_dir}/output"
mkdir -p "${output_dir}"

# Iterate over each JSON file in the directory
for file in "${json_dir}"/*.json; do
  filename=$(basename "$file")
  output_file="${output_dir}/new_${filename}"

  # Read the JSON file
  data=$(cat "$file")

  # Extract the values from the JSON file
  values=$(jq -r '.[].value' <<< "$data")

  # Initialize an array to store the results
  results=()

  # Iterate over each value and search in master_viz2.json
  while IFS= read -r value; do
    result=$(jq -r --arg value "$value" 'to_entries[] | select(.key == $value) | .value' "$master_file")
    results+=("$result")
  done <<< "$values"

  # Combine the keys and values into a new JSON object
  keys_values=$(jq -n --argjson keys '[]' --argjson values "$(printf '%s\n' "${results[@]}" | jq -s '.' || echo '[]')" '$keys + $values')

  # Create a new JSON array with the keys and values
  output="[$(printf '%s\n' "$keys_values" | jq -s '.[]')]\n"

  # Create a new JSON file with the results
  echo -e "$output" > "$output_file"

  echo "Created: $output_file"
done

Here's an example output from `vrm/male/out/output/new_1000.json`

[[
  [
    {
      "Category": "CLOTHING",
      "Name": "Martial_Arts_Robe",
      "Size": 7393.4,
      "Images": 2,
      "Draw calls": 4,
      "Triangles": 6663,
      "File Path": "files/male/CLOTHING/Martial_Arts_Robe/Martial_Arts_Robe.glb"
    }
  ],
  [
    {
      "Category": "HALOS",
      "Name": "Bloody_Halo",
      "Size": 280.06,
      "Images": 1,
      "Draw calls": 1,
      "Triangles": 2044,
      "File Path": "files/shared/HALOS/Bloody_Halo/Bloody_Halo.glb"
    }
  ],
  [
    {
      "Category": "WEAPON",
      "Name": "Katana_Arsenal",
      "Size": 1623.03,
      "Images": 1,
      "Draw calls": 1,
      "Triangles": 35208,
      "File Path": "files/shared/WEAPON/Katana_Arsenal/Katana_Arsenal.glb"
    }
  ],
  [
    {
      "Category": "SIGIL",
      "Name": "Angel_Sigil_plus_Spike_Sigil",
      "Size": 235.4,
      "Images": 1,
      "Draw calls": 1,
      "Triangles": 2,
      "File Path": "files/shared/SIGIL/Angel_Sigil_plus_Spike_Sigil/Angel_Sigil_plus_Spike_Sigil.glb"
    }
  ],
  [
    {
      "Category": "BRACE",
      "Name": "Festival_Mask_Oni_Brace",
      "Size": 1680.41,
      "Images": 1,
      "Draw calls": 1,
      "Triangles": 34836,
      "File Path": "files/shared/BRACE/Festival_Mask_Oni_Brace/Festival_Mask_Oni_Brace.glb"
    }
  ]
]]

Here's the original metadata for Anata 1000 for reference:

{"description":"Anata NFT is a brand new form of NFT identity. It's the first NFT you can literally be. Anatas use your webcam or phone camera to mimic your movements exactly. When you speak, your Anata speaks. When you blink, your Anata blinks. Move your head back and forth, raise your eyebrows, squint your eyes — it's all reflected immediately by your Anata. Each Anata is bundled with a set of emotes so you can be extremely expressive. Fire eyes, anger, peace signs, shame, heart eyes, and more.","type":"live2d","project":"anata","name":"1000","image":"https://arweave.net/cz9lS2Pi4jLa3DLtf2_nhNn0DWwtihd6u8lcnr9Xco4/1000.png","animation_url":"https://arweave.net/iCNUMyKGcOowz4vtJ6KMMCvseWT-bM4oMPl1Gpi166A/M_SET0973.mp4","model_url":"https://arweave.net/rhNYuVOZ4DoGmjDWc3fp4vTObr2ghH5q0p0Cd88f5q8/M_SET0973.zip","attributes":[{"trait_type":"Body","value":"Masculine"},{"trait_type":"Clothing","value":"Martial Arts Robe"},{"trait_type":"Hair","value":"Mid Gray"},{"trait_type":"Type","value":"Human"},{"trait_type":"Halos","value":"Bloody Halo"},{"trait_type":"Weapon","value":"Katana Arsenal"},{"trait_type":"Sigil","value":"Angel Sigil + Spike Sigil"},{"trait_type":"Brace","value":"Festival Mask Oni Brace"}],"animation_preview_url":"https://arweave.net/MJrr0JFrSMf9rQyUnbtHZVQ5g6RehULRI6QcO0tBrFg/M_SET0973.png","pfp_url":"https://arweave.net/cz9lS2Pi4jLa3DLtf2_nhNn0DWwtihd6u8lcnr9Xco4/1000.png"}

Now we can get some useful data from this! For example, to see all the file sizes you can do this:

`$ jq '[.. | .Size? // empty]' vrm/male/out/output/new_1000.json`

[
  7393.4,
  280.06,
  1623.03,
  235.4,
  1680.41
]

This command will add them all up

`$ jq '[.. | .Size? // empty] | add' vrm/male/out/output/new_1000.json`

Which totals to about `11212.3 KB or 11.2 MB`

When we add in the base body (3.6 MB) its ~ <15 MB total, not bad!

We can generate a CSV containing information about all the files per BODY (male / female) like so:

echo "ID,Size,Draw Calls,Images,Triangles" > filesizes.csv && for i in *.json; do echo "$(basename "$i" .json)","$(jq '[.. | .Size? // empty] | add + 6000' "$i")","$(jq '[.. | ."Draw calls"? // empty] | add' "$i")","$(jq '[.. | .Images? // empty] | add' "$i")","$(jq '[.. | .Triangles? // empty] | add' "$i")" >> filesizes.csv; done

After getting the filesizes.csv, you can print out some useful data using this script:

#!/bin/bash

csv_file="filesizes.csv"
columns=(2 3 4 5)

for column in "${columns[@]}"; do
  # Extract the desired column
  column_values=$(tail -n +2 "$csv_file" | cut -d',' -f"$column")

  # Find the minimum value
  min=$(echo "$column_values" | awk 'NR==1 { min=$1 } $1 < min { min=$1 } END { print min }')

  # Find the maximum value
  max=$(echo "$column_values" | awk 'NR==1 { max=$1 } $1 > max { max=$1 } END { print max }')

  # Calculate the sum
  sum=$(echo "$column_values" | awk '{ sum += $1 } END { printf "%.2f", sum }')

  # Calculate the average
  count=$(echo "$column_values" | wc -l)
  average=$(awk -v sum="$sum" -v count="$count" 'BEGIN { avg=sum/count; printf "%.2f", avg }')

  # Print the results
  case $column in
    2)
      echo "**Size**"
      ;;
    3)
      echo "**Draw Calls**"
      ;;
    4)
      echo "**Images**
      ;;
    5)
      echo "**Triangles**"
      ;;
  esac

  echo "Minimum: $min"
  echo "Maximum: $max"
  echo "Sum: $sum"
  echo "Average: $average"
  echo
done

To get the stats below. Reminder: These are stats roughly reflect the *combination* of assets per each VRM, not as each individual trait but moreso the sum of their parts.

## Female VRM Stats

**Size**
Minimum: 6472.08
Maximum: 51319.89
Sum: 14245453.95
Average: 14245.45

**Draw Calls**
Column: Draw Calls
Minimum: 1
Maximum: 22
Sum: 5977.00
Average: 5.98

**Images**
Minimum: 1
Maximum: 12
Sum: 4802.00
Average: 4.80

**Triangles**
Column: Triangles
Minimum: 2900
Maximum: 380283
Sum: 56560831.00
Average: 56560.83

## Male VRM Stats

**Size**
Minimum: 3525.66
Maximum: 45812.55999999999
Sum: 16579220.61
Average: 16579.22

**Draw Calls**
Minimum: 1
Maximum: 17
Sum: 7450.00
Average: 7.45

**Images**
Minimum: 0
Maximum: 19
Sum: 7055.00
Average: 7.05

**Triangles**
Minimum: 804
Maximum: 205288
Sum: 27566221.00
Average: 27566.22

---