# **Overview**

We are using v-tubing techniques to bring paintings and sculptures to life at a museum.
There are currently 3 types of vtubing -
* Png Vtuber
* 2d Vtuber and
* 3d Vtuber

What's the Difference Between PNG, Live2D, and 3D VTuber Models?
<iframe width="560" height="315" src="https://www.youtube.com/embed/SIK2ahKt6I8?si=QDp_R4115Sal7yEq" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

---

# **The pipeline for 2d**
(currently working well)
Prep artwork in photoshop or krita. Strictly follow naming conventions
.PSD files are used
Rig in [Live2D Cubism](https://www.live2d.com/en/) or [Inochi Inochi2D](https://inochi2d.com/)
[VTube studio](https://denchisoft.com/) for camera tracking and deployment.

Essential Process:


* Edit the image in Photoshop or Krita(any program that can output a .psd file), remove background and seperate facial features and limbs to different layers.


* Import the image and rig it in Live 2d Editor
* Export .moc3 and json files.
* For idle animation data export .can3 files
* Copy your character folder into live2d's folder structure.

* import the character into VTube Studio and configure it.

*Reference material*:
**For facial rig**
BEGINNER Live2D tutorial Intro + layering (+Free model PSD giveaway)
<iframe width="560" height="315" src="https://www.youtube.com/embed/QWxgh1PP4z0?si=tCo9n7LjWPY4bJ7q" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

And for **hand rig**
Live 2D Hand Rigging Tutorial - Part 1 - Setting Up Asset(view the whole playlist)
<iframe width="560" height="315" src="https://www.youtube.com/embed/Uea8g4iRR6I?si=P8P5jmHk732eF7CQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

---

# **The pipeline for 3d**
(currently working not so well)
Modelling and texturing with Blender(or other similar software)
.VRM files are used
Rig with rigify or Mixamo or adapting a rig from a vroid studio model
**Warudo for camera tracking and live deployment.** [3D VTubing Primer - BlendShape](https://docs.warudo.app/docs/tutorials/3d-primer)
Alternative is [Vseeface](https://www.vseeface.icu/)
Alternative workflow for creating is using vroid studio’s creation tools. Or [Live 3d](https://live3d.io/)

Essential Process:
* Import or create the 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into the camera tracker of choice

*Reference material*:
The Art of Making 3D Vtuber Models (For Dummies)
<iframe width="560" height="315" src="https://www.youtube.com/embed/eFkwu7p1LhU?si=YBjIILi1xBsTRZVM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

How to Make a 3D VTuber Avatar From Scratch, Part 1: Design
<iframe width="560" height="315" src="https://www.youtube.com/embed/4fH0wNnM0Sw?si=07tMZTWgreZaLuvZ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

---
# Rokoko workflow

13 June 2025
Tried out Deepmotion and Rokoko Studio
<iframe width="560" height="315" src="https://www.youtube.com/embed/mifGsXHitvs?si=gYFO4nQ_Yt9vkLw2" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

Abandoned due to realtime playback not working the way we need it to. The tool is meant for motion capture and not live performance display.

---
# WebcamMotionCapture

Discovered a program called **Webcam Motion Capture** via this tweet

<blockquote class="twitter-tweet" data-media-max-width="560"><p lang="en" dir="ltr">Take a look at this dreamy anime-inspired VTuber model created by <a href="https://twitter.com/_Mari_Art?ref_src=twsrc%5Etfw">@_Mari_Art</a> in collaboration with <a href="https://twitter.com/Kana_Fuyuko?ref_src=twsrc%5Etfw">@Kana_Fuyuko</a> and <a href="https://twitter.com/TakimiLive2D?ref_src=twsrc%5Etfw">@TakimiLive2D</a>.<br><br>See more: <a href="https://t.co/bLhXkcqw7q">https://t.co/bLhXkcqw7q</a> <a href="https://t.co/TSroh8vkBt">pic.twitter.com/TSroh8vkBt</a></p>&mdash; 80 LEVEL (@80Level) <a href="https://twitter.com/80Level/status/1933413316472250830?ref_src=twsrc%5Etfw">June 13, 2025</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

---

Currently using Webcam Motion Capture
Found really good results

https://webcammotioncapture.info/beta.php


Essential Process:
* Import or create the 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into Webcam Motion Capture

---
# 2.5D workflow

19th June 2025
Currently creating a workflow where we convert 2d paintings and characters to 2.5d and use them in the 3d system - WebcamMotionCapture.

Essential Process:
* Edit the image in photoshop, remove background and get a splayed or t-pose.
* Import the image and create an extruded 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into Webcam Motion Capture