---
# System prepended metadata

title: Vtuber
tags: [art, xr, design]

---

# **Overview**
![Screenshot 2025-06-21 141418](https://hackmd.io/_uploads/H16Aug4Vxl.jpg)

We are using v-tubing techniques to bring paintings and sculptures to life at a museum. 
There are currently 3 types of vtubing - 
* Png Vtuber 
* 2d Vtuber and 
* 3d Vtuber 

What's the Difference Between PNG, Live2D, and 3D VTuber Models? 
<iframe width="560" height="315" src="https://www.youtube.com/embed/SIK2ahKt6I8?si=QDp_R4115Sal7yEq" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



---

# **The pipeline for 2d**  
(currently working well)
Prep artwork in photoshop or krita. Strictly follow naming conventions 
.PSD files are used
Rig in [Live2D Cubism](https://www.live2d.com/en/) or [Inochi Inochi2D](https://inochi2d.com/) 
[VTube studio](https://denchisoft.com/) for camera tracking and deployment.

Essential Process:

![image](https://hackmd.io/_uploads/HJsQFJVVgl.png)

* Edit the image in Photoshop or Krita(any program that can output a .psd file), remove background and seperate facial features and limbs to different layers.

![image](https://hackmd.io/_uploads/r1f5ty44xe.png)

* Import the image and rig it in Live 2d Editor
* Export .moc3 and json files.
* For idle animation data export .can3 files
* Copy your character folder into live2d's folder structure.
![image](https://hackmd.io/_uploads/Hk8R_kNVxx.png)

* import the character into VTube Studio and configure it.

*Reference material*: 
**For facial rig**
 BEGINNER Live2D tutorial Intro + layering (+Free model PSD giveaway)
<iframe width="560" height="315" src="https://www.youtube.com/embed/QWxgh1PP4z0?si=tCo9n7LjWPY4bJ7q" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

And for **hand rig**
Live 2D Hand Rigging Tutorial - Part 1 - Setting Up Asset(view the whole playlist) 
<iframe width="560" height="315" src="https://www.youtube.com/embed/Uea8g4iRR6I?si=P8P5jmHk732eF7CQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



---

# **The pipeline for 3d** 
(currently working not so well)
Modelling and texturing with Blender(or other similar software)
.VRM files are used
Rig with rigify or Mixamo or adapting a rig from a vroid studio model
**Warudo for camera tracking and live deployment.** [3D VTubing Primer - BlendShape](https://docs.warudo.app/docs/tutorials/3d-primer)
Alternative is [Vseeface](https://www.vseeface.icu/) 
Alternative workflow for creating is using vroid studio’s creation tools. Or [Live 3d](https://live3d.io/)

Essential Process:
* Import or create the 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into the camera tracker of choice

*Reference material*: 
The Art of Making 3D Vtuber Models (For Dummies)
<iframe width="560" height="315" src="https://www.youtube.com/embed/eFkwu7p1LhU?si=YBjIILi1xBsTRZVM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>


How to Make a 3D VTuber Avatar From Scratch, Part 1: Design 
<iframe width="560" height="315" src="https://www.youtube.com/embed/4fH0wNnM0Sw?si=07tMZTWgreZaLuvZ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



---
# Rokoko workflow

13 June 2025
Tried out Deepmotion and Rokoko Studio
<iframe width="560" height="315" src="https://www.youtube.com/embed/mifGsXHitvs?si=gYFO4nQ_Yt9vkLw2" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

Abandoned due to realtime playback not working the way we need it to. The tool is meant for motion capture and not live performance display.



---
# WebcamMotionCapture

Discovered a program called **Webcam Motion Capture** via this tweet
![image](https://hackmd.io/_uploads/Syx-eQJNEll.png)

<blockquote class="twitter-tweet" data-media-max-width="560"><p lang="en" dir="ltr">Take a look at this dreamy anime-inspired VTuber model created by <a href="https://twitter.com/_Mari_Art?ref_src=twsrc%5Etfw">@_Mari_Art</a> in collaboration with <a href="https://twitter.com/Kana_Fuyuko?ref_src=twsrc%5Etfw">@Kana_Fuyuko</a> and <a href="https://twitter.com/TakimiLive2D?ref_src=twsrc%5Etfw">@TakimiLive2D</a>.<br><br>See more: <a href="https://t.co/bLhXkcqw7q">https://t.co/bLhXkcqw7q</a> <a href="https://t.co/TSroh8vkBt">pic.twitter.com/TSroh8vkBt</a></p>&mdash; 80 LEVEL (@80Level) <a href="https://twitter.com/80Level/status/1933413316472250830?ref_src=twsrc%5Etfw">June 13, 2025</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>



---


Currently using Webcam Motion Capture
Found really good results

 https://webcammotioncapture.info/beta.php

![image](https://hackmd.io/_uploads/HyWvIk4Elx.png)

Essential Process:
* Import or create the 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into Webcam Motion Capture

---
# 2.5D workflow

19th June 2025
Currently creating a workflow where we convert 2d paintings and characters to 2.5d and use them in the 3d system - WebcamMotionCapture.

Essential Process:
* Edit the image in photoshop, remove background and get a splayed or t-pose.
* Import the image and create an extruded 3d model in Blender.
* Optimise the geometry to reduce memory footprint.
* Rig using a rig from a free model from [Vroid Studio](https://vroid.com/en/studio)
* Export a .VRM file.
* Import the .VRM file into Webcam Motion Capture