In this guide I'll explain how you can train a LoRA of your VRM avatar that you can use to draw images of your 3D avatar in any Stable Diffusion Model that you can find. In this Tutorial we are going to train the LoRA for the tokenized identity Nature and I'm going to provide the file so you can play around with it in your own Stable Diffusion installation. Via the images below you can get a sense of the LoRA training results of my own VRM avatar. Find some vtubing content on my Youtube Channel to get an impression about the actual 3D avatar. A LoRA captures and conserves a concept or an idea in a way that it can be aggregated into larger models to be part of its outputs. A LoRA can be anything but in this case it is a character. Install the Stack You'll need the following 3 ingrediences to train a Stable Diffusion model on your VRM avatar. All links that you need are provided in the step-by-step guide below. The Stable Diffusion and Kohya resources are linked in the descriptions of the Youtube Videos and outlined in that content. VRM Live Viewer Stable Diffusion Kohya
4/18/2023Introduction I've been researching and tinkering a lot with locally hosted LLMs recently. There are several great tutorials out there which explain how to run LLaMa or Alpaca locally. After digged through a couple of them I decided to write a step-by-step on how to run Alpaca 13B 4-bit via KoboldAI and have chat conversation with different characters through TavernAI - entirely on your local machine. The performance of the quantized model loaded on the GPU is incredible and shows the potential for on-prem LLM systems. I'm aware of CPU based solutions like alpaca.cpp and played around with them. However in this guide I'll dig how to install the Alpaca that I personally like most on gaming hardware. It runs incredibly well on my RTX 4080 - see the video below. What makes this stack special? The ability to run this setup locally on gaming hardware is pretty neat. It amazed me for the same reasons Stable Diffusion amazes me. The modularity is another reason. You can configure the language model interface in KoboldAI and plug that API into other frontends: Instead of TavernAI you could embed it into Hyperfy, Webaverse or other web3xr platforms. We already saw ChatGPT integrations in Hyperfy and the Webaverse Character Studio already showed very powerful AI integrations. I hope this guide helps you to understand the modularity aspect. I'm currently exploring the Langchain framework which is going to allow the creation of more sophisticated LLM systems that are open and can be hosted on premise.
4/14/2023Inspired by M3: How to git gud I'm writing this document to reflect on my existing git gud journey and to learn more about HackMD as a platform for recording and sharing my experiences, knowledge aswell as future progression in an open and structured way. It's the beginning of my journal. 1. Mindset The past few years were quite exhausting but very fruitful at the same time. I've joined the web3 space in late 2017 when I found Decentraland and visited my first Ethereum conference (EthCC in Paris) in February 2018. One year later at ETHDenver 2019 I joined Patricio, helped him to build the first version of POAP (frontend code, design) which we deployed to Ethereum Mainnet. Over the past 5 years it was always about building while connecting with like-minded people in the space. It was about finding the right tribes while positioning the devshop Qwellcode that I co-founded 10 years ago inside the web3 space with long-term ambitions on our mind. We were among the first NFT builders on Polygon which allowed us to gain an understanding of how-to build accessible multi-chain EVM dapps early on. Our greatest achievement so far is the MOCA product ecosystem which I as CTO designed and our devs at Qwellcode have implemented. In parallel I soaked up any Metaverse related stuff that I could find while expanding hands-on research around various technologies with a focus on avatars and identity. I did read Ready Player One back in 2017 months before the movie came out and finally started to read Snowcrash during an Ayurvedic retreat in India earlier this year which was great timing.
12/24/2022