Create a 3D Vtuber Setup in Unreal Engine

What is a 3D VTuber?

You must know what a VTuber is if you have spent any time on the GrifNMore blog. Go ahead and check out this article if you don’t know much about the term VTuber and would like to know more. Virtual YouTubers or VTubers come in a variety of designs and dimensions and are created using a wide array of programs. When it comes to 3D Vtubers you have even more control over your world and interaction- it can even be like a video game!

For a couple of years, Grif used Adobe Character Animator as his software of choice. Now Grif is transitioning to a new dimension through the power of Unreal Engine, the Rokoko Smart Suit, and Live Link. I want to be clear that I won’t be giving you all the juicy details yet; just the basics on how you can create your own 3D VTuber in Unreal Engine. It is an extremely time-consuming and technologically-challenging process that will require several articles to cover! I will however provide an overview of what goes into creating a setup and will point at some resources to get you started.

Support us on Patreon for exclusive content

This is probably a good time to mention that Grif has a Patreon where you can get access to all the behind-the-scenes information as well as exclusive tutorials that cover Adobe Character Animator and this hot new Unreal tech. Grif is also publishing YouTube videos covering VTuber creation using Adobe Character Animator, and Unreal Engine. Here is a video showing the first run of the Rokoko Smart Suit Pro.

That this isn’t the only way to create a 3D VTuber or the easiest, but I do think it is the most robust. Other options include the popular FaceRig and setups that rely on VR Chat and a VR Headset. Digital Puppets recently produced this video where they use Live Link and IClone. These are all great solutions with their own merits, but Grif chose the Unreal Engine route to leverage the power of having a game engine behind his setup.

So how does this all work?

First off, you don’t need an expensive motion-capture suit to get started, but it does add a whole world of possibilities you can’t achieve without it. At the base level, you only need an iPhone with a TrueDepth Camera. You can use the Epic Games launcher for a free Unreal Engine install, and a 3D model setup with the AppleAR face shapes. With all of those requirements in place, you will only need to do a bit of setup to get your character running and tracking your face in Unreal engine.

So what about controlling the body? Unreal is a game engine friends, and the means you can create any animations you want! Maybe you want your character to sit in a chair with a controller in-hand, or perhaps you want to be able to walk across the screen? You can do all of that and more! Now’s the time to start brainstorming ways to interact with your chat and build your world. Stay tuned to this blog to get all the details.

Check out this first run of Grif’s 3D VTuber setup

Still not sure about starting your career as a 3D Vtuber? You can dip a toe in with a sound reactive talksprite option, commonly called a PNGtuber to use with OBS and hotkeys. Grif has plenty More to help you start, from the shop you can browse lots of premade puppets and accessories and read more about drawing your 2D Vtuber in layers. Learn even More when you join the More Club!