Important! For any workflow-related issues, please join our Discord server.

Introduction

You have imported your Custom Character into your Unreal Engine 4 or 5 project. Next, you want to livestream your Rokoko Studio data to that Character.

Requirements

• Install Rokoko Studio here
• A customer character. You can find our Mime character and its Body and Face bonemap files attached below.
• Unreal Engine 4.26, 4.27, 5.X
• Install the Rokoko Studio Plugin in Unreal Engine 5. More info in Getting Started: Unreal Engine 4.26/4.27/5 Plugin from Epic Marketplace – Rokoko

Requirements if you are using Unreal Engine 5.1

• Rokoko Studio which you can install here
• A custom character
• Install the Rokoko Studio Plugin in Unreal Engine 5.1. More info in Getting Started: Unreal Engine 4.26/4.27/5 Plugin from Epic Marketplace – Rokoko

Requirements if you are using a later Unreal Engine version

If you have worked with 5.1 in the past, the only thing that changes is the plugin installation and the rokoko receiver part.

• Rokoko Studio which you can install here
• A custom character
• For the Rokoko plugin follow the instructions for a manual installation:
  1. Download the Rokoko Studio Live Plugin for Unreal Engine from GitHub here
  2. Download the necessary Win64 binaries here.
  3. The plugin file is basically a UE project. We will need the move the Rokoko plugin there to our Engine.
  4. Navigate into the extracted folder to the Plugins folder: rokoko-studio-live-unreal-engine-master\Plugins.
  5. Copy the "Smartsuit" folder from the Plugins Folder to C:\Program Files\Epic Games\UE_5.2\Engine\Plugins
  6. Create a folder called "Binaries" inside the path C:\Program Files\Epic Games\UE_5.2\Engine\Plugins\Smartsuit
  7. Extract and then place the "Win64" folder inside the Binaries folder: C:\Program Files\Epic Games\UE_5.2\Engine\Plugins\Smartsuit\Binaries
  
  
• The above applies to UE_5.3 engine as well.

Make sure your Custom Character is in a proper T-pose

1. Make sure that the character's hands and fingers are modeled as close to the Newton model as possible, to get the best possible retargeting of finger animation. The fingers should be straight and the thumb pointing 45 degrees away from the other fingers. 

   Proper finger placement
   

   
   
   

    

2. Studio Live supports up to a 4 finger joint setup: Metacarpal, Proximal, Medial, and Distal finger joints.

   Finger Joint setup
   

   

3. Example of how our Mime T-Pose looks can be found in the Dropdown below. You can download the character here

   Mime's T-Pose
   

   
   
   

       

Plugin

Launch a project. Go to Edit>Plugins and search for “Rokoko Studio Live'' and in that section make sure the “Enabled” checkbox is checked. Then, restart your project.

Setting up your Custom Character

• Import your character. A pop-up window will appear. Expand the “Mesh” tab and make sure “Use T0 as Ref Pose” and “Import Morph Targets” checkboxes are checked. 

• Right-click on your Skeletal Mesh and click Create>Anim Blueprint. Rename it to something like “character_AnimBP”
• Open the “character_AnimBP” and in the Anim Graph create a “Rokoko Body Pose” and a “Rokoko Face Pose” node. Connect them as shown below. (The “Component to Local” node will appear automatically)

Note: You can skip any step regarding the “Rokoko Face Pose” if you want to livestream just the body/hands.

• Create a variable, set it as “Name” and rename it something like “Rokoko Actor Name”.
• Drag the “Rokoko Actor Name” variable into the graph, select “get Rokoko Actor name” and connect it to “Rokoko Body Pose” and “Rokoko Face Pose” inputs. Then, compile and save.

  Note: Leave the “Rokoko Actor Name” as “None”! Do not rename it to your Actor's  Name as it 's on Studio!

• In the content browser, right-click, create a Blueprint class, expand the “All Classes” tab, and create a “RokokoBodyMapData” and “RokokoFaceMapData” blueprint. Name them something like “character_boneMap” and “character_faceMap”.

• Open the “character_boneMap” blueprint and write the names of the corresponding joints in the hierarchy. 

It should be different for each custom character! (Unless the bone names are the same)

“Mime” bone map setup:

• Compile and save the blueprint. (If you use the same joint naming on more characters you can reuse this asset).
• Open the “character_faceMap” blueprint and write the names of the blendshape names you'd like to override(first, try not making any changes). (If you use the same blendshape names on more characters you can reuse this asset). 

  When done, compile and save.

• Back in the “character_AnimBP” graph, create a variable, set it at “Rokoko Body Map Data” class reference and name it something like “BodyMapRetargetAsset”.

Compile and set its default value to the bone map you just created.

• Drag the “BodyMapRetargetAsset” variable to the graph, select “get BodyMapRetarget Asset” and connect it to the “Retarget Asset” input of “Rokoko Body Pose” node.

• Create one more variable, set it at “Rokoko Face Map Data” class reference, and name it something like “FaceMapRetargetAsset”.

Compile and set its default value to the face map you just created.

• Drag the “FaceMapRetargetAsset” variable to the graph, select “get FaceMapRetarget Asset” and connect it to the “Retarget Asset” input of “Rokoko Face Pose” node.

Finally, your animation graph will look like this:

Note: Ignore the warnings!!!!

• Right-click on the content browser and create an “Actor” blueprint class. Name it something like “character_BP”. Open it and on top left, click on Add Component and select skeletal mesh.

• To this skeletal mesh, apply your character’s skeleton and the Animation Blueprint you just created. 

Hit compile and save.

• On the event graph of the “character BP” drag the skeletal mesh to the graph. From its pin, drag and create a “Get Anim Instance” node. From its pin, drag and create a “Cast to character_AnimBP” node. From “Cast to character_AnimBP” blue pin drag and create a “Set Rokoko Actor Name”. Set the name as it is on your Actor Profile in Rokoko Studio. Connect the rest pins, Compile, and Save the Blueprint.

• Drag the Actor Blueprint “character_BP” into the scene.

Livestream settings

• In Unreal, under Place Actors(top-left of Unreal Editor) search for “Rokoko Receiver” and drag it into the scene. Set the Rokoko Port Number as 14045, in the Details Panel.

• Open Rokoko Studio, open a scene and make sure your Smartsuit is connected and paired with your Profile. Click on Start Livestream, navigate to Unreal Engine and click on the cogwheel. Make sure the port number is 14045 and enable the Livestream.

• Back in Unreal, go to Window>Live Link and then Source>Rokoko Studio Source>Studio.

If you are using UE 5.2 you will need to type the IP and port found in the Rokoko Studio plugin window. In our case, it's 127.0.0.1:14045, then click "Studio" as shown by the arrow.

This basically replaces the need for the Rokoko Receiver in Step 27. If you have already added it, there is nothing wrong and you can continue.

Finally, hit play and start Livestreaming with your Custom Character!