Unreal Engine 5.6 - Livestream to Custom Characters and Metahumans

Introduction

You have imported your Custom Character or Metahuman into your Unreal Engine 5.6 project. Next, you want to livestream your Rokoko Studio data to that Character.

Useful

Most of the process here is the same as the 5.5 version equivalent. There are some extra steps needed for Metahumans at the end of the guide regarding the Face of the Metahuman we are using.

Make sure to follow the whole guide from scratch regardless.

Requirements

• Install Rokoko Studio here
• A Custom Character or Metahuman. You can find our Mime character here.
• A default Metahuman from Quixel Bridge, if you plan on using a custom Metahuman for your project.
• ARKit Blendshapes for Face Capture on your character. Metahumans have this by default.
• Unreal Engine 5.6
• Install the Rokoko Studio Plugin in Unreal Engine 5

Rokoko Studio and Unreal Engine Settings

• Enable Live streaming for Unreal Engine 5.4+ in Rokoko Studio
• In the Forward IP, use the IP of your computer that runs Unreal Engine. 127.0.0.1 is your localhost IP and is used when Rokoko Studio and Unreal Engine are on the same computer.
   

• In Unreal Engine, enable the LiveLink, LiveLinkHub, and Rokoko Plugin. Make sure to Restart Unreal Engine afterwards.
  
  Warning! Some of the used plugins are in beta version and may crash your project or cause issues.

• Go to Live Link and enable click Add Source and do nothing else. The Source should come up automatically.

• If you see your characters rotated after hitting Play, select Studio (first rectangle box in the image under Subject name on the left, and click "Use Rotation Order ZYX". And add 90 degrees in the "Hip Pre rotation".

• Make sure the Plugin Files are visible in your Project, by clicking the settings icon on the far right and selecting "Show Plugin Content". You need to show "Engine Content" to see Rokoko's Plugins folder.
  
  
• In the "All" folder, search for Newton, open the newtonalpha_AnimBP. This will be our base for livestreaming.
  
  
   
• In the newtonalpha_AnimBP, Enter the AnimGraph, right click in the viewport and use two Live Link Pose Nodes that connect straight to the Output Pose.
  
  In each one, select the Body and Face respectively from the drop down menu.
  
  
• Compile and Save the Blueprint. You may need to exit and re-enter the Blueprint for this to work.

Process for Custom Characters

• Import your Character in Unreal Engine. It can be either a Custom Character in FBX format or a Metahuman. Metahumans are imported in Unreal Engine via Quixel Bridge.
• If you want to use a custom metahuman, import a default metahuman as well via Quixel Bridge
   

• Make sure the T0 Ref pose is selected.

• In the main Viewport, drag and drop the AnimBP in the scene, click the AnimBP and search for "Update animation in Editor" and enable it. This is found under the 'Animation' tab.
  
  
• Drag and drop the Skeletal Mesh of your character in the scene.
• Go to the newtonalpha_AnimBP and right click on it. Select "Retarget Animations"
  
  
• Select your custom character's skeleton from the list and hit "Export Retarget Assets"
  
  
• In the Main Viewport, click on your character, then click the +ADD button on the right side in the Details Panel, and search for "Retarget Components". Click it.
  
  
• Click the "Retarget" component and select the SkeletalMeshComponent from newtonalpha_AnimBP from the dropdownlist for the Source Skeletal Mesh Component.
  
  Select the "RTG_Autogenerated" skeleton from the Retarget Asset dropdown list.
  
  

Process for Metahumans

• Import your Metahuman from Quixel Bridge. Import a Default Metahuman as well if you plan on using your own custom Metahuman.
• Open your custom Metahuman Blueprint. It is usually located in All->Content->Metahumans->MetahumanName named as BP_Metahuman
  
  
   
• Click on the "LODsync" and change the "ForcedLOD" to 0 in the "Details" Panel on the right.
  
  
  

• Click on the “Body” on the top-left and take note of the skeletal mesh blueprint used. In your case, it might be different.
  
  Save the name as we will need to use this later.

  

  
   

• Go to the newtonalpha_AnimBP and right click on it. Select "Retarget Animations"
  
  
• Select your custom metahuman's skeleton from the list and hit "Export Retarget Assets". It's the blueprint we checked above. Yours might be different from m_med_nrw_body.
  
  
• Next, you can drag and drop the Metahuman Blueprint in the scene.

• In the Main Viewport, click on your Metahuman Blueprint, then click the +ADD button on the right side in the Details Panel, and search for "Retarget Components". Click it.

  

• Click the "Retarget" component we just added and select the SkeletalMeshComponent from newtonalpha_AnimBP from the dropdownlist for the Source Skeletal Mesh Component.
• Select the "RTG_Autogenerated" skeleton from the Retarget Asset dropdown list.
• Select the "Body" from the Controlled Skeletal Mesh Component dropdown menu.
  
  
• Here is the extra step. Left click Face.
• For Anim Class select the Face_AnimBP_C. This is what we need from the default Metahuman we added at the start.

Now we should be able to hit Play and see our characters stream the data from Rokoko Studio!