Unreal Engine 5.5 - Livestream to Custom Characters and Metahumans

Important! For any workflow-related issues, please join our Discord server.

Important

Earlier Unreal Egnine workflows are available on newer versions of Unreal Engine as well.

Introduction

You have imported your Custom Character or Metahuman into your Unreal Engine 5.5 project. Next, you want to livestream your Rokoko Studio data to that Character.

A full guide on how to set up the livestream workflow for Unreal Engine 5.5. can be found on our Youtube channel

Requirements

• Install Rokoko Studio here
• A Custom Character or Metahuman. You can find our Mime character here and its Body and Face bonemap files attached at the end of this article.
• ARKit Blendshapes for Face Capture on your character.
• Unreal Engine 5.5
• Install the Rokoko Studio Plugin in Unreal Engine 5

Process

• Enable Live streaming for Unreal Engine 5.4+ in Rokoko Studio
  

• In Unreal Engine, enable the LiveLink, LiveLinkHub, and Rokoko Plugin. Make sure to Restart Unreal Engine afterwards.
  
  Warning! Some of the used plugins are in beta version and may crash your project or cause issues.

• Import your Character in Unreal Engine. It can be either a Metahuman or a Custom Character in FBX format. Normally, Metahumans are imported in Unreal Engine from Quixle Bridge.
  

• Make sure the T0 Ref pose is selected.

• Go to Live Link and enable click Add Source and do nothing else. The Source should come up automatically.

• Make sure the Plugin Files are visible in your Project, by clicking the settings icon on the far right and selecting "Show Plugin Content".
  
  
• In the "All" folder, search for Newton, open the newtonalpha_AnimBP. This will be our base for livestreaming.
  
  
  
  
• In the newtonalpha_AnimBP, right click and use two Live Link Pose Nodes that connect straight to the Output Pose.
  
  In each one, select the Body and Face respectively from the drop down menu.
  
  
• Compile and Save the Blueprint. You may need to exit and re-enter the Blueprint for this to work.
• In the main Viewport, drag and drop the AnimBP in the scene, click the AnimBP and search for "Update animation in Editor" and enable it.
  
  
• Drag and drop the Skeletal Mesh of your character in the scene.
• Go to the newtonalpha_AnimBP and right click on it. Select "Retarget Animations"
  
  
• Select your custom character's skeleton from the list and hit "Export Retarget Assets"
  
  
• In the Main Viewport, click on your character, then click the +ADD button on the right side in the Details Panel, and search for "Retarget Components". Click it.
  
  
• Click the "Retarget" component and select the SkeletalMeshComponent from newtonalpha_AnimBP from the dropdownlist for the Source Skeletal Mesh Component.
  
  Select the "RTG_Autogenerated" skeleton from the Retarget Asset dropdown list.
  
  

Process Continuation for Metahumans

• Import your Metahuman from Quixel Bridge.
• Open your Metahuman Blueprint. It is usually located in All->Content->Metahumans->MetahumanName named as BP_Metahuman
  
  
  
  

• Click on the "LODsync" and change the "ForcedLOD" to 0 in the "Details" Panel on the right.
  
  
  

• Click on the “Body” on the top-left and take note of the skeletal mesh blueprint used. In your case, it might be different.
  
  Save the name as we will need to use this later.

  

  
  
  
• Go to the newtonalpha_AnimBP and right click on it. Select "Retarget Animations"
  
  
• Select your custom metahuman's skeleton from the list and hit "Export Retarget Assets". It's the blueprint we checked above. Yours might be different such as m_med_nrw_body, etc.
  
  
• Next, you can drag and drop the Metahuman Blueprint in the scene.
  
  Click on it and select the Face input from ARKit Face Subj
  
  Then Check "use ARKit Face"
  
  
• For the Body, click the "Retarget" component and select the SkeletalMeshComponent from newtonalpha_AnimBP from the dropdownlist for the Source Skeletal Mesh Component.
  
  Select the "RTG_Autogenerated" skeleton from the Retarget Asset dropdown list.
  
  Select the "Body" from the Controlled Skeletal Mesh Component dropdown menu.