Unreal Engine 5.7 - Livestream to Custom Characters and Metahumans

Important! For any workflow-related issues, please join our Discord server.

Introduction

You have imported your Custom Character or Metahuman into your Unreal Engine 5.7 project. Next, you want to livestream your Rokoko Studio data to that Character.

Prerequisites

• Unreal Engine 5.7
• Rokoko Studio's latest version with a Plus or Pro subscription (required for live streaming).
• Rokoko Studio Live Plugin and Metahuman Plugin installed in Unreal Engine.
• A Smartsuit Pro II or Smartgloves, and a facial capture device (iPhone or Android with Metahuman Animator app). 

Rokoko Studio Setup

Before opening Unreal Engine, ensure your data stream is configured correctly in Rokoko Studio.

• Open Rokoko Studio and set up your Smartsuit Pro or Smartgloves (For Smartgloves you do not need to set them up individually if you are using them with a Smartglove-ready Smartsuit Pro)

  

• Navigate to the Streaming tab

• Enable the Unreal Engine 5.4+ livestreaming option

  

• Optional - Treadmill Mode. If you are doing a stationary performance (e.g., VTubing) and want to prevent positional drift, enable Treadmill Mode to lock the character to the origin while maintaining rotation and translation height.

  

TIP: To improve performance while streaming, minimize the Rokoko Studio window or reduce its size. This lowers the rendering load on your GPU.

Unreal Engine Project Setup

• Launch Unreal Engine 5.7 and open your project.

• Go to Edit > Plugins.

  

• Search for and enable the following plugins.

      *   Rokoko (Plugin)

      *   Metahuman (Enable all related Metahuman plugins to ensure dependency coverage).

      *   Restart the project to apply changes.

Workflow: Metahumans with Metahuman Animator

This workflow combines Rokoko body mocap with facial capture from the Metahuman Animator app (iOS or Android).

STEP 1: Prepare the Metahuman.

• Create or import your Metahuman (e.g., "Victor") into the scene.
• Fix LOD Issues.
      *   Open the Metahuman Blueprint (`Ctrl + E`), go to LODSync, and set Forced LOD to '0'. This prevents the mesh from switching quality levels when the camera moves. 

• Fix Groom/Clothing Glitches:
      *   Hair: If hair appears detached, check the Binding Asset in the Groom component and ensure the Target     *   Skeletal Mesh and Source Skeletal Mesh match.
      *   Clothing: If Fab assets (e.g., suits) appear invisible, move the asset files to a new folder, right-click the folder, and select Fix Up Redirectors.

STEP 2: Combine Streams (Body + Face).

In UE 5.7, Metahumans use a single "Live Link Subject" that must contain both body and face data. We will create a Virtual Subject to merge these streams.

• Export Body to Rokoko Studio:
      *   In the Metahuman Blueprint, select the Body mesh.
      *   Navigate to the Skeletal Mesh asset, right-click it, and select Asset Actions > Export.
      *   Save as an FBX (uncheck "Export Morph Targets").
      *   Import into Rokoko Studio: Drag this FBX into Rokoko Studio > Characters. Drag the imported mesh onto your actor to see live retargeting.
• Set up Live Link in Unreal:
      *   Go to Window > Virtual Production > Live Link.
      *   You should see your Rokoko Studio Source (Body) listed.
      *   Connect your facial capture app (Live Link Face) via Wi-Fi. It will appear as a separate source.
• Create Virtual Subject:
      *   In the Live Link panel, click + Add > Add Virtual Subject.
      *   Select Live Link Animation Virtual Subject.
      *   In the Virtual Subject settings, add:
          *   Body: Select your Rokoko Studio character stream.
          *   Face: Select your Android/iOS Live Link stream.
• Apply to Metahuman:
      *   Select your Metahuman in the level.
      *   In the Details panel, find Live Link.
      *   Set the Subject to your new Virtual Subject.
      *   Check Use Live Link.

Workflow: Custom Characters (Import Method)

This method is best for characters that can be cleanly imported into Rokoko Studio for superior retargeting control.

• Export Character: In Unreal, right-click your character's Skeletal Mesh > Asset Actions > Export.
• Import to Studio: Import the FBX into Rokoko Studio and drag it onto your live actor. Use Rokoko's native tools to adjust shoulder width, arm spacing, and posture.
• Create Animation Blueprint:
      *   In Unreal, right-click the Skeletal Mesh > Create > Anim Blueprint.
      *   In the AnimGraph, add a Live Link Pose node.
      *   Select your character's subject name (streamed from Studio).
      *   Connect it to the Output Pose.
• Stream: Drag the Anim Blueprint into the scene. The character will now move in real-time.

Workflow: Custom Characters (Retarget Component)

Use this method if your character cannot be imported into Rokoko Studio (e.g., complex meshes or atypical skeletons).

• Add Newton Actor: In the Content Browser, search for the Newton actor (included with the Rokoko plugin) and drag it into your scene. This actor natively receives Rokoko body data.
• Retarget Animation:
      *   Right-click the Newton Anim Blueprint > Retarget Animations.
      *   Select your custom character as the target.
      *   This automatically creates an IK Retargeter.
• Setup Retarget Component:
      *   Select your custom character in the scene.
      *   In the Details panel, click + Add and search for **Retarget Component.
      *   Source: Select the Newton actor from the dropdown.
      *   Retarget Asset: Select the IK Retargeter created in Step 2.
• The custom character will now mimic the Newton actor's movements in real-time without needing to be in Rokoko Studio.

Troubleshooting

• Drifting Character: Ensure Treadmill Mode is enabled in Rokoko Studio if you want the character to stay in place.
• Face Capture Orientation Issues: If the head rotates incorrectly when using a Virtual Subject, go to the Live Link Face app settings (on your phone) and disable Head Rotation and Head Position streaming. Let Rokoko Studio handle the neck/head rotation.
• Performance Lag: Close unnecessary background applications and minimize the Rokoko Studio window. Ensure both devices are on the same 5GHz Wi-Fi network.