Unreal Engine 5 - Livestream to Custom Character and Metahumans

Let's take a look at what we need and how we can set the Custom Metahuman up in steps.

A test project for Unreal Engine 5.2 with a ready custom metahuman can be found here.

A test project for Unreal Engine 5.3 with a ready custom metahuman can be found here.

You will still need to change the Actor Profile name and the port in the livelink if needed.

Requirements

• Rokoko Studio which you can install here
• A custom Metahuman
• Install the Rokoko Studio Plugin in Unreal Engine.
  
• Install the Metahuman plugin from the Epic Marketplace
• BoneMaps and Mocap Folders which you can find here

Important detail!

• This article goes through the setup of a Male Metahuman. If you are using a Female Metahuman, please use the following files instead. They already exist in the project along with the male respective files (f stands for Female).
  • f_med_nrw_body,
  • f_med_nrw_animbp_C
  • f_med_nrw_animbp

Process

Follow the steps carefully. Missing a step or a detail will result in the setup not working. Let's start!

1. Download the zip files attached at the end of the article.
2. Extract the zipped folder in C:\MyDocuments\Unreal Projects\ProjectName\Content.
   Your path may be named differently, but the final Content folder inside your Project will be the same.
   
   
   
   
3. Open your Unreal Engine 5 project.
4. Open, compile, and save both the "Rokoko_MetaHuman_T_Pose" and the "Metahuman BoneMap" file in the BoneMaps Folder we added before applying any actions (important!)
5. Open your Metahuman Blueprint. It is usually located in All->Content->Metahumans->MetahumanName named as BP_Metahuman
   
   
   
   

6. Click on the "LODsync" and change the "ForcedLOD" to 0 in the "Details" Panel on the right.
   
   
   

7. Click on the “Body” on the top-left and double-click the “Skeletal Mesh Asset” box in the "Details" Panel. This should open the Skeletal Mesh of the Metahuman. 

   

   
   
   

8. A window will open. Take note of its name. In this case, it's m_med_nrw_body. If you are using a female Metahuman, it will be different and it will have an"f" in its name. We will need to choose the correct one as shown in the next step. We will need to edit this file.
   
   
   
   

9. Click the dots on the side of this "Blueprint" button and select the correct file. In this case, it's "m_med_nrw_body"

   
   
   

10. A window directly to the "AnimGraph" will open. If it does not, double-click the "AnimGraph" on the lower left.
    
    
    
    Right-click in the viewport and add the "Rokoko Body Pose" node.
    
    If you don't get Rokoko Body pose or any other "Rokoko" node/component, check that the Rokoko Plugin is active in the Edit > Plugins menu from the main menu.
    
    
    
    

11. Connect it to the two existing nodes on each side and the "Local To Component" and "Component To Local" nodes will appear.
    
    
    

     

12. Drag a pin from “Rokoko Actor Name” or right-click on it and select “Promote to variable”.

    
    
    

13. Drag a pin from “Retarget Asset” or right-click on it and select “Promote to variable”, as well.

    
    
    

14. Compile, and save

15. Click on the “Rokoko Body Pose” node and in Default Value, select the “Metahuman_BoneMap” asset that was included in the BoneMaps folder, and hit Apply.

    
    
    
    

    
    Note: “Rokoko Actor Name“ should be set as None.
    
    

16. Compile, then save. If you get any errors after compiling, check point 4 above.
17. Click on the “Control Rig” node and in Default Value, select the “Rokoko_MetaHuman_T_Pose” asset that was included in the BoneMaps folder.
    
    
    
    

18. Compile, save, and exit the Animation Blueprint. If you get any errors after compiling, check point 4 above.

19. Back to the Metahuman Blueprint, click on the “Body” on the top left and make sure that the Metahuman uses the "m_med_nrw_animbp" Animation Blueprint. This is because our Metahuman has the Male Body as explained earlier.

    
    

    

    

    
    
    

20. Drag the body into the Event Graph:

    
    
    

21. Drag a pin and select “Get Anim Instance”

    
    
    

22. Drag a pin and select “Cast to” and the Animation Blueprint you modified in previous steps.

    
    
    

23. Drag a pin from the "Cast to" node and select "set Rokoko Actor Name".
    
    
    
    

24. Set the “Rokoko Actor Name” as the Actor Name inside Studio and connect the wires to the nodes.

    

    
    
    

25. Compile and save.

26. Drag the “BP_Metahuman” into the scene.

    
    
    

27. Omit this step if you are using a version later than 5.1
    
    Find the Rokoko Receiver Actor and drag it into the scene, by clicking on the top “Quickly add to the project”.

    
    
    

28. In Studio, click on Livestream and on the cogwheel next to the Unreal icon.

    The port should be enabled and the same as the Rokoko Receiver port inside Unreal.
    
    

    

29. Click on Activate next to the Unreal icon inside Studio.

    
    
    

30. Back in Unreal, select Windows -> Virtual Production -> Live Link.

    
    
    

31. Click on Source -> Rokoko Studio Source -> Studio
    

    

     
    You should see the following:

     

    If you are using UE 5.2 you will need to type the IP and port found in the Rokoko Studio plugin window. In our case, it's 127.0.0.1:14045, then click "Studio" as shown by the arrow.

    This basically replaces the need for the Rokoko Receiver in Step 27. If you have already added it, there is nothing wrong and you can continue.
    
    
    
    
    

32. Click Play.
    
    

33. For Face Capture, click on the Metahuman inside the editor

    
    
    

34. Then, select the Face data in “ARKit Face Subj” in the Details panel.

    
    
    

35. Tick the “Use ARKit Face”.

    
    
    

36. For the body, the live link data should remain as None. Don't tick the box either.
    

    
    
    

37. Click Play.
    You should see the movement from inside Rokoko Studio forwarded to your Metahuman in Unreal Engine 5.