Livestream to Custom Metahumans in Unreal Engine 5

Who is this article relevant to?

This article is for anyone who wants to know how to livestream their Rokoko Studio animations to their Custom Metahumans in Unreal Engine 5

Which plans is this article relevant for?

Plus Pro Enterprise

Important! For any workflow-related issues, please join our Discord server.

Introduction

You have imported your Custom Metahuman into your Unreal Engine 5 project. Next, you want to live stream your Rokoko Studio data to that character.

A full guide on how to create your Custom Metahuman, import it into a project, and set it up for livestreaming can be found on our Youtube channel




Let's take a look at what we need and how we can set the Custom Metahuman up in steps.

A test project for Unreal Engine 5.2 with a ready custom metahuman can be found here.

A test project for Unreal Engine 5.3 with a ready custom metahuman can be found here.

 

You will still need to change the Actor Profile name and the port in the livelink if needed.

Requirements

  • Rokoko Studio which you can install here
  • A custom Metahuman
  • Install the Rokoko Studio Plugin in Unreal Engine. More info in Getting Started: Unreal Engine 4.26/4.27/5 Plugin from Epic Marketplace – Rokoko

    When a Marketplace version of the plugin doesn’t work or is out of date, please follow the steps to manually install it on your project by visiting the link above
  • Install the Metahuman plugin from the Epic Marketplace
  • BoneMaps and Mocap Folders which you can find here

Important detail!

  • If you have worked with 5.1 in the past, the only thing that changes in later UE versions is the rokoko receiver part. Now, you do not need to add a Rokoko Receiver actor in the viewport. We will enter the IP and port in the Live Link instead.
  • This article goes through the setup of a Male Metahuman. If you are using a Female Metahuman, please use the following files instead. They already exist in the project along with the male respective files.

    (f stands for female)
  • f_med_nrw_body,
  • f_med_nrw_animbp_C and
  • f_med_nrw_animbp assets

Process

Follow the steps carefully. Missing a step or a detail will result in the setup not working. Let's start!

  1. Download the zip files attached at the end of the article.
  2. Extract the zipped folder in C:\MyDocuments\Unreal Projects\ProjectName\Content.
    Your path may be named differently, but the final Content folder inside your Project will be the same.

    content.PNG

  3. Open your Unreal Engine 5 project.
  4. Open, compile, and save both the "Rokoko_MetaHuman_T_Pose" and the "Metahuman BoneMap" file in the BoneMaps Folder we added before applying any actions (important!)
  5. Open your Metahuman Blueprint. It is usually located in All->Content->Metahumans->MetahumanName named as BP_Metahuman

    BP.PNG

  6. Click on the "LODsync" and change the "ForcedLOD" to 0 in the "Details" Panel on the right.

    LOD.PNG
    LODsaync.PNG

  7. Click on the “Body” on the top-left and double-click the “Skeletal Mesh Asset” box in the "Details" Panel. This should open the Skeletal Mesh of the Metahuman. 

    body.PNG

    skeletal.PNG

  8. A window will open. Take note of its name. In this case, it's m_med_nrw_body. If you are using a female Metahuman, it will be different and it will have an"f" in its name. We will need to choose the correct one as shown in the next step. We will need to edit this file.

    skeletal

  9. Click the dots on the side of this "Blueprint" button and select the correct file. In this case, it's "m_med_nrw_body"

    m_mde.PNG

  10. A window directly to the "AnimGraph" will open. If it does not, double-click the "AnimGraph" on the lower left.

    an.PNG

    Right-click in the viewport and add the "Rokoko Body Pose" node.

    If you don't get Rokoko Body pose or any other "Rokoko" node/component, check that the Rokoko Plugin is active in the Edit > Plugins menu from the main menu.

    bodypose.PNG

  11. Connect it to the two existing nodes on each side and the "Local To Component" and "Component To Local" nodes will appear.

    connectg.PNG

     

  12. Drag a pin from “Rokoko Actor Name” or right-click on it and select “Promote to variable”.

    rokokovart.PNG

  13. Drag a pin from “Retarget Asset” or right-click on it and select “Promote to variable”, as well.

    var2.PNG

  14. Compile, and save
  15. Click on the “Rokoko Body Pose” node and in Default Value, select the “Metahuman_BoneMap” asset that was included in the BoneMaps folder, and hit Apply.

    bodynma.PNG

    ewew.PNG


    Note
    : “Rokoko Actor Name“ should be set as None.

  16. Compile, then save. If you get any errors after compiling, check point 4 above.
  17. Click on the “Control Rig” node and in Default Value, select the “Rokoko_MetaHuman_T_Pose” asset that was included in the BoneMaps folder.

    1.PNGsawq.PNG

  18. Compile, save, and exit the Animation Blueprint. If you get any errors after compiling, check point 4 above.

  19. Back to the Metahuman Blueprint, click on the “Body” on the top left and make sure that the Metahuman uses the "m_med_nrw_animbp" Animation Blueprint. This is because our Metahuman has the Male Body as explained earlier.


    body.PNG

    m)med.PNG



  20. Drag the body into the Event Graph:



  21. Drag a pin and select “Get Anim Instance



  22. Drag a pin and select “Cast to” and the Animation Blueprint you modified in previous steps.

    cast.PNG

  23. Drag a pin from the "Cast to" node and select "set Rokoko Actor Name".

    ee.PNG

  24. Set the “Rokoko Actor Name” as the Actor Name inside Studio and connect the wires to the nodes.

    sam.PNG



  25. Compile and save.

  26. Drag the “BP_Metahuman” into the scene.

    drag.PNG

  27. Omit this step if you are using a version later than 5.1

    Find the Rokoko Receiver Actor and drag it into the scene, by clicking on the top “Quickly add to the project”.



  28. In Studio, click on Livestream and on the cogwheel next to the Unreal icon.

    The port should be enabled and the same as the Rokoko Receiver port inside Unreal.

    aad.PNG

  29. Click on Activate next to the Unreal icon inside Studio.

    ad.PNG

  30. Back in Unreal, select Windows -> Virtual Production -> Live Link.



  31. Click on Source -> Rokoko Studio Source -> Studio

     
    You should see the following:

     

    If you are using UE 5.2 you will need to type the IP and port found in the Rokoko Studio plugin window. In our case, it's 127.0.0.1:14045, then click "Studio" as shown by the arrow.

    This basically replaces the need for the Rokoko Receiver in Step 27. If you have already added it, there is nothing wrong and you can continue.

    weq.PNG


  32. Click Play.

  33. For Face Capture, click on the Metahuman inside the editor

    mm.PNG

  34. Then, select the Face data in “ARKit Face Subj” in the Details panel.



  35. Tick the “Use ARKit Face”.



  36. For the body, the live link data should remain as None. Don't tick the box either.

    fafa.PNG

  37. Click Play.
    You should see the movement from inside Rokoko Studio forwarded to your Metahuman in Unreal Engine 5.

    asasa.PNG

    aaaa.PNG

     

 

Articles in this section

Knowledge base