Rokoko Studio - Command API

In order to send commands to and receive messages from Studio, the following values will need to be defined:

• IP address (of the computer running Rokoko Studio that you want to interface with)
• Port (the port value defined in Rokoko Studio settings; defaults to 14053. This needs to be changed if this port is blocked or used by another application)
• API key (the API key defined in Rokoko Studio settings; defaults to 1234)

Communication with a Studio Command API server is managed by making HTTP POST requests and processing the responses.

Request

The command is then sent as an http request in the following format:

• http://{ip address}:{port}/{version}/{apiKey}/{command name}

Example of a simple http request, that you can do in any internet browser

• http://127.0.0.1:14053/v2/1234/info

The body of the request has to contain command arguments, in a JSON form

Example of a call to calibrate in python:

response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v1/{API_KEY}/calibrate",
json = {
'device_id': SMARTSUIT_NAME,
'countdown_delay': COUNTDOWN_DELAY,
'skip_suit' : False,
'skip_gloves' : False,
'use_custom_pose' : False,
'pose' : 'straight-arms-down' # tpose, straight-arms-down, straight-arms-forward
}
)

Response

The response has a json formatting:
string description;
CommandAPIResponceCodes response_code;
long startTime;
dynamic[] parameters;

where:
public enum CommandApiResponseCodes
{
OK = 0,
NO_CALIBRATEABLE_ACTORS = 1,
NO_TRACKABLE_ACTORS = 2,
CALIBRATION_ALREADY_ONGOING = 3,
RECORDING_ALREADY_STARTED = 4,
RECORDING_NOT_STARTED = 5,
UNEXPECTED_ERROR = 6,
MUST_BE_NUMBER = 7,
INVALID_REQUEST = 8,
COMMAND_INPUT_IS_EMPTY = 9,
COMMAND_INPUT_INVALID = 10,
INVALID_BONE_NAME = 11,
INVALID_TIMECODE_FORMAT = 12,
NO_ACTOR_FOUND_BY_DEVICE_ID = 13,
NO_LIVE_DEVICES = 14,
NO_PAIRED_LIVE_DEVICES = 15,
DEVICE_IS_NO_SMARTSUIT = 16,
WORLD_REF_NOT_SET = 17,
NOT_SUPPORTED_IN_SUBSCRIPTION_PLAN = 18
}

and parameters are arrays of different content. For instance, response from a tracker command contains two objects there - position array of values (X, Y, Z) and rotation array of values (quaternion X, Y, Z, W) of a requested tracker transform.

Start/Stop Recording

Used to start and stop a recording

• Command Tag: recording/start or recording/stop
• Parameters:
  • filename: The name of the clip you want to record (optional)
  • time: Timecode in SMPTE format "hh:mm:ss:ff". When it comes to start recording, the timecode will define a recorded clip start time. when it comes to stop recording and timecode provided is not with zeros, the timecode will be used to calculate clip duration and trim the data if required (optional)
  • frame_rate: Defined framerate for the clip you want to record (optional)
  • back_to_live: Set to true if you want to stay in "live view" after ending a recording (optional)
    
    
• Code for Start recording:
  response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v1/{API_KEY}/recording/start",
  json = {
  'filename': CLIP_NAME,
  'time' : TIME_CODE,
  'frame_rate' : FRAME_RATE,
  }
  )
• Code for Stop recording:
  response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v1/{API_KEY}/recording/stop",
  json = {
  'filename': CLIP_NAME,
  'time' : TIME_CODE,
  'frame_rate' : FRAME_RATE,
  'back_to_live' : BACK_TO_LIVE
  }
  )

Calibrate

Calibrates all paired live input devices in the scene or for the specified actor

• Command Tag: calibrate
• Parameters:
  • device_id: The name/id of the input device, e.g. a Smartsuit Pro, that you want to calibrate (optional)
    • If no value is given, all live devices in the scene will be calibrated
  • countdown_delay: The countdown in seconds before the actual calibration happens (optional)
  • skip_suit: If set to true, a paired Smartsuit Pro to the targeted actor will be skipped during calibration (optional)
  • skip_gloves: If set to true, paired Smartgloves to the targeted actor will be skipped during calibration (optional)
  • use_custom_pose: If set to true, a custom calibration pose can be used, as defined below (optional)
  • pose: (optional)
    • straight-arms-down
    • straight-arms-forward
    • tpose
      
      
• Code for Calibrate:
  http://{IP_ADDRESS}:{PORT}/v1/{API_KEY}/calibrate
  json = {
  'device_id': SMARTSUIT_NAME,
  'countdown_delay': COUNTDOWN_DELAY,
  'skip_suit' : False,
  'skip_gloves' : False,
  'use_custom_pose' : False,
  'pose' : 'straight-arms-down' # tpose, straight-arms-down, straight-arms-forward
  }

Reset Actor

Resets the actor pose and local position

• Command Tag: resetactor
• Parameters:
  • device_id: The name/id of the input device, e.g. a Smartsuit Pro, that is attached to the actor you want to reset
    
    
• Code for Reset Actor:
  response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v2/{API_KEY}/resetactor",
  json = {
  'device_id': SMARTSUIT_NAME
  }
  )

Scene Info

Returns information about the currently open scene. The response contains parameters array, fill with the information

• in case devices_info argument was true, then array fills in with the number of scene devices and the name of each device
• in case clips_info argument was true, the array fills in with the number of scene-recorded clips and the name of each clip

• Command Tag: info
• Parameters:
  • devices_info: Set to true if you want information about live input devices in the scene
  • clips_info: Set to true if you want information about the recorded clips in the scene
• Code for Scene Info:
  response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v2/{API_KEY}/info",
  json = {
  'devices_info': True,
  'clips_info': False
  }
  )

Attach Tracker

Attaches an external tracker, e.g. a HTC Vive Tracker, to an actor, which can then determine its global position.

The response contains parameters array with global position and quaternion orientation of a skeleton joint transform with a given “bone_attached” name

• Command Tag: tracker
• Parameters:
  • device_id: The name/id of the input device, e.g. a Smartsuit Pro, that is attached to the actor you want to attach the tracker to
  • bone_attached: The bone that the tracker should be attached to. Defaults to "Hips"
  • position: Position of the tracker being attached
    • {'X': <value>, 'Y': <value>, 'Z': <value>}
  • rotation: Rotation of the tracker being attached (optional)
    • {'X': <value>, 'Y': <value>, 'Z': <value>, 'W': <value>}
  • timeout: Time in seconds until the attached tracker releases the attached object (optional)
  • is_query_only: Use the command to only query a specified bone position and orientation rather than attaching the actual tracker and take control of the bone (optional)
• Code for Attach Tracker:
  response = requests.post(f"http://{IP_ADDRESS}:{PORT}/v2/{API_KEY}/tracker",
  json = {
  'device_id': SMARTSUIT_DEVICE_NAME,
  'bone_attached': '', # 'Hips'
  'position' : {'X': 1.0, 'Y': 1.0, 'Z': 0.0},
  'rotation' : {'X': 0.0, 'Y': 0.0, 'Z': 0.0, 'W': 1.0},
  'timeout' : 2.0,
  'is_query_only' : True
  }
  )

The Live Stream plugin provides methods to send Command API commands in your blueprint code and events to process the requested feedback.

The exposed blueprint methods for CommandAPI are categorized under "Command API" and require two common input variables, along with other options: the CommandAPI object and the CommandAPI IP info.

To avoid preparing these objects every evaluation tick, we can initialize them during script construction. To do so, we need to create two variables of type "Rokoko Studio Command API" and "Rokoko Command API IPInfo."

In a construction script, drag a drop every of those two variables into a the canvas in order to create set methods for them.

To assign a first variable, we have to create a new call “construct object from class” and class will be “Rokoko Command API” in that case.

For second variable IP Info, from a SET block you can drag a variable name into an empty canvas space which opens a context menu with different options and choose “Make RokokoCommandAPI_IPInfo”. That will create a new block with options of IP Address, port and API Key as input and Rokoko Command API IPInfo as output connected to the SET block.

The last action in construction script is to connect our blocks execution flow one by one, like it shown on the image below.

We are ready to go to the event graph and make our command API calls.

Create a block for our command API variable, to do that drag and drop the variable into an event graph canvas and choose “Get Command API” from a context menu

Then, drag and drop output of the command API into a canvas empty space. From an opened context menu, open a “Command API” category where all new related functions will be presented

The command API allows us to perform various calls such as starting or stopping recording, calibrating, restarting, and tracking.

After making a tracker node, create a node to set its parameters. Most of these parameters can be kept as default values, but it is necessary to pass the transform in global space to take control over the absolute position of the actor.

If you only want to request a bone transform, there is a checkbox labeled "Should Query Only". You need to create and bind an event for the Command API Tracker Request, which will receive a position and rotation in world space for a specified bone.

Here is an example how to create a request event

Command API Events

• request event - returns a general status of the command, description
• info request event - returns list of scene hardware names and scene clip names
• tracker request event - returns a position and rotation in global space

Github repository https://github.com/Rokoko/rokoko-studio-live-unity

Recent feature branch - feature/stream-character

Components:

Studio Manager - general streaming configuration (port, lz4 compression, prefabs, command api)

Studio Command API - contains functionality to connect to a command API server and send commands

How to use Tracker command.

You can find an example in Assets\Rokoko\Scenes\RokokoPluginExampleScene_DriftCorrection

1. In the scene, you can find a “Tracker” node with a transform component. That is used by a tracker command to pass drift correction global position

   

2. If you select CommandAPI node under the Rokoko parent, then in Inspector view you will see a Studio Command API component with a Tracker node assigned in the tracker parameters \ tracker transform property.
3. There are several ways to execute the tracker command, you can do that manually by calling the Tracker method in the Studio Command API component. Or you can assign a command API instance in the StudioManager and set checkbox “Auto Send Tracker Command”. In that case Tracker method will be executed on a fixed update of the StudioManager component.

https://github.com/Rokoko/studio-command-api-examples