RL Policy & Tron1

Categories:

• Examples
• Demos

Demo available here. Note: registration required.

Overview

The Tron1 robot is a multi-modal biped robot that can be used for Humanoid RL Research. Much of the software is open source, and can be found here.

Here we run a movement test on the robot using two different RL policies. One using isaaclab, and another using isaacgym.

The test itself is relatively straightforward. We ask the robot to move forwards 5 meters and rotate 150 degrees relative to its starting position.

Parameterizing the Test

Using the artefacts.yaml file, we setup our test as so:

 policy_test:
    type: test
    runtime:
      framework: ros2:jazzy
      simulator: gazebo:harmonic
    scenarios:
      defaults:
        pytest_file: test/art_test_move.py
        output_dirs: ["test_report/latest/", "output"]
      settings:
        - name: move_face_with_different_policies
          params:
            rl_type: ["isaacgym", "isaaclab"]
            move_face:
              - {name: "dyaw_150+fw_5m", forward_m: 5.0, dyaw_deg: 150, hold: false, timeout_s: 35}

Key points from above:

• The test is conducted using pytest (and so pytest_file points to our test file).
• Artefacts will look in the two folders described in output_dirs for uploads to the dashboard
• We have two sets of parameters:
  1. rl_type: two parameters: “isaacgym” and “isaaclab”
  2. move_face: one parameter set specifying forward distance (5m), rotation angle (150 degrees), and timeout (35 seconds)

The test will run twice, once using the isaacgym policy, and again using the isaaclab policy. Both tests will use the move_face parameter to determine how far, and how much rotation the robot should do.

Quick analysis

isaacgym

When using the isaacgym policy, we can see (from a birdseye recording) the robot successfuly rotating, and then moving forwards:

The dashboard notes the test as a success:

And a csv we created during the test plotting the ground truth movement is automatically converted to an easy to read chart by the dashboard.

isaaclab

With the isaaclab policy, we see there is still work to be done. The dashboard notes the test as a fail (and shows us the failing assertion), and the birdseye video shows the robot failing to setout its goal.

We have a csv (automatically converted to a chart) of estimated trajectory, (i.e what the robot thinks it has done), which we can see is widely different to the ground truth:

Data available after the tests

• ROSbag
• Video of the active robot in Gazebo, both birdseye and first person,
• stdout and stderr
• debug log
• csv of the trajectory (estimated) automatically displayed as a graph in the dashboard
• csv of the trajectory (ground truth) automatically displayed as a graph in the dashboard

Artefacts Toolkit Helpers

For this project, we used the following helpers from the Artefacts Toolkit:

• get_artefacts_params: to determine which policy to use, and the trajectory (movement, rotation)
• extract_video: created the videos from the recorded rosbag