This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

The Artefacts Toolkit (beta)

Using the Artefacts Toolkit

The Artefacts Toolkit is a Python package designed to improve developer productivity. It provides a collection of helper functions that simplify common testing tasks and integrate with the Artefacts platform.

The toolkit is organized into the following modules:

  • Configuration Helpers - Simplify parametric testing and bring in configurations from the artefacts.yaml file
  • Chart Helpers - Generate visualisations from test data
  • Gazebo Helpers - Interact with Gazebo simulations during tests
  • Rosbag Helpers - Record, extract, and analyze data from ROS bag files
  • Rerun Helpers - Record Rerun .rrd files during simulation and make test assertions against them

Installation

pip install artefacts-toolkit

Example Projects using the Artefacts Toolkit:

  • Nav2 - An example project using the Nav2 navigation stack and Gazebo.

1 - Configuration Helpers in the Artefacts Toolkit

The Artefacts Toolkit Configuration Helpers are functions that allow you to interact with configurations you have set in your artefacts.yaml file.

Import with:

from artefacts_toolkit.config import get_artefacts_param

Functions

Function Reference

get_artefacts_param

Returns a parameter set in the artefacts.yaml file. If param_type is set to launch, it will be returned as type string so that it can be used as a ROS launch argument.

This function is particularly useful for parametric testing. When you define a list of parameter values in your artefacts.yaml file (e.g., launch/world: ["empty.sdf", "bookstore.sdf", "restaurant.sdf"]), the artefacts run command will execute your test multiple times - once for each value in the list. During each test execution, get_artefacts_param("launch", "world") will automatically return the current parameter value for that specific test run.

This becomes especially powerful for grid-based testing! With three parameters that each have three possible values, Artefacts will automatically execute your test 27 times (3 × 3 × 3) while your launch file code remains unchanged.

get_artefacts_param(
    param_type,
    param_name,
    default=None,
    is_ros=True
)

Parameters

Parameter Type Description Default
param_type str The namespace/category of the parameter (e.g., “launch” in “launch/world”) Required
param_name str The specific parameter name (e.g., “world” in “launch/world”) Required
default any Value to return if the parameter isn’t found in artefacts.yaml. None
is_ros bool Whether the parameter should be converted to a ROS parameter format True

Returns

The function returns the parameter value with the following behavior:

  • If param_type is "launch" and is_ros is True: Returns the value as a str, regardless of its original type in the artefacts.yaml file. This is so that it can be used as a ros launch argument.
  • if default is set to anything other than None, then the value will be returned if artefacts is unable to find the requested parameter. This can be useful when (for example) using launch_test instead of artefacts run but you do not wish to make any code changes. It will also prevent KeyError exceptions.
  • For all other cases: Returns the value with its original type from the YAML file (e.g., list, dict, int, float, str, etc.)

Example

Given an artefacts.yaml file with the following configuration set:

scenarios:
  defaults:
    output_dirs: ["output"]
    metrics:
        - /odometry_error
    params:
      launch/world: ["bookstore.sdf", "empty.sdf"]

Getting the parameter in our test launch file like so:

def generate_test_description():
    try:
        world = get_artefacts_param("launch", "world")
    except FileNotFoundError:
        world = "empty.world"

    run_headless = LaunchConfiguration("run_headless")
    launch_navigation_stack = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(
            [
                os.path.join(
                    get_package_share_directory("sam_bot_nav2_gz"),
                    "launch",
                    "complete_navigation.launch.py"
                ),
            ]
        ),
        launch_arguments=[("run_headless", run_headless), ("world_file", world)],
    )

...# Rest of launch test file   

This will run the same test twice - once in an empty world and once in a bookstore world - without any changes to your launch file code between runs.


get_artefacts_params

Loads all parameters set in artefacts.yaml for a given run, providing them as dict. Can be used with regular dict methods such as .get()

get_artefacts_params()

Parameters

None

Returns

The function returns all parameters for a given run as type dict.

Example

Given an artefacts.yaml file with the following configuration set:

scenarios:
    defaults:
        params:
            headless: "True"

We can set a flag on our launch_description for a headless simulation as follows:

@launch_pytest.fixture(scope="module")
def launch_description(rosbag_recording):
    pkg_path = Path(get_package_share_directory("my_package"))
    try:
        headless = get_artefacts_params().get("headless", "False")
    except RuntimeError:
        # If artefacts params are not available, default to False
        headless = "False"

    start_launch = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(
            str(pkg_path / "launch" / "mytestfile.launch.py")
        ),
        launch_arguments={
            "headless": headless,
        }.items(),
    )

merge_ros_params_files

Merges two ROS2 yaml parameter files into one, overriding the values in the first one (source) with the values in override

merge_ros_params_files(
    source,
    override,
    destination,
    rosify=False
)

Parameters

Parameter Type Description Default
source str Path to original parameter file to be overridden Required
override str Path to parameter file containing parameters to override Required
destination str Path to where new parameter file should be saved Required
rosify bool Converts params to ros2 param file nested format if True False

Returns

None: The newly made merged parameter file will be available at the path specified in destination

Example

Given the following source and override yaml:

# source
outer:
    inner1: 1
    inner2: 
        leaf: keep
flat: src

# override
outer:
    inner2:
        leaf: replaced
        new_leaf: 321
    inner3: added
flat: override

We will get the following new yaml file saved to destination as follows:

# rosify=False (Default)
outer:
    inner1: 1
    inner2:
        leaf: replaced
        new_leaf: 321
    inner3: added
flat: override

With rosify=True, parameters will be nested under ros__parameters for the appropriate node:

#source
controller_server:
    ros__parameters:
        enable_stamped_cmd_vel: true
        controller_frequency: 20.0
        min_x_velocity_threshold: 0.001

#override
controller_server/controller_frequency: 30
controller_server/min_x_velocity_threshold: 0.005
controller_server/min_y_velocity_threshold: 0.5   
    
#destination
controller_server:
    ros__parameters:
        enable_stamped_cmd_vel: true
        controller_frequency: 30
        min_x_velocity_threshold: 0.005
        min_y_velocity_threshold: 0.5

2 - Chart Helpers in the Artefacts Toolkit

The Artefacts Toolkit Chart helpers are designed to help you with visualising data received from topics while running tests.

Import with:

from artefacts_toolkit.chart import make_chart

Functions

Function Reference

make_chart

Creates an interactive HTML chart based on data from two provided topics.

make_chart(
    filepath,
    topic_x,
    topic_y,
    axis_x_name="",
    axis_y_name="",
    field_unit=None,
    output_dir="output",
    chart_name="chart",
    file_type="rosbag"
    output_format="html"
)

Parameters

Parameter Type Description Default
filepath str Path to the data file (rosbag) Required
topic_x Union[str, list] Topic name for x-axis. Use “Time” to plot against time Required
topic_y Union[str, list] Topic name for y-axis. Use “Time” to plot against time Required
axis_x_name str Label for x-axis. Auto-generated for single topics; recommended when using a list of topics ""
axis_y_name str Label for y-axis. Auto-generated for single topics; recommended when using a list of topics ""
field_unit str Unit of measurement for the field data (e.g., “m/s”, “rad”) None
output_dir str Directory where the chart will be saved "output"
chart_name str Name of the generated chart file "chart"
file_type str Type of data file. Currently supports “rosbag” "rosbag"
output_format str Output file type. Choose from “html” or “csv” "html"

Returns

None:

When output_format="html"
  • Creates a plotly html chart <chart_name>.html at output_dir but doesn’t return any value.
  • Can create single, or multiple plot charts (See Example)
When output_format="csv"
  • Creates a csv file <chart_name>.csv at output_dir but doesn’t return any value. The Dashboard will automatically try to convert csv files to charts upon upload. Useful when wanting to keep file sizes down.
  • Please note that the dashboard can only currently automatically create single topic charts.

Example

Single Topics

The following example adds the make_chart function post shutdown, i.e after the test has completed and rosbag saved.

# my_test_file.launch.py
# test code

...

@launch_testing.post_shutdown_test()
class TestProcOutputAfterShutdown(unittest.TestCase):
    def test_exit_code(self, rosbag_filepath):
        make_chart(
            rosbag_filepath,
            "/odom.pose.pose.position.x",
            "/odom.pose.pose.position.y",
            field_unit="m",
            chart_name="odometry_position",
            output_format="csv"
        )

Producing a csv file that will automatically display as a chart in the dashboard as below:

Example Chart

Topic vs Topics
make_chart(
    rosbag_filepath,
    "time",
    ["/critics_cost/ConstraintCritic.data", "/critics_cost/GoalAngleCritic.data", "/critics_cost/GoalCritic.data", "/critics_cost/CostCritic.data", "/critics_cost/PathAlignCritic.data", "/critics_cost/PathAngleCritic.data", "/critics_cost/PathFollowCritic.data", "/critics_cost/PreferForwardCritic.data"],
    axis_x_name="time",
    axis_y_name="critics",
    chart_name="Critics vs Time",
)

Will produce the following chart:

Example Chart, topic vs topics

Topics vs Topics
 make_chart(
        rosbag_filepath,
        topic_x=["/odom.pose.pose.position.x", "/gt_odom.pose.pose.position.x"],
        topic_y=["/odom.pose.pose.position.y", "/gt_odom.pose.pose.position.y"],
        axis_x_name="x (m)",
        axis_y_name="y (m)",
        chart_name="odom vs gt positions"
    )

Will produce the following chart:

Example Chart, topics vs topics

3 - Gazebo Helpers in the Artefacts Toolkit

The Artefacts Toolkit Gazebo Helpers provide convenient functions to interact with your Gazebo simulations while running tests. These utilities allow you to inspect simulation objects, access model positions, as well as bridge topics between Gazebo and ROS2.

Import with:

from artefacts_toolkit.gazebo import bridge, gz

Functions

Function Reference

bridge.get_camera_bridge

Creates a gazebo / ros2 topic bridge for a camera topic

bridge.get_camera_bridge(
    topic_name,
    condition=None
)

Parameters

Parameter Type Description Default
topic_name str The camera topic name to bridge from Gazebo to ROS2 Required
condition str Optional launch condition that determins when this bridge should be created None

Returns

Node: Returns a ROS 2 Node object that runs the parameter_bridge for the specified camera topic. This node can be included in your launch description.

Example

The following example shows a camera bridge being created conditionally based on a launch argument. When record_video is set to “true”, the bridge will be activated, allowing ROS 2 nodes to subscribe to camera images from Gazebo. The returned Node is added to the LaunchDescription to be included in the launch process.

@pytest.mark.launch_test
def generate_test_description():
    from artefacts_toolkit.gazebo import bridge
    ...

    camera_topic = "/observation_camera/image"
    bag_recorder, rosbag_filepath = get_bag_recorder([camera_topic])
    sim = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(["bringup", "/robot.launch.py"])
    )

    record_video_launch_arg = DeclareLaunchArgument(
        "record_video", default_value="true"
    )
    record_video = LaunchConfiguration("record_video")
    camera_bridge = bridge.get_camera_bridge(camera_topic, condition=IfCondition(record_video))
    
    return LaunchDescription(
        [
            record_video_launch_arg,
            sim,
            camera_bridge,
            controller_process,
            launch_testing.actions.ReadyToTest(),
        ]
    )

gz.add_entity

Inserts an entity XML element into a Gazebo world XML structure. This is an in-place operation that modifies the provided world XML directly.

gz.add_entity(world_xml, entity_xml)

Parameters

Parameter Type Description Default
world_xml xml.etree.ElementTree.Element The world XML structure in <sdf>...</sdf> format Required
entity_xml xml.etree.ElementTree.Element The entity XML element to insert into the world Required

Returns

None: This function modifies world_xml in-place and doesn’t return any value.

Example

The following example shows how to add an actor to an existing world file. After loading the world file with gz.load_world, an actor is created and then inserted into the world XML structure.

from artefacts_toolkit.gazebo import gz

# Load existing world file
world_xml = gz.load_world("worlds/env.sdf")

# Create an actor with waypoints
waypoints = [
    [4, -3.2, 1.0],
    [0.8, -3.2, 1.0],
    [0.8, -6, 1.0],
    [4, -6, 1.0],
]
actor_xml = gz.make_actor("pedestrian_1", waypoints)

# Add the actor to the world
gz.add_entity(world_xml, actor_xml)

gz.load_world

Loads and parses a Gazebo world file, returning the root XML element.

gz.load_world(world_file)

Parameters

Parameter Type Description Default
world_file str Path to the Gazebo world file to load Required

Returns

xml.etree.ElementTree.Element: Returns the root XML element of the parsed world file in <sdf>...</sdf> format.

Raises

FileNotFoundError: If the specified world file does not exist.

Example

from artefacts_toolkit.gazebo import gz

# Load a world file
world_xml = gz.load_world("worlds/env.sdf")

# Now you can modify the world and add entities
actor_xml = gz.make_actor("pedestrian", [[0, 0, 0], [1, 1, 0]])
gz.add_entity(world_xml, actor_xml)

gz.get_sim_objects

Extracts model information from a Gazebo world file by parsing its XML structure. This function returns the name and original pose of all models defined in the world file.

This can useful when you need to know the initial poses of models as defined in the world file, rather than querying the running simulation (where positions might have changed due to physics, randomness, or interactions).

gz.get_sim_objects(world_file)

Parameters

Parameter Type Description Default
world_file str Path to the Gazebo world file to parse Required

Returns

tuple: The function returns a tuple with two values that should be unpacked:

  • A list of dictionaries:
objects = [
    {
        "name": "model_1",
        "pose": "0 0 0 0 0 0"
    }, 
    {
        "name": "model_2",
        "pose": "1 2 3 0 0 0"
    },
    ...
]
  • A dictionary mapping model names to poses:
objects_positions = {
    "model_1": "0 0 0 0 0 0", 
    "model_2": "1 2 3 0 0 0",
    ...
}

Example

# Working with the objects_position dict
_, objects_positions = gz.get_sim_objects("world.sdf")
model_pose = objects_positions["model_1"]  # "0 0 0 0 0 0"

# Working with the objects list
objects, _ = gz.get_sim_objects("world.sdf")
for model in objects:
    print(f"{model['name']} is at position {model['pose']}")

gz.get_model_location

Gets the current (x, y, z) position of a model in a running Gazebo simulation. Unlike get_sim_objects which reads original positions from a world file, this function queries the live simulation to get the model’s current position.

gz.get_model_location(model_name)

Parameters

Parameter Type Description Default
model_name str Name of the model to query in the simulation Required

Returns

tuple: Returns a tuple of three float values representing the (x, y, z) position in meters:

(x_position, y_position, z_position)

Example

The example below demonstrates how to combine get_sim_objects and get_model_location to validate a pick-and-place task. It shows how to:

  1. Get initial positions from the world file
  2. Wait for a robot to complete its task
  3. Compare initial positions with current positions to detect which objects were moved
class TestProcOutput(unittest.TestCase):
    def test_moved_meatballs(self, proc_output, controller_process):
        # Original locations
        sim_objects, sim_objects_positions = gz.get_sim_objects("worlds/env.sdf") # this includes models poses
        meatball_models = [obj["name"] for obj in sim_objects if "karaage" in obj["name"]]
        #  Wait for the control loop to finish
        proc_output.assertWaitFor("Done with tasks execution", timeout=300)
        picked_meatballs = 0 # karaage moved for more than 10cm
        for meatball in meatball_models:
            x, y, z = gz.get_model_location(meatball)
            x_original, y_original, z_original = sim_objects_positions[meatball]
            dist = ((x - x_original) ** 2 + (y - y_original) ** 2 + (z - z_original) ** 2) ** 0.5
            if dist > 0.1: #10cm
                # Likely to have been picked and moved somewhere else
                picked_meatballs += 1
        self.assertEqual(picked_meatballs, 4)

gz.kill_gazebo

Kills the currently running gazebo process.

gz.kill_gazebo()

Returns

None: This function doesn’t return any value.

Example

from artefacts_toolkit.gazebo import gz
...

@launch_testing.post_shutdown_test()
class TestProcOutputAfterShutdown(unittest.TestCase):
    def test_exit_code(self, proc_info, controller_process, rosbag_filepath):
        gz.kill_gazebo()
        ...

gz.make_actor

Creates the necessary XML to add in an actor to your simulation. A path for the actor to move will be created based on provided waypoints, walking speed, and speed of rotation.

gz.make_actor(
    actor_name,
    waypoints,
    walk_speed=0.8,
    rotate_speed=1.8,
    enable_loop=True,
    output_type="xml"
)

Parameters

Parameter Type Description Default
actor_name str The name to call your actor. Noted in xml as <actor name="{actor_name}"> Required
waypoints list List of waypoint coordinates. Can be either [[x,y,z], ...] for position only, or [[x,y,z,roll,pitch,yaw], ...] for position and rotation Required
walk_speed float Walking speed of the actor in meters per second 0.8
rotate_speed float Rotation speed of the actor in radians per second 1.8
enable_loop bool Whether the actor should continuously repeat the waypoint path True
output_type str Whether to return as xml object or str string "xml"

Returns

output_type can be set to "xml" or "str":

  • xml: Returns an XML object representation of a Gazebo actor (xml.etree.ElementTree.Element)
  • str : Returns an XML string representation of a Gazebo actor
<actor name="{actor_name}">
    <skin>
        <filename>https://fuel.gazebosim.org/1.0/Mingfei/models/actor/tip/files/meshes/walk.dae</filename>
    </skin>
    
    <animation name="walking">
        <filename>https://fuel.gazebosim.org/1.0/Mingfei/models/actor/tip/files/meshes/walk.dae</filename>
        <interpolate_x>true</interpolate_x>
    </animation>

    <script>
        <loop>true / false</loop>
        <delay_start>0.0</delay_start>
        <auto_start>true</auto_start>
        
        <trajectory id="0" type="walking">
            <!-- The generated waypoints given the provided inputs -->
        </trajectory>
    </script>
</actor>

Example

Given the following waypoints:

waypoints = [
    [4, -3.2, 1.0],
    [0.8, -3.2, 1.0],
    [0.8, -6, 1.0],
    [4, -6, 1.0],
]
actor_xml = gz.make_actor("my_actor", waypoints, walk_speed=0.8, rotate_speed=1.8, enable_loop=True)

will return the following xml:

<actor name="my_actor">
    <skin>
        <filename>https://fuel.gazebosim.org/1.0/Mingfei/models/actor/tip/files/meshes/walk.dae</filename>
    </skin>
    
    <animation name="walking">
        <filename>https://fuel.gazebosim.org/1.0/Mingfei/models/actor/tip/files/meshes/walk.dae</filename>
        <interpolate_x>true</interpolate_x>
    </animation>

    <script>
        <loop>true</loop>
        <delay_start>0.0</delay_start>
        <auto_start>true</auto_start>
        
        <trajectory id="0" type="walking">
            <waypoint>
                <time>0</time>
                <pose>4 -3.2 1.0 0 0 3.141592653589793</pose>
            </waypoint>
            <waypoint>
                <time>4.0</time>
                <pose>0.8 -3.2 1.0 0 0 3.141592653589793</pose>
            </waypoint>
            <waypoint>
                <time>6.617993877991495</time>
                <pose>0.8 -3.2 1.0 0 0 -1.5707963267948966</pose>
            </waypoint>
            <waypoint>
                <time>10.117993877991495</time>
                <pose>0.8 -6 1.0 0 0 -1.5707963267948966</pose>
            </waypoint>
            <waypoint>
                <time>10.99065850398866</time>
                <pose>0.8 -6 1.0 0 0 0.0</pose>
            </waypoint>
            <waypoint>
                <time>14.99065850398866</time>
                <pose>4 -6 1.0 0 0 0.0</pose>
            </waypoint>
            <waypoint>
                <time>15.863323129985826</time>
                <pose>4 -6 1.0 0 0 1.5707963267948966</pose>
            </waypoint>
            <waypoint>
                <time>19.363323129985826</time>
                <pose>4 -3.2 1.0 0 0 1.5707963267948966</pose>
            </waypoint>
            <waypoint>
                <time>20.23598775598299</time>
                <pose>4 -3.2 1.0 0 0 3.141592653589793</pose>
            </waypoint>
            
        </trajectory>
    </script>
</actor>

4 - Rosbag Helpers in the Artefacts Toolkit

The Artefacts Toolkit Rosbag Helpers provide convenient functions to create, record, and extract data from ROS bag files while running tests. These utilities help you capture topic data to be processed for analysis and visualisation.

Import with:

from artefacts_toolkit.rosbag import get_bag_recorder, image_topics, message_topics

Functions

Function Reference

get_bag_recorder

Creates a rosbag2 recorder for a given list of topic names and returns the node and the filepath

rosbag.get_bag_recorder(
    topic_names,
    directory="rosbags",
    use_sim_time=False
)

Parameters

Parameter Type Description Default
topic_names list[str] List of ROS topics to record Required
directory str Directory where the rosbag will be saved "rosbags"
use_sim_time bool Whether to use simulation time instead of system time False

Returns

tuple: Returns a tuple containing:

  • bag_recorder(ExecuteProcess): Launch Action that runs the recorder process.
  • rosbag_filepath(str): Path to the rosbag file that will be created.

Example

The following example shows a test launch file using the bag_recorder helper to record a rosbag, add the bag_recorder to the launch description, and later use the rosbag_filepath for an assertion test (using another rosbag helper get_final_message):

@pytest.mark.launch_test
def generate_test_description():
    camera_topics = ["/depth_cam/rgb"]
    extra_topics = ["/odom", "/noisy_estimate"]

    bag_recorder, rosbag_filepath = rosbag.get_bag_recorder(
        camera_topics + extra_topics, use_sim_time=False
    )

    test_odometry_node = ExecuteProcess(
        cmd=[
            "python3",
            "src/test_odometry_node.py",
        ]
    )
    return LaunchDescription(
        [
            test_odometry_node,
            launch_testing.actions.ReadyToTest(),
            bag_recorder,
        ]
    ), {
        "test_odometry_node": test_odometry_node,
        "rosbag_filepath": rosbag_filepath,
    }


@launch_testing.post_shutdown_test()
class TestProcOutputAfterShutdown(unittest.TestCase):
    def test_end_position(self, rosbag_filepath):
        final_distance_from_start = message_topics.get_final_message(
            rosbag_filepath, "/distance_from_start.data"
        )

        assert final_distance_from_start < 0.1, (
            f"Final distance from start is {final_distance_from_start}, expected less than 10cm"
        )

image_topics.extract_camera_image

Returns the last recorded image from a provided camera topic.

image_topics.extract_camera_image(
    rosbag_file_path,
    camera_topic,
    output_dir="output"
)

Parameters

Parameter Type Description Default
rosbag_file_path str Path to the recorded rosbag Required
camera_topic str Name of the ROS camera topic to take the image from Required
output_dir str Directory where to save the extracted image to "output"

Returns

None: The image will be saved to the output_dir specified.

Example

The following example shows how to extract the last camera image from the rosbag after the test has concluded. We use the rosbag_filepath returned by the rosbag.get_rosbag_recorder function.

from artefacts_toolkit.rosbag import rosbag, image_topics
def test_exit_code(
    self, proc_info, test_odometry_node, rosbag_filepath
):
    ...

    image_topics.extract_camera_image(rosbag_filepath, "/depth_cam/rgb")

image_topics.extract_video

Creates a WebM video by combining all images from a provided camera topic.

image_topics.extract_video(
    bag_path,
    topic_name,
    output_path,
    frame_rate=20
)

Parameters

Parameter Type Description Default
bag_path str Path to the recorded rosbag Required
topic_name str Name of the ROS camera topic to create the video from Required
output_path str Path where the video will be saved (.webm) Required
frame_rate int Frame rate to use for the created video 20

Returns

None: The video file will be saved to the output_path specified.

Notes

Example

The following example shows how to extract a video from the rosbag after the test has concluded. We use the rosbag_filepath returned by the rosbag.get_rosbag_recorder function.

from artefacts_toolkit.rosbag import rosbag, image_topics
def test_exit_code(
    self, proc_info, test_odometry_node, rosbag_filepath
):
    ...

    image_topics.extract_video(rosbag_filepath, "/depth_cam/rgb", "output/depth_cam.webm")

message_topics.get_final_message

Retrieves the final message from a specified topic in a rosbag, with optional attribute access using dot notation.

message_topics.get_final_message(
    rosbag_filepath,
    topic
)

Parameters

Parameter Type Description Default
rosbag_filepath str Path to the recorded rosbag Required
topic str Topic name. Use dot notation to drill down message attributes (e.g., “/distance.data”) Required

Returns

Any: Returns the value of the specified message attribute. The type depends on the accessed field.

Example

We will use the same example from the rosbag.get_bag_recorder explanation. Note we get the data from the distance_from_start topic in order to make our assertion test.

@launch_testing.post_shutdown_test()
class TestProcOutputAfterShutdown(unittest.TestCase):
    def test_end_position(self, rosbag_filepath):
        final_distance_from_start = message_topics.get_final_message(
            rosbag_filepath, "/distance_from_start.data"
        )

        assert final_distance_from_start < 0.1, (
            f"Final distance from start is {final_distance_from_start}, expected less than 10cm"
        )

5 - Rerun Helpers in the Artefacts Toolkit

The Artefacts Toolkit Rerun Helpers provide convenient functions for tests that record a Rerun .rrd file during a simulation and then make their assertions against that recording, rather than against the simulator itself.

This splits a test into two independent halves. The simulator (or a logger node next to it) records while the test itself then asserts against the recording, so no ROS or simulator knowledge is needed to write the assertions.

Import with:

from artefacts_toolkit.rerun import recorder, reader, video

Functions

Function Reference

recorder.get_output_dir

Returns (and creates) the directory that recordings and other test outputs should be written to.

recorder.get_output_dir(
    directory=None
)

Parameters

Parameter Type Description Default
directory str or Path Explicit directory to use instead of the defaults None

Returns

Path: The absolute path of the output directory.

Example

The same directory is resolved on the simulator side and on the test side, so both agree on where the recording is:

from artefacts_toolkit.rerun import recorder

OUTPUT_FOLDER = recorder.get_output_dir()
rrd_path = OUTPUT_FOLDER / "my-rerun-recording.rrd"

recorder.start_recording

Starts a Rerun recording saved to directory/filename and returns the recording stream and the file path.

recorder.start_recording(
    application_id,
    filename="recording.rrd",
    directory=None,
    handle_sigterm=True
)

Parameters

Parameter Type Description Default
application_id str Name of the application shown in the Rerun viewer Required
filename str or Path Name of the .rrd file. May also be an absolute path, in which case directory is ignored "recording.rrd"
directory str or Path Directory to save to, resolved like get_output_dir None
handle_sigterm bool Close the recording and exit cleanly when the process receives SIGTERM True

Returns

tuple: Returns a tuple containing:

  • recording (rerun.RecordingStream): The started recording. It is also made the global recording, so plain rr.log(...) and rr.set_time(...) calls go to it.
  • path (Path): Path to the .rrd file being written.

Example

In the following example the simulator records the base pose and the joystick commands on a sim_time timeline, and is stopped by the test harness once the mission ends. Setting your own timeline with rr.set_time is what lets the test later line data up by simulation time rather than wall-clock time.

import rerun as rr
import simulator as sim
from artefacts_toolkit.rerun import recorder


class RerunSimulator(sim.SimulatorMujoco):
    def _set_sim_time(self):
        rr.set_time("sim_time", duration=float(self.mujoco_data.time))

    def _log_joy(self, joy):
        self._set_sim_time()
        rr.log("cmd/fwd", rr.Scalars(joy.axes[1]))
        rr.log("cmd/yaw", rr.Scalars(joy.axes[2]))

    def _read_state(self):
        super()._read_state()
        q = self.mujoco_data.qpos  # [0:3] base xyz, [3:7] base quat wxyz
        w, x, y, z = q[3:7]
        self._set_sim_time()
        rr.log("base", rr.Transform3D(translation=q[0:3],
                                      quaternion=rr.Quaternion(xyzw=[x, y, z, w])))
        rr.log("base/yaw", rr.Scalars(yaw_from_quat(w, x, y, z)))


def main():
    robot = sim.Robot(sim.RobotType.Tron2, True)
    robot.init("127.0.0.1")

    # After robot.init(): the SDK resets signal handlers
    recorder.start_recording("tron2_loop", "recording-loop.rrd")

    RerunSimulator(...).run()

reader.load

Loads a recording into memory once, so several assertions can share it without re-reading the file.

reader.load(
    rrd
)

Parameters

Parameter Type Description Default
rrd str or Path Path to the .rrd file Required

Returns

ChunkStore: The loaded recording. Every other reader and video function accepts it in place of the path.

Example

from artefacts_toolkit.rerun import reader


@pytest.fixture(scope="module")
def recording(recording_path):
    assert recording_path.exists(), f"Recording not found at {recording_path}"
    return reader.load(recording_path)

reader.get_entity_paths

Returns all entity paths logged in the recording.

reader.get_entity_paths(
    rrd
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required

Returns

list[str]: The entity paths, for example ["/base", "/base/yaw", "/cmd/fwd"].

Example

assert "/bodies/uwb_tag" in reader.get_entity_paths(recording), "target was never logged"

reader.get_timelines

Returns the names of the timelines in the recording.

reader.get_timelines(
    rrd
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required

Returns

list[str]: The timeline names, for example ["log_time", "sim_time"].


reader.get_columns

Returns the data column names of the recording, optionally only those of one entity.

reader.get_columns(
    rrd,
    entity=None
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
entity str Only list the columns of this entity None

Returns

list[str]: Column names in the "/entity:Archetype:field" form.

Example

>>> reader.get_columns(recording, "/base")
['/base:Transform3D:quaternion', '/base:Transform3D:translation']

reader.get_column

Returns the times and values of one column, ordered by a timeline.

reader.get_column(
    rrd,
    column,
    timeline=None
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
column str Column name, e.g. "/base/yaw:Scalars:scalars" Required
timeline str Timeline to order by. Defaults to the recording’s own timeline (the first one that is not log_time), else log_time None

Returns

tuple: Returns a tuple containing:

  • times (numpy.ndarray): Float seconds for duration timelines, datetime64 for timestamp timelines, integers for sequence timelines.
  • values (numpy.ndarray): Shape (n,) for scalars and (n, 3) for 3-vectors. When rows hold a varying number of instances (e.g. point clouds), an object array of per-row arrays.

Example

times, yaw = reader.get_column(recording, "/base/yaw:Scalars:scalars")
assert times[-1] > 60.0, "mission ended early"
assert np.abs(yaw).max() <= np.pi

reader.get_final_message

Retrieves the last value logged to a column, or the value of a static column.

reader.get_final_message(
    rrd,
    column
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
column str Column name, e.g. "/base:Transform3D:translation" Required

Returns

Any: The value, unwrapped: a float for Scalars, a length-3 array for a translation, an (n, 3) array for a Points3D with several points.

Example

Waypoints are logged once (static) as Points3D, and the test measures how far the robot strays from the track they describe:

wp_col = "/network/mission_waypoints:Points3D:positions"
assert wp_col in reader.get_columns(recording), f"Could not find {wp_col} in recording"

track = np.vstack(reader.get_final_message(recording, wp_col))[:, :2]  # N x 2

reader.get_message_count

Returns the number of log calls made to an entity.

reader.get_message_count(
    rrd,
    entity
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
entity str Entity path, e.g. "/cam" Required

Returns

int: The number of (non-static) rows logged to the entity, 0 if it does not exist.

Example

assert reader.get_message_count(recording, "/cam") >= 100, "camera stopped publishing"

reader.to_dataframe

Returns the recording as a pandas DataFrame, one row per time on the chosen timeline.

Different entities are usually logged at different rates: a pose every simulation step, a joystick command only when it changes. With step, every selected entity is resampled onto one uniform time grid and, with fill_latest_at, gaps are filled with the latest earlier value, so a sparse command topic lines up row by row with a dense pose stream and the two can be compared directly.

reader.to_dataframe(
    rrd,
    contents=None,
    index=None,
    step=None,
    fill_latest_at=True
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
contents str or list[str] Entities to include: "/base" for one entity, "/base/**" for a subtree, or a list of those. All entities when None None
index str Timeline to use as the row index. Defaults like get_column None
step float Resample onto a uniform grid of this many seconds, from the first to the last time None
fill_latest_at bool Fill gaps with the latest earlier value of each column True

Returns

pandas.DataFrame: The timeline column comes first (float seconds for duration timelines), followed by one column per "/entity:Archetype:field". Static data is joined onto every row. Scalars columns are plain float64 (NaN where nothing was logged); vectors are arrays.

Example

@pytest.fixture(scope="module")
def df(recording_path):
    """Whole recording resampled onto a uniform 20 ms sim-time grid."""
    frame = reader.to_dataframe(recording_path, ["/base/**", "/cmd/**"], index="sim_time", step=0.02)

    out = frame.dropna(subset=["/base:Transform3D:translation"]).reset_index(drop=True)
    out["x"] = [t[0] for t in out["/base:Transform3D:translation"]]
    out["y"] = [t[1] for t in out["/base:Transform3D:translation"]]
    out["yaw"] = out["/base/yaw:Scalars:scalars"]
    out["cmd_fwd"] = out["/cmd/fwd:Scalars:scalars"].fillna(0.0)
    return out


def test_straights_cover_side_length(df, straights):
    for i, (a, b) in enumerate(straights):
        dist = math.hypot(df["x"][b - 1] - df["x"][a], df["y"][b - 1] - df["y"][a])
        assert abs(dist - SIDE_LENGTH) < DIST_TOLERANCE

video.extract_video

Creates an MP4 video from a camera entity in the recording.

video.extract_video(
    rrd,
    entity,
    output_path,
    timeline=None,
    frame_rate=20
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
entity str Entity path of the camera, e.g. "/cam" Required
output_path str or Path Path where the video will be saved (.mp4) Required
timeline str Timeline to order frames by. Defaults like get_column None
frame_rate int Frame rate used when encoding rr.Image frames 20

Returns

Path: The path of the saved video.

Example

from artefacts_toolkit.rerun import recorder, video

video.extract_video(recording, "/cam", recorder.get_output_dir() / "head_camera.mp4")

video.extract_camera_image

Saves the last image logged to a camera entity as a PNG.

video.extract_camera_image(
    rrd,
    entity,
    output_dir="output"
)

Parameters

Parameter Type Description Default
rrd str, Path or ChunkStore Path to the recording, or a loaded one Required
entity str Entity path of the camera, e.g. "/cam" Required
output_dir str or Path Directory where to save the extracted image "output"

Returns

Path: The path of the saved image, output_dir/<entity>.last.png with / replaced by _ (for example output/_cam.last.png).

Example

video.extract_camera_image(recording, "/cam", recorder.get_output_dir())