Running tests in artefacts cloud simulation

Overview

When using artefacts run [jobname] the tests will be run locally. However if your tests take time, for example if you have multiple parameterized scenarios, you may want to run them on Artefacts cloud simulation.

In that case you can use artefacts run-remote [jobname]. Your local code will be compressed in an archive file and sent to our servers for execution.

Note: While the job should appear within a few seconds in the dashboard, it can take several minutes before actually starting. Therefore it is advised to use this function for longer tests, or for checking that the tests are properly set up for continuous integration (see section on Continuous Integration with Github) It is currently required to link your project to a github repository to use this feature.

Below is an overview of the execution model.

Execution time on Artefacts cloud simulation will be counted against your usage quota.

The .artefactsignore file

You may have files within your project that are not required for running your test (e.g rosbags, some assets). If that is the case, and in order to keep the upload archive filesize down, you may add an .artefactsignore file to the root of your project. It can be used in the same way as a .gitignore file, i.e:

rosbags/
venv/
.DS_Store

will not bundle the rosbags and venv folders, as well as the hidden file .DS_Store.

Packaging for Cloud Simulation

Artefacts supports some simulators and frameworks out of the box. In that case, all you need to do is provide a test_launch file (See Running and Tracking Tests)

Currently supported are:

• ROS2 with Gazebo Ignition (Fortress)

runtime:
  simulator: gazebo:fortress
  framework: ros2:humble

• ROS2 with Gazebo (Harmonic)

runtime:
  simulator: gazebo:harmonic
  framework: ros2:humble 

Make sure that those are properly specified in the runtime section of the config file (see configuration syntax). Alternatively, (such as if your project does not use ROS) you may need to prepare a Docker package to run on Artefacts cloud-simulation.

Customizing Packaging for Cloud Simulation

In the majority of cases, just providing the framework and simulator to be used in the runtime block, e.g.:

runtime:
  simulator: gazebo:fortress
  framework: ros2:humble

and a test_launch file will be enough for Artefacts to build and test your project wthout any other input. However, we appreciate that for some projects, some customization / fine-tuning is necessary.

To provide for these cases, the following keys are available to you in the artefacts.yaml file, package['custom'] section:

package:
  custom:
    os: # string
    include: # List
    commands: # List

• os: string Input a base image (e.g. ubuntu:22.04). This overrides the base image that Artefacts will use based on your framework and simulator choice.

Example:

package:
  custom:
    os: ubuntu:22.04

• include: list By default, Artefacts will copy over your github repo (continuous integration) or current working directory recursively('run-remote') to the container running on our servers. Use include to instead specify which directories / files you want available in the container.

Example:

package:
  custom:
    include:
      - ./path/to/my_necessary_files
      - ./makefile

• commands: list If you require any additional bash commands to be performed before the build stage of your project, enter them here. A common use case is when a custom source is required in addition to the regular ros source when building a ros / gazebo project.

Example:

package:
  custom:
    commands:
      - source simulator/my_workspace/install/setup.bash
runtime:
  framework: ros2:humble
  simulator: gazebo:fortress