Skip to content

Running tests in artefacts cloud simulation

Overview

When using artefacts run [jobname] the tests will be run locally. However if your tests take time, for example if you have multiple parametrized scenarios, you may want to run then on artefacts cloud simulation.

In that case you can use artefacts run-remote [jobname]. Your local code will be compressed in an archive file and sent to our servers for execution.

Note: While the job should appear within a few seconds in the dashboard, it can take several minutes before actually starting. Therefore it is advised to use this function for longer tests, or for checking that the tests are properly set up for continuous integration (see section on Continuous Integration with Github) It is currently required to link your project to a github repository to use this feature.

Below is an overview of the execution model.

graph TD subgraph artefacts ac(cloud simulation) --> dashboard(dashboard) end subgraph LocalMachine lc(local code) -.-CLI CLI --run_local-->dashboard CLI --run_remote-->ac end lc --push--> github github --pushEvent/pullCode--> ac

Execution time on artefacts cloud simulation will be counted against your usage quota.

Packaging for Cloud Simulation

Artefacts supports some simulators and frameworks out of the box. In that case, all you need to do is provide a test_launch file (See Running and Tracking Tests)

Currently supported are:

  • ROS2 with Gazebo Ignition
runtime:
    simulator: gazebo:fortress
    framework: ros2:humble # or ros2:galactic
  • ROS1 with Gazebo classic
runtime:
    simulator: gazebo:11
    framework: ros1:noetic

Make sure that those are properly specified in the runtime section of the config file (see configuration syntax). Alternatively, (such as if your project does not use ROS) you may need to prepare a Docker package to run on artefacts cloud-simulation.

Customizing Packaging for Cloud Simulation

In the majority of cases, just providing the framework and simulator to be used in the runtime block, e.g.:

runtime:
  simulator: gazebo:11
  framework: ros1:noetic

and a test_launch file will be enough for Artefacts to build and test your project wthout any other input. However, we appreciate that for some projects, some customization / fine-tuning is necessary.

To provide for these cases, the following keys are available to you in the artefacts.yaml file, package['custom'] section:

package:
  custom:
    os: # string
    include: # List
    commands: # List
  • os: string Input a base image (e.g. ubuntu:22.04). This overrides the base image that Artefacts will use based on your framework and simulator choice.

Example:

package:
  custom:
    os: ubuntu:22.04
  • include: list By default, Artefacts will copy over your github repo (continuous integration) or current working directory recursively('run-remote') to the container running on our servers. Use include to instead specify which directories / files you want available in the container.

Example:

package:
  custom:
    include:
      - ./path/to/my_necessary_files
      - ./makefile
  • commands: list If you require any additional bash commands to be performed before the build stage of your project, enter them here. A common use case is when a custom source is required in addition to the regular ros source when building a ros / gazebo project.

Example:

package:
  custom:
    commands:
      - source simulator/my_workspace/install/setup.bash
runtime:
  framework: ros1:noetic
  simulator: gazebo:11