Making your test plan Orchestrator-ready

In this guide, we’ll take the Test Plan from the “Writing test plans” tutorial series and update it so it can be run under the RTF Orchestrator. To do this we will be adding rtf.io docker compose labels that tell the Orchestrator how to handle deploying and interacting with your services.

Prerequisites

• Completed the “Writing test plans” tutorial series
• An rtf-hello-world directory in the state it was at the end of that series

Restrictions on Orchestrator Test Plans

The Orchestrator enforces that Test Plans submitted to it are compatible for running in a Kubernetes cluster.

Compatible Environments are:

• docker compose based. The Orchestrator converts your docker compose based environments into Kubernetes resources using kompose which are then patched with kustomize according to the labels detailed below.
• null(if the skip = true field is set in the Environment config). This skips environment provisioning in the Orchestrator and just runs the Scenario.

The only compatible Scenario is a docker based one.

The Test Plan you built in the “Writing test plans” series already uses a docker compose based Environment and docker based Scenario, so all we need to add is the appropriate labels and it will be ready to run!

The rtf.io labels

The Orchestrator uses docker compose service labels in the rtf.io namespace to determine how it should handle your services during deployment and execution. These labels have no effect when you run a Test Plan locally with rtf run: they exist solely to allow the Orchestrator to replicate the execution behaviour of rtf run in Kubernetes where we can’t rely on shared local filesystem.

Currently there are three labels available:

Adding log collection

When set to "true" on a service, the rtf.io/log-collection label will instruct the orchestrator to collect that service’s container logs after the Scenario completes and includes them in the output zip file that gets pushed to GCS under output/logs/.

Let’s add it to the hello-world service in our environment.yaml:

name: Docker compose environment config
description: A docker compose environment config
compose_files:
  - name: docker-compose.yaml
    kind: inline
    content: |
      services:
        hello-world:
          image: nginx:alpine
          ports:
            - "8080:80"
# -------- Add rtf.io labels --------
          labels:
            rtf.io/log-collection: "true"
# -----------------------------------

Note Docker compose label values must be strings. Unquoted true is interpreted as a boolean and will not match the expected string value "true". Always quote the value.

Remember: there is no automatic log collection. You must add the rtf.io/log-collection label to each service you want to collect logs from.

Accessing file provider output

Any service that needs to access the output from RTF file providers needs to be annotated with the rtf.io/file-providers: "true" label. This instructs the Orchestrator to add an init container that will resolve and mount the required file provider output into the container via a shared volume. As with rtf run the environment variables you specify in your test plan will contain the correct absolute path for the requested resources, regardless of whether you run locally or under the Orchestrator.

services:
  router:
    image: "ghcr.io/apollographql/router:v2.14.0"
    labels:
      rtf.io/log-collection: "true"
      rtf.io/file-providers: "true"
    # The SUPERGRAPH_SCHEMA and ROUTER_CONFIG environment variables here
    # are set according to the file providers that have been added using
    # the label.
    command: -s ${SUPERGRAPH_SCHEMA} -c ${ROUTER_CONFIG}

Note A service using rtf.io/file-providers must not attempt to define explicit volume mounts for individual provider paths. Doing so will cause the Orchestrator to reject your Test Plan as invalid.

Verifying the test plan

As with local execution, we run rtf template to confirm the updated test plan still parses correctly:

rtf template test-plan.yaml --check

The output should be similar to:

name: Hello World
description: A test plan created as a guide for writing test plans
variables:
  scenario_variable: scenario env var value from test plan
matrix:
  variant_names: null
  dimensions: {}
  compound: {}
custom_providers: []
scenario:
  ...
environment:
  ...

Your test plan is now ready to submit to the Orchestrator.

Next steps

In this guide, we added rtf.io labels to our docker compose environment so the Orchestrator knows how to handle our services. Next, we’ll submit the Test Plan to the Orchestrator for execution and look at how we monitor the status of the execution before fetching the results.

Running and fetching results