Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Running test plans with the RTF Orchestrator Service

This section guides you through running RTF test plans using the RTF Orchestrator Service: a managed, remote execution service for running RTF Test Plans at Apollo. Instead of running your Test Plan locally with rtf run, you submit it to the Orchestrator which provisions an isolated Kubernetes namespace for deploying your Environment before then executing your Scenario and storing the results in GCS for you to retrieve.

Why use the Orchestrator?

When you run a Test Plan with rtf run, RTF handles spinning up your test environment and running your scenario on your local machine. When using scripted environments and scenarios this is incredibly flexible but also highly susceptible to being affected by how that local machine is configured. Running locally using a docker compose based environment and docker based scenario helps with making things more reproducible, but you are still subject to the constraints of the local machine you are running on.

In contrast, the Orchestrator provides a dedicated execution environment that handles running your test plan in an isolated Kubernetes namespace. This provides several advantages:

  • Minimal local requirements: the machine triggering the test run only needs to be able to submit the Test Plan to the Orchestrator, not run the services under test.
  • Output storage: logs and output artifacts are uploaded to GCS and can be retrieved by anyone with access to the Orchestrator after the run completes. Output is only retained for a limited time.
  • Consistent environments: every execution runs in an isolated namespace, giving far more consistent performance.
  • Parallel matrix execution: matrix dimensions run as independent executions managed by the Orchestrator concurrently, leading to a significant speed up in wall-clock execution time.

Accessing the Orchestrator

The Orchestrator is deployed at api.rtf.apollographql.com, with the Orchestrator UI alongside it, and is protected by Google Cloud IAP. Access is managed by the Runtime Readiness team.

Before starting this tutorial, follow the Getting set up with the Orchestrator guide to request access, authenticate with GCP, and check that you are able to reach the Orchestrator.

Prerequisites


Next: Making your test plan Orchestrator-ready