Working with the Orchestrator UI
The Orchestrator UI is the primary way to interact with the Orchestrator. From it you can
monitor and cancel Test Runs, inspect their results, trigger new runs, and manage your
Known Test Plans. The rtf CLI is only needed for triggering runs from scripts and for pulling
run output to your local machine.
The UI uses the same Google sign-in as the rest of the Orchestrator (see Getting set up with the Orchestrator). Once signed in, a badge in the header shows your role for the page you are viewing: Admin, Owner, or Can trigger.
Finding a run
There are three ways to find a run in the UI:
- Recent runs: the home page lists recent Test Runs, which can be filtered by who started them and how recently they were started.
- By ID: enter a Test Run ID in the box in the header and select View status.
- By Known Test Plan: open the Known Test Plan from Registered Test Plans and look under Previous runs.
Runs triggered using rtf remote run or rtf remote run-known also print a direct link to the new
run’s page.
Checking queue and cluster capacity
The home page shows the current state of the Orchestrator:
- Current Queue State: queued and running Test Executions compared to the Orchestrator’s total capacity
- Workload Pools: queued work for each Workload Pool, along with the utilization of each cluster and any Dedicated Cluster claims
- Recent Executions: the number of Test Executions run over the last 24 hours
- Workload pool configuration: the rate limits, clusters, and nodes for each pool
If the Orchestrator is busy, your run will remain queued until capacity becomes available. Check these sections before triggering particularly large runs.
Inspecting a run
A run’s page refreshes automatically while the run is in progress. It shows the run’s overall status, the Trigger Variables it was started with, and a breakdown of its Test Executions by status. The list of executions can be filtered by status, and each execution provides links to:
- The execution’s own page, showing its exit code and status history
- The container logs for the execution in GCP
- The execution’s metrics in Grafana
- The execution’s
log.txtandoutput.zip
To download the output for every execution in the run at once, select Download full run output
and run the rtf command that it shows.
Execution output is only retained for a limited time, so these downloads stop working for older runs.
Cancelling and re-running
To cancel a run, select Cancel run on the run’s page and confirm. This button is only shown on runs that you started or that belong to a Known Test Plan that you own. Admins are able to cancel any run.
To run a Known Test Plan again with the same variables, select Re-run. This opens the Known Test Plan’s trigger form, pre-filled with the variables from the original run.
Triggering a run
To trigger a Known Test Plan, open its page and either select Trigger N executions to run with the current variable values, or expand Customize ref & variables to run from a different branch, tag, or commit and to override individual Trigger Variables. If you aren’t permitted to trigger the Known Test Plan, the page lists its owners for you to contact.
To trigger a Test Plan that hasn’t been registered, select Trigger from GitHub and provide the
organization, repository, and path to the test-plan.yaml, along with an optional Git ref and any
variables as a JSON object. Matrix dimensions are provided as JSON arrays. Runs triggered this way
always run in the default Workload Pool.
Editing a Known Test Plan
Owners and Admins can edit a Known Test Plan from the Edit test plan section of its page. Fields that can only be changed by an Admin are disabled for owners.
See Registering and managing Known Test Plans for what each setting does.