Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Limits, permissions, and concepts

This page describes Orchestrator behavior that isn’t visible from the rtf CLI: identities, permissions, rate limits, statuses, and output storage. See Workload Pools for how runs are scheduled.

Identities

The Orchestrator identifies each user from their Google sign-in:

IdentityDescription
UserAny Apollo user with access to the Orchestrator.
AdminA member of the Runtime Readiness team who administers the Orchestrator.
Automation UserA CI workflow using the Orchestrator GitHub Action, identified separately for each repository.

Users can also hold a role on individual Known Test Plans:

RoleDescription
OwnerCan configure the Known Test Plan and cancel any of its runs.
Can triggerCan trigger the Known Test Plan when its triggers are restricted to named users.

The badge in the header of the Orchestrator UI shows your identity and your role on the Known Test Plan you are viewing.

Permissions

ActionAdminOwnerCan triggerUserAutomation User
Trigger an unregistered Test PlanYesYesYesYesNot supported
Trigger a Known Test PlanYesYesYesUnless triggers restrictedUnless automated triggers disabled
Register a Known Test PlanYesNoNoNoNo
Edit a Known Test PlanYesYes¹NoNoNo
Add or remove ownersYesNoNoNoNo
Cancel a runYesYes²Own runsOwn runsNo
View runs, executions, output, and plansYesYesYesYesYes
  1. Except the pinned Workload Pool, the Dedicated Cluster setting, and owners.
  2. Runs of Known Test Plans they own, and runs they started.

Rate limits

Rate limits are configured per Workload Pool and are applied to each user separately. Automation Users are limited separately for each repository. Admins are exempt. Each pool limits the number of:

  • Runs in progress at once
  • Runs waiting to start
  • Test Executions waiting to start
  • Runs triggered in the last hour

The current limits for each pool are shown under Workload pool configuration in the Orchestrator UI, and are set in the Orchestrator configuration file.

Triggering a run that would exceed one of these limits fails with a 429 Too Many Requests error naming the pool and the limit that was exceeded. The Orchestrator also has an overall limit on queued Test Executions: when the queue is full, triggers fail with a 503 Service Unavailable error until capacity frees up.

Statuses

Test Runs and Test Executions progress through these statuses:

StatusTerminalDescription
INITIALISINGNoAccepted and queued
RESOLVINGNoResolving the Environment configuration
PROVISIONINGNoCreating the namespace and deploying services
ENVIRONMENT_READYNoServices are healthy and the Scenario is starting
RUNNINGNoThe Scenario is running
SUCCESSFULYesThe Scenario exited with code 0
FAILEDYesThe Scenario exited with a non-zero code
UNRUNNABLEYesThe execution couldn’t complete, for example due to a failed deploy
CANCELLEDYesThe run was cancelled by a user

Output storage

Each Test Execution’s log.txt and output.zip are stored in Google Cloud Storage and are available once the execution has finished. Downloads from the Orchestrator UI and the rtf CLI are served using time-limited signed URLs.

Stored output is retained for 14 days. A lifecycle policy on the bucket deletes each object 14 days after it was uploaded, after which the output can no longer be downloaded. Download any output you need to keep before then.

Limitations

  • Known Test Plans must template without any Trigger Variable overrides.
  • The Orchestrator UI only shows details for Known Test Plans on their repository’s default branch.
  • Each Test Execution runs a single Scenario pod.