Regression Sets & Dynamic Runs

Regression Sets let you define reusable collections of test cases and section subscriptions, then create Test Runs from them through the API. They are the recommended way to drive automated run creation from CI/CD pipelines or automation frameworks.

What is a Regression Set?#

A Regression Set is a project-scoped template that contains two kinds of items:

Item type	Behavior
`case`	A static reference to a specific test case. The case is resolved once at run creation time.
`section`	A dynamic subscription to a section. New cases added to the section later automatically join any `planned` or `in_progress` run that subscribed to it.

Use section items when you want runs to stay in sync with the library as your team adds new cases. Use case items when you need a fixed, repeatable subset.

Required API scopes#

Operation	Scope
Read regression sets / preview	`testsets:read`
Create regression set	`testsets:create`
Update regression set	`testsets:update`
Delete regression set	`testsets:delete`
Create test runs from sets	`testruns:create`
Split a run	`testruns:create`

Create a Regression Set#

POST /api/v1/projects/{id}/test-sets
Content-Type: application/json
Authorization: Bearer <api_key>

{
  "name": "Sprint Smoke",
  "description": "Smoke coverage for every sprint",
  "items": [
    { "item_type": "case", "case_id": "<case-uuid>", "position": 0 },
    { "item_type": "section", "section_id": "<section-uuid>", "position": 1 }
  ]
}

position controls the order items appear in. A set can contain up to 1,000 items.

Preview a Regression Set#

Before creating a run, you can preview what cases a set currently resolves to:

GET /api/v1/projects/{id}/test-sets/{setId}/preview
Authorization: Bearer <api_key>

The response includes the current total case count and a sample of cases:

{
  "id": "<set-id>",
  "name": "Sprint Smoke",
  "current_case_count": 42,
  "sample_cases": [
    { "id": "<case-id>", "title": "Login", "display_code": "TC-MOB-0001" }
  ]
}

Create a Test Run from Regression Sets#

POST /api/v1/testruns
Content-Type: application/json
Authorization: Bearer <api_key>

{
  "name": "Sprint 12 Regression",
  "project_id": "<project-uuid>",
  "test_set_ids": ["<set-id-1>", "<set-id-2>"],
  "mode": "unified",
  "environment": "staging",
  "release_id": "<release-uuid>",
  "build_id": "<build-uuid>"
}

mode: "unified" creates a single run containing every resolved case and section subscription.
mode: "split_by_set" creates one run per regression set. Each child run is tagged with source_test_set_id.
test_case_ids and test_set_ids are mutually exclusive.

Dynamic section subscriptions are only attached in unified mode. Runs created with split_by_set are independent static runs.

Dynamic section behavior#

When a test case is created or moved into a section that an in-flight run subscribes to, the case automatically becomes an execution in that run. Runs in planned or in_progress status receive new cases; submitted, completed, aborted, and superseded runs do not.

If a case is moved out of a subscribed section:

planned runs: the execution is removed.
in_progress runs: the execution is removed only if it has not yet been executed (result IS NULL).
Executed cases are never removed automatically.

This fan-out is best-effort. The underlying create/move/delete operation succeeds even if subscription propagation fails.

Split a Test Run#

Splitting moves unexecuted cases from an existing run into one or more child runs.

POST /api/v1/testruns/{id}/split
Content-Type: application/json
Authorization: Bearer <api_key>

{
  "strategy": "by_section",
  "name": "Sprint 12 Regression — Split",
  "assignee_id": "<user-uuid>"
}

Alternatively, name the assignee by email — assignee_email is resolved case-insensitively to a user inside your organization (touchless T4) and follows the same validation and notifications as assignee_id (send one or the other, never both: 400 assignee_conflict).

Strategies:

Strategy	Behavior
`by_section`	One child run per section that has unexecuted cases.
`manual`	Move only the `execution_ids` you provide.
`by_set`	Move all portable cases into a single child run (no set provenance is preserved on executions).

Executed cases stay locked in the original run. If the original run is left with no executions, its status becomes superseded. The original run keeps its dynamic subscriptions; new cases in subscribed sections still land there.

Common errors#

Error	Cause
`VALIDATION_ERROR`	Missing required fields, invalid UUIDs, or mutually exclusive `test_case_ids`/`test_set_ids`.
`RESOURCE_NOT_FOUND`	Project, regression set, run, release, or build does not exist or is not accessible.
`FORBIDDEN`	API key scope or project scoping does not allow the operation.
`CONFLICT`	Split is requested on a run that has no portable (unexecuted) cases.