Regression Sets
Regression Sets are reusable collections of test cases and whole sections. Build a set once, then spin up consistent test runs for every build without re-picking cases each time. They are the fastest way to turn "what we test for a release" into a repeatable, one-click run.
Why use Regression Sets?#
- Repeatable coverage — define a Smoke or Regression suite once and reuse it for every build.
- Less busywork — no hunting through the library to assemble the same run again and again.
- Stay in sync — subscribe to a section and new cases added there flow into the set automatically.
- Consistent reporting — runs built from the same set are directly comparable build over build.
Static vs. dynamic items#
A Regression Set holds two kinds of items, shown on the set's detail page:
| Item type | Behavior |
|---|---|
| Static item | A specific case (or section) you added individually. It is pinned and only changes when you edit the set. |
| Dynamic case | A case pulled in live from a subscribed section. Add a case to that section and it is automatically included here. |
The Current Total on the detail page is the de-duplicated count of static cases plus the current cases from every subscribed section — exactly what a run created right now would contain.
Tip: hover the ⓘ icons on the detail page for an inline reminder of what each number means. Expand a subscribed section row to preview its live cases.
1. Create a Regression Set#
- Go to Regression Sets from the project sidebar.
- Click New Regression Set.
- Give it a name (e.g., Smoke Tests, Release Regression) and an optional description of when to use it.
- Add items from the left panel:
- Cases — pin individual test cases.
- Sections — subscribe to a whole section so it stays live.
- Reorder items to control how they appear in runs, then Save.
2. Create a test run for a build#
This is the core workflow — turn a Regression Set into a run targeting a specific build.
- Open the Regression Set and click Create Run.
- Name the run (e.g., Sprint 12 — Build 142).
- Pick the Release and Build you are testing, plus an environment.
- Choose a run mode:
- Unified — one run containing all resolved cases. Best for a single pass over everything.
- Split by Set — one run per set. Useful when you create from multiple sets and want them tracked separately.
- Click Create Run and start executing.
You can also assemble a run from sets, sections, and individual cases together on the New Run page (Test Runs → New Run) — the Regression Sets and Sections tabs there pull from the same resolution logic.
Useful ways to use Regression Sets#
- Per-build smoke runs — keep a small Smoke set of P0 cases and create a run against every build for a fast go/no-go signal.
- Release regression — subscribe whole feature sections so the regression set grows automatically as the library grows.
- Risk-based suites — build a set of just the cases touching an at-risk area for a focused run before shipping.
- Recurring cadences — reuse the same set for nightly, per-PR, or pre-release runs so results stay comparable over time.
- Automation / CI — drive run creation straight from your pipeline via the API (see below).
Automating from CI/CD#
Everything above is available through the API, which is the recommended way to create runs automatically from a build pipeline. See the Regression Sets API reference for endpoints, scopes, and the static vs. section-subscription behavior.