Change size is supporting information, not the starting point
Regression scope should be set not by how much code changed but by how far a wrong change reaches, how late it is detected, and how hard it is to reverse.
This does not mean ignoring file counts and line counts. Change size is supporting information for estimating review and verification effort, and the complexity and uncertainty of a change. It is just insufficient as the starting point for deciding impact scope. ISTQB also describes performing impact analysis before regression testing, and weighs the risk of the change, the size of the existing system, and the size of the change together when scoping.[2][6]
A one-line change to a permission condition can break access control across several roles. Conversely, hundreds of changed lines can need a fairly narrow regression scope if external interfaces are unchanged and the code is confined to an independent internal module.
The question to ask when scoping regression is this: if this change is wrong, which user actions, permissions, data, and integrations could fail? How large is that failure, and how quickly can we notice and reverse it?
Conceptually, regression scope = impact reach × failure consequence × change uncertainty × control difficulty. This is not an instruction to multiply numbers into a score. It describes the order in which questions are reviewed.
Finding impact candidates and choosing the run scope are different jobs
Regression test selection has two distinct steps. The first is finding what could be affected — following the links between requirements, changes, code, tests, and defects to build a candidate list. The second is deciding which of those tests to actually run for this change. This article covers only the second step.
Being on the candidate list does not mean every test has to run. Conversely, a test cannot be excluded just because no direct link is visible. Shared components, permission policies, data flows, and external systems are boundaries with large indirect impact.
ISTQB's Test Analyst syllabus likewise says no single selection technique is superior in every situation. Risk-based selection, defect history, usage patterns, coverage, and impact analysis have to be combined to fit the circumstances.[1][7]
Maintenance that keeps results trustworthy — Run with stable selectors, classify failures, repair their causes and verify the result again.
Regression scope is divided into three levels
This article distinguishes three levels: Focused Regression, Expanded Regression, and Broad / Release-level Regression. Their definitions are below.
Broad Regression does not mean running every test unconditionally. Even at this level, tests with low risk and low relevance can be excluded. The difference is that the scope becomes the important failure boundaries of the whole release, rather than the vicinity of one feature.[3]
- Focused Regression
- Focused Regression applies when there is evidence that the change is genuinely localized. It checks the changed behavior, its direct consumers, the representative happy and failure paths, and past-defect tests for that area. A small commit is not a reason to pick it.
- Expanded Regression
- Expanded Regression applies when the change touches one important user journey or a permission, data, or integration boundary but its impact is confined to a specific area. It covers the whole affected journey plus major exception paths, roles, data states, integration branches, and known consumers of shared components.
- Broad / Release-level Regression
- Broad / Release-level Regression applies when a shared boundary of several critical journeys changed, or when failure impact is high and detection or recovery is hard. It checks the critical journey set, role and permission combinations, major data lifecycles, integrations, and environment or configuration differences at release level.
Change Impact × Regression Scope Matrix
The Matrix below is not a scorecard for computing test counts. After reviewing each axis, the highest level that has evidence becomes the regression scope for this change. The axes form three groups: five for impact reach, four for the nature and uncertainty of the change, and three for failure consequence and controllability.
Regression selection based on real usage patterns prioritizes frequent and important business processes. Authentication and authorization must cover not just normal login but cross-role access and horizontal and vertical privilege escalation. Data changes can lead to CRUD lifecycle, integrity, access control, and consistency problems, and integrations need data transformation, communication flow, protocol, and end-to-end functionality examined.[7]
Defect history is not a stigma that "code with many defects is always risky". It is the basis for reselecting tests that actually found defects on similar changes and areas that reacted sensitively. Conversely, the absence of a record does not prove low risk. ISTQB describes History-based Testing, which includes in regression scope the tests that revealed defects in past runs or were sensitive to similar changes.[7]
Configuration changes must not be treated narrowly because they look smaller than code changes. As a case published by Google SRE shows, global configuration produces widespread outages through unexpected dependencies and interactions.[8]
Detectability and rollback difficulty are not functional classifications of regression tests themselves, but they are important control conditions when setting verification scope. If failure cannot be seen quickly or a wrong change is hard to reverse, widen the scope up front instead of relying on the assumption that production will catch it. Google SRE describes problems that only surface under real traffic and ambiguous canary signals, and has published a case where an unverified rollback procedure prolonged an outage.[8][9]
How to actually implement rollback-capable deployment is a separate deployment-architecture question. Here, the fact that rollback is hard is used only as an upward signal for regression scope.
Group | Axis | Focused Regression | Expanded Regression | Broad / Release-level Regression |
|---|---|---|---|---|
| Impact reach | Critical User Journey | The change is limited to a non-critical, single step. Check that behavior and its immediate neighbors. | One critical journey or a major exception branch is affected. Check the whole journey. | Shared entry, exit, or state transitions of several critical journeys changed. Check the related journey set. |
| Impact reach | Permission / Authentication | Permission rules, sessions, and auth tokens are untouched; only a role's display or behavior changes. | A specific role, resource, action, or login or session flow changes. Include allow and deny across roles and negative cases. | Central auth middleware, SSO, session policy, or the shared permission model changes. Check related roles and key actions at release level. |
| Impact reach | Data Mutation | Stored data is unchanged, or only an isolated query or presentation changes. | One entity gains create, update, delete, validation, or mapping changes. Check its lifecycle and failure handling. | Schema, migration, shared transactions, deduplication, or cross-entity consistency changes. Include several journeys and data states. |
| Impact reach | External Integration | Contract, auth, mapping, and retry conditions are unchanged; only the local adapter internals change. | One integration's contract, auth, mapping, timeout, webhook, or failure handling changes. Check success, failure, and retry flows. | A shared gateway, several providers, a shared contract, or hard-to-reverse external side effects change. Check the end-to-end integration set. |
| Impact reach | Shared Component | A single consumer, or a clearly isolated boundary. | A shared module or component with several known consumers changes. Include the main consumers. | A central component that many critical journeys depend on — auth, validation, routing, shared UI, common libraries — changes. |
| Nature and uncertainty | Configuration | A local setting for one feature or one environment with a clear boundary. | Different paths open by environment, tenant, region, or feature flag. Check representative combinations. | Global routing, security, permission, traffic, or shared policy settings change, or hidden dependencies are unclear. |
| Nature and uncertainty | Dependency | An isolated fix with no change to runtime behavior or interface. | A direct dependency's version, contract, serialization, or call pattern changes. Check main consumers and contracts. | A runtime or library shared by several services, a transitive dependency, a major version, or unclear compatibility. |
| Nature and uncertainty | Change Novelty | A familiar, repeated change with verification evidence from similar work. | A first-time branch, integration, state, or unfamiliar technology is involved. | A new architecture, runtime, or data flow with little prior evidence and uncertain behavior on real input. |
| Nature and uncertainty | Historical Defect Evidence | Recent results for similar changes and representative tests are stable. | Similar changes produced defects, or particular tests reacted sensitively more than once. | Regression defects recur in the same area, or structural defects span several components. |
| Consequence and controllability | Failure Impact | Failure is an immediately recoverable inconvenience with no effect on permissions, money, sensitive data, or core work. | A critical journey stops, recoverable data errors, work delays, or some customers are affected. | Unauthorized access, monetary errors, data integrity, privacy, or regulatory issues, or widespread outage. |
| Consequence and controllability | Detectability | Failure shows immediately; test verdicts and operational signals are clear. | Errors are delayed or intermittent and show only for some users, roles, or data states. | Silent data corruption, permission exposure, or duplicate processing that looks successful or is hard to see with current observability. |
| Consequence and controllability | Rollback Difficulty | The change is local, leaves no state, and reverts immediately by a verified method. | Extra recovery steps are needed — data reprocessing, cache clearing, external system adjustment. | Irreversible data changes, external side effects, long recovery, or an unverified rollback are expected. |
Do not average scores — raise the scope on the strongest evidence
Attaching low, medium, and high scores to twelve axes and averaging them looks simple but dilutes important risk. One permission change must not become a low grade by being averaged with several low-risk axes. The same holds for data integrity and irreversible external side effects.
So the scope is better chosen with the four rules below.
- 1. Focused is a conclusion of isolation, not a default
- Focused is not chosen because the commit is small. There must be evidence that user journeys, permissions, data, integrations, and shared components are unaffected and that failure is easy to detect and recover.
- 2. Follow the strongest upward factor
- If the central permission model or a shared data layer changed, the scope becomes Broad even when everything else looks simple.
- 3. Look at interactions between axes
- An integration change alone may be Expanded. If that integration creates data, duplicate processing has monetary consequences, failure is detected late, and rollback is hard, it rises to Broad.
- 4. Not knowing is also a reason to widen
- If the real consumers of a dependency, the impact of a configuration change, or an external system's failure behavior are unknown, that uncertainty must not be filed as low risk. Confirm the boundary with more analysis or choose the wider scope.
When a small change requires Broad Regression
Take a one-line fix to a permission condition. The code change may be a single line. But if it is a condition in shared permission middleware, several roles and key actions are affected. The change touches the Permission / Authentication, Shared Component, Failure Impact, and Detectability axes. Broad Regression covering allow and deny per role, horizontal and vertical privilege movement, session state, and key actions is the reasonable choice.
Changing only the retry setting of an external webhook is similar. Even a single value in a config file, if external events create data or change state, produces duplicate execution, reordering, and partial failure. With evidence that only one integration and limited data are affected, starting at Expanded is fine. If the same policy is shared across integrations, or the consequences are hard to reverse — money, orders, settlement — raise it to Broad.
Preserve product context as delivery capacity changes — A consistent lead maintains criteria and history while the squad adapts execution and gathers release evidence.
When a big change does not necessarily mean Broad Regression
Suppose an independent internal reporting module was heavily refactored, but external interfaces, permissions, stored data, shared libraries, and core user flows did not change. Even with a large amount of changed code, Focused or Expanded Regression is enough if the impact boundary is clear, results are easy to compare, and rollback is simple.
However, "independent" has to be proven by checking dependencies and consumers, not assumed. A large change raises review effort and uncertainty, but it is not by itself an automatic condition for Broad.[4]
Record the scope decision in a simple form
Keeping only the list of regression tests means repeating the same discussion at the next change. What was included and why things were excluded must be recorded together. The seven items below are sufficient.
If new failures, undocumented consumers, or unexpected data states appear during execution, the initial scope is not frozen. Scope selection is not a plan written once before testing starts; it is a judgment updated as new evidence arrives.[5]
Item | What to record |
|---|---|
| Change | What changed — the commit, PR, or configuration change identifier |
| Impact candidates | Candidates found by following links between requirements, change, code, tests, and defects |
| Axes that raised the scope | Which of the twelve axes were judged Expanded or Broad, and why |
| Selected scope | Focused | Expanded | Broad / Release-level |
| Tests included | Tests actually run for this change |
| Tests excluded and why | Candidates deliberately not run, with the reason |
| Conditions for widening during execution | Response to new failures, undocumented consumers, or unexpected data states |
Change size describes effort, not the direction of risk
Line counts and file counts are useful for estimating review time and complexity. They do not say what users will fail to do, how permissions and data change, or how far the change spreads into external systems and shared components.
The question that decides regression scope is not "how much changed". It is: if this change is wrong, how far can things break, and how quickly can we detect that failure and reverse it safely?
If the answer is local and clear, choose Focused Regression. If it leads to one important boundary, choose Expanded Regression. If shared boundaries of several critical journeys, large failure consequences, low detectability, or hard rollback overlap, widen to Broad / Release-level Regression.
Regression results chosen this way become important evidence for the release decision. But the fact that regression was run broadly does not by itself decide whether to release.
