When testing complex applications, test teams frequently turn to "modular testing" as a way to break down application functionality into small pieces. Creating new Tests becomes easier as some building blocks may already be present and can be reused across different Tests. One of the most common use cases for modular Tests is end-to-end Tests.

Xray must support modular Tests by allowing testers to compose Tests that "call" other Test cases from Manual Test Steps.

Modular Tests

When specifying manual Tests, users can create special steps that will call other Test cases.

The building blocks for modular Tests will be other manual Tests in Xray.

"Called" Tests can, in turn, call other Tests. Effectively, one can have a tree structure with up to 5 levels of depth.

Xray must check for cyclic dependencies when modular Tests are being composed. In case a cyclic dependency is detected, an error is shown to the user and the Test can't be "called".

On a modular Test issue screen, the number of steps of the called test must be displayed on the call step.

Called Tests are just references to other existing Tests. In other words, the called Tests specification will not be copied into the parent Test. Therefore, any change to called Tests must be propagated to the parent Tests.

In case a called Test is deleted, all Tests where the deleted Test is referenced must display an invalid step. At this point, users can either delete the step or change the referenced Test.

Called Tests will not be editable through the parent Test issue.

Parameterized Modular Tests

It will also be possible to call parameterized Tests. In this case, the user has the ability to define the called test parameters on the parent test dataset.

Data-driven modular tests can also be defined by specifying multiple iterations within the dataset of the parent test.

Example:
"Test A" has parameters P1 and P2. "Test A" calls "Test B", which in turn, has parameter P3 defined.

In order to define parameter values for P1, P2 and P3, users need to edit the dataset in "Test A".

Execution

On the Execution page, all called Tests must be unfolded with the respective steps and parameter names replaced by their values present on the dataset.

The step number for called Tests must be a composed number with reference to the ascendant(s) step number(s).

Examples

Scenario 1: Modular Test without parameterization

e2e Test order flow
│   Step 1
│   Step 2
│   Step 3
└───Step 4 (call: place order through mobile app)
│   │   Step 4.1
│   │   Step 4.2
│   │   Step 4.3
│   │   Step 4.4
│   └───Step 4.5 (call: choose payment method) 
│       │   Step 4.5.1
│       │   Step 4.5.2
│       │   ...
└───Step 5 (call: process order in backoffice)
│   │   Step 5.1
│   │   Step 5.2
│   │   ...
│   Step 6
│   Step 7

Scenario 2: Parameterized Modular Test

Dataset:
CardType | CardNumber
Visa     | 111111111111111


e2e Test order flow
│   Step 1
│   Step 2
│   Step 3
└───Step 4 (call: place order through mobile app)
│   │   Step 4.1
│   │   Step 4.2
│   │   Step 4.3
│   │   Step 4.4
│   └───Step 4.5 (call: choose payment method) 
│       │   Step 4.5.1
│       │   Step 4.5.2 Enter card details: Type ${CardType}, Number: ${CardNumber}
│       │   ...
└───Step 5 (call: process order in backoffice)
│   │   Step 5.1
│   │   Step 5.2
│   │   ...
│   Step 6
│   Step 7