Support configurable Test Run status aggregation for multiple results within the same Test Run

Description
Currently, when importing execution results that contain multiple outcomes for the same test within a single Test Run (e.g., due to test retries), Xray applies a fixed aggregation logic to determine the final Test Run status.

Some customers require different aggregation strategies depending on their testing and reporting needs.

Proposed Enhancement:
Provide a configurable mechanism to determine the final Test Run status when multiple results exist for the same test within a single execution import.

Possible aggregation strategies:

• Worst Result Wins (current behavior)

• • FAIL → PASS = FAIL
• Last Result Wins

• • FAIL → PASS = PASS
  • PASS → FAIL = FAIL
• Best Result Wins

• • FAIL → PASS = PASS
• Flaky Detection

• • FAIL → PASS = FLAKY
  • PASS → FAIL → PASS = FLAKY
    User friction
    Customers using test retry mechanisms see a discrepancy between the results reported by their CI/testing tools and the final status shown in Xray.

In scenarios where a test initially fails and subsequently passes after a retry, Xray continues to report the Test Run as FAIL based on its aggregation logic. This makes it difficult to accurately represent retry outcomes, identify flaky tests, and align quality reporting across tools.

Steps to reproduce (right now) / Actual Result:

• Configure a test framework that supports retries (e.g., Gradle Test Retry).
• Execute a test that:
  • Fails on the first attempt.
  • Passes on a subsequent retry.
• Generate JUnit XML containing multiple results for the same test.
• Import the results into Xray.

Example:

Test A   
Attempt 1 -> FAIL
Attempt 2 -> PASS

Actual Results:

Fail for the above example case even though the latest execution result is PASS.

IMPACT

What would improve if solved:{}

• Better alignment between Xray and modern CI/CD pipelines.
• More accurate reporting of retry executions.
• Improved visibility of unstable/flaky tests.
• Greater flexibility for customers with different reporting requirements.
• Reduced need for result preprocessing before import.
  Impact on stakeholders:
  QA Teams
• Improved test analysis and failure triage.
• Better identification of unstable tests.

Engineering Teams

• Consistent reporting across CI platforms and Xray.

Management/Release Stakeholders

• More accurate quality and release metrics.
  Current workaround:
  Customers must:
• Preprocess execution results before importing them into Xray.
• Remove intermediate results and retain only the desired final result.
• Maintain custom fields or external reporting mechanisms to identify flaky tests.
  CONTEXT & EXAMPLES:

  Concrete Example

A Gradle execution with retries enabled produces:
 

Test Login
   Attempt 1 -> FAIL
   Attempt 2 -> PASS

 
 
Last Result Wins -> PASS
Flaky Detection  -> FLAKY
Concrete example:

Workaround risk: