Uploaded image for project: 'Xray for Jira Cloud'
  1. Xray for Jira Cloud
  2. XRAYCLOUD-10990

The Xray GraphQL API returns the latest test execution status across all executions of a test, but does not provide the Test Plan-specific "Latest Status" shown in the UI, leading to inconsistencies between API results and Test Plan reporting.

    XporterXMLWordPrintable

Details

    • Suggestion
    • Status: New
    • Resolution: Unresolved
    • None
    • None
    • GraphQL, Test Plan
    • None
    • UNCOVERED

    • 20

    Description

      Description

      When retrieving test execution results through the Xray GraphQL API, the returned test run status reflects the most recent execution of the Test across all associated Test Executions rather than the status displayed within the context of a specific Test Plan. As a result, the API response does not align with the "Latest Status" shown in the Test Plan UI, making it difficult to programmatically reproduce the same information displayed to users.

      User friction

      Users expect the API to return the same "Latest Status" value that is visible in the Test Plan screen. Instead, the API returns the latest execution status globally for the Test, which can differ from the Test Plan-specific status and lead to confusion, incorrect reporting, and additional development effort.

      Steps to reproduce (right now) / Actual Result:

      1. Create or open a Test Plan containing one or more Tests.
      2. Associate one Test Execution with the Test Plan and execute the Tests.
      3. Execute the same Test in other Test Executions that are not linked to the Test Plan.
      4. Query the Test and its testRuns using the GraphQL API.
      5. Observe that the latest test run returned belongs to the most recently executed Test Execution, regardless of whether it is associated with the Test Plan.

      Actual Result:
      The API returns the latest Test Run globally for the Test.

      Expected Result:
      The API should provide a way to retrieve the "Latest Status" as calculated within the context of the specified Test Plan, matching the value displayed in the Test Plan UI.

      IMPACT

      Automated reporting, dashboarding, integrations, and custom applications cannot reliably reproduce the status shown in the Test Plan user interface. This creates discrepancies between API-driven reports and what stakeholders see in Xray.

      What would improve if solved:

      Providing a Test Plan aware status field or GraphQL query capability would ensure consistency between UI and API results, simplify integrations, reduce custom logic, and improve confidence in automated reporting.

      Impact on stakeholders:

       
      Inconsistent API and UI results create reporting discrepancies, increase integration complexity, and reduce confidence in Test Plan status accuracy.

      Current workaround:

      Retrieve the Test Plan, identify all linked Test Executions, manually filter the Test Runs belonging only to those executions, and calculate the effective status outside of Xray.

      CONTEXT & EXAMPLES:

      The Test Plan UI displays a status that appears to be derived from Test Executions associated with that specific Test Plan. However, the GraphQL API exposes all test runs for a Test, requiring consumers to determine which executions belong to the Test Plan and calculate the result independently.

      Concrete example:

      A Test shows:

      • Test Execution A (linked to the Test Plan): FAILED
      • Test Execution B (not linked to the Test Plan): PASSED and executed more recently

      The Test Plan UI displays FAILED because it considers only the execution associated with the Test Plan.

      The GraphQL API returns PASSED because it retrieves the most recent test run overall.

      This discrepancy makes it impossible to directly obtain the UI's "Latest Status" value through a straightforward GraphQL query.

      Workaround risk:

      The workaround requires custom filtering

      Attachments

        Activity

          People

            bernardo.cottim Bernardo Cottim
            harish.deshaboina Harish Deshaboina
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: