Uploaded image for project: 'Xray for Jira'
  1. Xray for Jira
  2. XRAY-1205

As a user, I can use a "Flackiness Score" gadget

    XporterXMLWordPrintable

Details

    • Suggestion
    • Status: New
    • Resolution: Unresolved
    • None
    • None
    • Report
    • UNCOVERED

    Description

      A new JIRA gadget "Flackiness Score" should provide a chart with the calculated flackiness score for the given test runs.

      Aim: This chart will provide the means to analyze what tests are most flacky/undeterministic and thus need to be reviewed.

      xx: testkey
      yy: calculated flackiness score
      order: *preferably *sorted in descending order of flackiness score

      Filters:

      • saved filter (for finding Test Executions)
      • testplan or project+version
      • test run's assignee, status, priority, component
      • test environment
      • test type

      The flackiness score should be calculated as follows.

      Flackiness Score = nr_of_changes / total_possible_changes
      nr_of_changes = amount of times a test changed from a final state to another final state through time
      total_possible_changes = total_test_runs_in_final_status - 1

      Chart Title: by default "Execution Burndown" but may be customized

      Note: it's expected flackiness to be evaluated against tests executed against the same code version.

      Attachments

        Issue Links

          Activity

            People

              helder.ferreira Helder Ferreira
              smsf Sergio Freire
              Votes:
              7 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: