Details
-
Suggestion
-
Status: New
-
Resolution: Unresolved
-
None
-
None
Description
A new JIRA gadget "Flackiness Score" should provide a chart with the calculated flackiness score for the given test runs.
Aim: This chart will provide the means to analyze what tests are most flacky/undeterministic and thus need to be reviewed.
xx: testkey
yy: calculated flackiness score
order: *preferably *sorted in descending order of flackiness score
Filters:
- saved filter (for finding Test Executions)
- testplan or project+version
- test run's assignee, status, priority, component
- test environment
- test type
The flackiness score should be calculated as follows.
Flackiness Score = nr_of_changes / total_possible_changes
nr_of_changes = amount of times a test changed from a final state to another final state through time
total_possible_changes = total_test_runs_in_final_status - 1
Chart Title: by default "Execution Burndown" but may be customized
Note: it's expected flackiness to be evaluated against tests executed against the same code version.
Attachments
Issue Links
- mentioned in
-
Page Loading...