Uploaded image for project: 'Xray for Jira Cloud'
  1. Xray for Jira Cloud
  2. XRAYCLOUD-387

As system, I support the TestNG format when importing execution results

    XporterXMLWordPrintable

Details

    • NOTRUN

    Description

      Xray must provide support for TestNG XML format when importing execution results.

      https://confluence.xpand-addons.com/display/XRAY/Taking+advantage+of+TestNG+XML+reports

      Xray should support NUnit v6.14.

      In TestNG, you have Test methods, Parameterized Test methods, Test classes, Test groups, suites.

      • A Test method is a test case;
      • Parameterized Tests are a way of specifying input values for a given Test (similar to an example in a Cucumber Scenario Outline). Parameters may come from the XML configuration file used by TestNG or from a data provider (i.e. a method that generates a set of values);
      • Tests code is implemented within Test classes;
      • Test groups are somehow similar to using Jira labels in Jira issues, in the way that they're used to mark the Test methods as belonging to some "category"; these groups can be used afterwards to more easily select the Test to be run;  
      • A suite is a configuration file (i.e. a XML file) that is used to enumerate all the Tests to be run, based on the Test classes, Test groups, packages, Test methods.

      TestNG uses "attributes" in order to ascribe behavior/characteristics to certain parts of your automated test code; these attributes can be set in the Test method's code.

       

      Test Cases are imported to Xray’s Generic Test issues. The “classname” and “methodname” attributes are concatenated and mapped to the Generic Test Definition field of the Generic Test.

      If a Test already exists with the same Generic Test Definition, then it is not created again.

      Test Cases are imported to a new (or a user-specified) Test Execution in the context of some project, along with their respective execution results.

      Parameterized Tests are imported to the same Test Run, where each set of parameters is identified by a different context.

      The context (shown on the left side of the execution screen details) is with the name of the "suite" plus the name of the "test" as defined in the XML configuration file used by TestNG. If the Test method corresponds to a parameterised test, then the values used for that specific run are appended to the context.

      Mapping of fields from the report to the Test issue

      TestNG XML report Test in Jira
      "methodname" attribute Summary field
      "classname" attribute + "." + "methodname" attribute Generic Test Definition custom field
      groups labels
      "labels" attribute under the <attributes> tag, within some <test-method>, containing one or more labels delimited by space labels
      "test" attribute under the <attributes> tag, within some <test-method>, containing a Jira key of the Test issue identification of Test issue key in Jira
      "requirement" attribute under the <attributes> tag, within some <test-method>, containing a Jira key of requirement(s) link to requirement(s)

      The status of the Test Run will be set based on the Test case result:

      If all the mapped results of the Test Case was PASS PASS
      If any of the mapped results of the Test Case was FAIL FAIL
      Other cases TODO

      Notes:

      • linked requirements may be identified by setting an attribute named "requirement" with the respective issue key at the Test method level, using the ITestResult object"; multiple requirements may be specified by using the space delimeter
      • the Test to report results against to may be identified by using and attribute named "test";
        • if the "Test" attribute is used explicitly, then the Test must exist or else it will not be imported.

       REST API endpoint: /api/v1/import/execution/testng

      PATH PARAMETERS
      parameter type description
      projectKey String key of the project where the Test Execution (if the testExecKey parameter wasn't provided) and the tests (if they aren't created yet) are going to be created.
      testExecKey String key of the Test Execution.
      testPlanKey String key of the Test Plan; if you specify the Test Plan, the Tests will be added automatically to the Test Plan if they're not part of it.
      testEnvironments String a string containing a list of test environments separated by ";"
      revision String source code and documentation version used in the test execution.
      fixVersion String the Fix Version associated with the test execution (it supports only one value).

       

      Attachments

        Issue Links

          Activity

            People

              jcso jcso
              bmpc Bruno Conde
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1 week
                  1w
                  Remaining:
                  Time Spent - 1 day Remaining Estimate - 4 days
                  4d
                  Logged:
                  Time Spent - 1 day Remaining Estimate - 4 days
                  1d