Details
-
Story
-
Status: Closed
-
Major
-
Resolution: Done
-
None
-
None
-
R1.006.000 S4
-
Description
As system, tests and results can be imported by REST API and from Test Execution issue screen using xUnit.net XML v2 reports
Example of xUnit.net v2 XML
<?xml version="1.0" encoding="utf-8"?> <assemblies timestamp="07/31/2018 14:58:48"> <assembly name="C:\Users\smsf\Documents\Visual Studio 2015\Projects\xUnitDemo\xUnitDemo\bin\Debug\xUnitDemo.DLL" environment="64-bit .NET 4.0.30319.42000 [collection-per-class, parallel (1 threads)]" test-framework="xUnit.net 2.3.1.3858" run-date="2018-07-31" run-time="14:58:47" config-file="C:\Users\smsf\Documents\Visual Studio 2015\Projects\xUnitDemo\packages\xunit.runner.console.2.4.0\tools\net452\xunit.console.exe.Config" total="15" passed="14" failed="1" skipped="0" time="0.257" errors="0"> <errors /> <collection total="2" passed="1" failed="1" skipped="0" name="Test collection for xUnitDemo.SimpleTests" time="0.070"> <test name="xUnitDemo.SimpleTests.PassingTest" type="xUnitDemo.SimpleTests" method="PassingTest" time="0.0636741" result="Pass"> <traits> <trait name="test" value="CALC-1" /> <trait name="requirement" value="CALC-1" /> <trait name="labels" value="core UI" /> </traits> </test> <test name="xUnitDemo.SimpleTests.FailingTest" type="xUnitDemo.SimpleTests" method="FailingTest" time="0.0059474" result="Fail"> <failure exception-type="Xunit.Sdk.EqualException"> <message><![CDATA[Assert.Equal() Failure\r\nExpected: 5\r\nActual: 4]]></message> <stack-trace><![CDATA[ at xUnitDemo.SimpleTests.FailingTest() in C:\Users\smsf\documents\visual studio 2015\Projects\xUnitDemo\xUnitDemo\SimpleTests.cs:line 30]]></stack-trace> </failure> </test> </collection> <collection total="13" passed="13" failed="0" skipped="0" name="Test collection for xUnitDemo.CalculatorTests" time="0.001"> <test name="xUnitDemo.CalculatorTests.CanAddTheoryClassData(value1: 1, value2: 2, expected: 3)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryClassData" time="0.0002722" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryClassData(value1: -4, value2: -6, expected: -10)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryClassData" time="0.0000248" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryClassData(value1: -2, value2: 2, expected: 0)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryClassData" time="0.0000028" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryClassData(value1: -2147483648, value2: -1, expected: 2147483647)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryClassData" time="0.0000017" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryMemberDataMethod(value1: 1, value2: 2, expected: 3)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryMemberDataMethod" time="0.0001179" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryMemberDataMethod(value1: -4, value2: -6, expected: -10)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryMemberDataMethod" time="0.0000054" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheoryMemberDataMethod(value1: -2, value2: 2, expected: 0)" type="xUnitDemo.CalculatorTests" method="CanAddTheoryMemberDataMethod" time="0.0000024" result="Pass" /> <test name="xUnitDemo.CalculatorTests.PassingTest" type="xUnitDemo.CalculatorTests" method="PassingTest" time="0.0000952" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheory(value1: 1, value2: 2, expected: 3)" type="xUnitDemo.CalculatorTests" method="CanAddTheory" time="0.0001095" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheory(value1: -4, value2: -6, expected: -10)" type="xUnitDemo.CalculatorTests" method="CanAddTheory" time="0.0000083" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheory(value1: -2, value2: 2, expected: 0)" type="xUnitDemo.CalculatorTests" method="CanAddTheory" time="0.000002" result="Pass" /> <test name="xUnitDemo.CalculatorTests.CanAddTheory(value1: -2147483648, value2: -1, expected: 2147483647)" type="xUnitDemo.CalculatorTests" method="CanAddTheory" time="0.0000017" result="Pass" /> <test name="xUnitDemo.CalculatorTests.FailingTest" type="xUnitDemo.CalculatorTests" method="FailingTest" time="0.0000915" result="Pass" /> </collection> </assembly> </assemblies>
Mapping of fields from the report to the Test issue:
| xUnit.net XML report | Test in Jira |
|---|---|
| "method" attribute | Summary field |
| "type" attribute + "." + "method" attribute | Generic Test Definition custom field |
| "value" attribute of the "trait" whose name is "labels" under the <traits> tag, within some <test>, containing one or more labels delimited by space | labels |
| "value" attribute of the "trait" whose name is "test" under the <traits> tag, within some <test>, containing a Jira key of the Test issue | identification of Test issue key in Jira |
| "value" attribute of the "trait" whose name is "requirement" under the <traits> tag, within some <test>, containing one Jira key of the requirement/coverable issue | link to requirement |
Mapping results to the Test Run
- if the "name" attribute of the "test" item is something like "xxxxx(val1: .., val2: ...., valN: ...)" as seen below
xUnitDemo.CalculatorTests.CanAddTheory(value1: -2, value2: 2, expected: 0)
then the test is parameterized, having the values (-2,2,0) for the variables "value1", "value2", "expected" (for the above example)
-
- a new Test Run Result entry should be created for each similar "test" item (i.e. having a similar type+method attributes). The "Context" column should have the parameters in the form "(val1: .., val2: ...., valN: ...)"
- The value of the "message" item, under "failure", should be concatenated with the "stack-trace" item value.
- the value of the "time" attribute, which comes in seconds, should be mapped to the Duration which is presented in miliseconds
Status
The status of the Test Run will be set based on the Test case result:
| "result" attribute on "test" item | Test Run status |
|---|---|
| Fail | FAIL |
| Pass | PASS |
| Skip | TODO |
References:
