XCUITest integration with TestRail

Question

Currently working on integrating my UITest-run results into TestRail, so after every test run it marks my tests as Pass\Fail in testrail.

The idea I have is:

1. In CI create a "pre-build" script that will create a test run in testrail.
2. During execution of automation, in test tearDown() get the result of a test(if a test failed or not), save it all into json file. - here is the first question, how can I get if a test failed?
3. Once all tests are done, run a "post-build" script to get updated json file and send request to test rail(which will mark pass\fail tests)

Anyone who already worked on this, does it sound right for you? Any advice?

example of the test:

import XCTest

class MyUITests: XCTestCase {

    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        appEntry.app.launch()
        dismissSystemAlerts()
    }

    override func tearDown() {
        super.tearDown()
    }

    func test_Elements() {
        // MARK: Sample test
        // my test actions are here
    }
}

Answer 1

This is how I implemented. First I have pre-build script in my CI that will create new Test Run in TestRail. And then UITestObserver will send APIs to TR to update statuses.

new class that I've added:

import Foundation
import XCTest

class UITestObserver: NSObject, XCTestObservation {
    // Handle Test Case Failure
    public func testCase(_ testCase: XCTestCase,
                         didFailWithDescription description: String,
                         inFile filePath: String?,
                         atLine lineNumber: Int) {
        var testCaseId: [String] = []
        if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
        if testCaseId != ["NA"] {
            postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: description)
        }
    }

    // Handle Test Case Pass
    public func testCaseDidFinish(_ testCase: XCTestCase) {
        let testRun = testCase.testRun!
        let verb = testRun.hasSucceeded
        var testCaseId: [String] = []
        if let MyTestCaseID = testCase as? BaseTest { testCaseId = MyTestCaseID.inegrateTestRailId() }
        if verb == true && testCaseId != ["NA"] {
            postTestRailAddResultAPI(for: testCase, testCaseId: testCaseId, description: "PASS")
    }
}

In BaseTest setUp added this line:

XCTestObservationCenter.shared.addTestObserver(UITestObserver())

And implemented function postTestRailAddResultAPI that sends actual requests to update status. All my tests now have testCaseId which stores value of TestRail TestCase number, and this is how it knows which TCs to update.

Answer 2

This is what I do,

1. Run tests using scan plugin of fastlane
2. Generate a junit format result
3. Parse the xml and get results of each test
4. Post it to our test case management tool

Kindly share your solution with all of us.