I am working on Selenium test Automation. I create my Selenium test-suite to automate my test suite. Now i want to integrate the Selenium results with the TestRail. I am not sure how to integrate the results of the Selenium test runs to TestRail Test suite. I wrote all test cases in java. I am stuck now. It would be helpful to me with an example.
I am using testng framework, Maven build tool.
The basic idea is that you need to be able to link your results back to the unique Test ID in TestRail, within the context of a given User. This can be done either as each test is executed and passes / fails, or after the entire run is complete.
If you want to push the results to TestRail after each test passes / fails, you would either create a TestNG listener which will listen for test results and then call the API to submit the result to TestRail. This approach is much cleaner than adding a function to each test.
If you want to push the results to TestRail after the run is completed, you may have to write a parser to read / process the entire results file and then call the TestRail APIs appropriately.
In terms of the APIs you need to call, you can either use the API methods "add_result" or "add_result_for_case" to do this. The key difference between the two methods is that "add_result_for_case" takes the Case ID and the Run ID, whereas "add_result" takes the Test ID. Either can be useful depending on your automation approach.
There is a Java API binding available at:
https://github.com/gurock/testrail-api
This is documented here.
You instantiate the API connection in Java via:
import com.gurock.testrail.APIClient;
import com.gurock.testrail.APIException;
import java.util.Map;
import java.util.HashMap;
import org.json.simple.JSONObject;
public class Program
{
public static void main(String[] args) throws Exception
{
APIClient client = new APIClient("http://<server>/testrail/");
client.setUser("..");
client.setPassword("..");
}
}
Here's an example of a GET request:
APIClient client = new APIClient("http://<server>/testrail/");
client.setUser("..");
client.setPassword("..");
JSONObject c = (JSONObject) client.sendGet("get_case/1");
System.out.println(c.get("title"));
And here's a POST request:
Map data = new HashMap();
data.put("status_id", new Integer(1));
data.put("comment", "This test worked fine!");
JSONObject r = (JSONObject) client.sendPost("add_result_for_case/1/1", data);
This should work. We have test infrastructure of Jenkins running with Maven Styled Project with TestNg as the Test Framework and Java as the scripting language. Once this is deployed on Jenkins, it should be PARAMETERIZED JOB with parameters i.e.PROJECT_ID [mandatory], MILESTONE_ID [optional] as per TestRail's API.
***********PageBase Class [Generic methods]***********
public static int TEST_RUN_ID;
public static String TESTRAIL_USERNAME = "atul.xxma@xxx.com";
public static String TESTRAIL_PASSWORD = "oXafTubi/wsM7KZhih73-ZZ38v";
public static String RAILS_ENGINE_URL = "https://xxxx.testrail.io/";
public static final int TEST_CASE_PASSED_STATUS = 1;
public static final int TEST_CASE_FAILED_STATUS = 5;
public static APIClient client = new APIClient(RAILS_ENGINE_URL);
@BeforeSuite() //TestRail API
public void connectAndCreateRunTestRail() throws MalformedURLException, IOException,
APIException {
System.out.println("before suite: connectAndCreateRunTestRail");
String project_id = System.getProperty("PROJECT_ID"); // jenkins parameter
String milestone_id = System.getProperty("MILESTONE_ID"); // jenkins parameter
client.setUser(TESTRAIL_USERNAME);
client.setPassword(TESTRAIL_PASSWORD);
Map data = new HashMap();
data.put("suite_id", 1); // default
data.put("name", "Test Run:");
data.put("description", "Desc:XXXE");
data.put("milestone_id", milestone_id);
data.put("assignedto_id", 6); // User ID
data.put("include_all", false); // set to false as need to select required TCs.
int[] arrCaseIds = { 45, 93, 94, 97, 96, 99, 174 };
List<Object> lstCaseIds =
Arrays.stream(arrCaseIds).boxed().collect(Collectors.toList());
data.put("case_ids", lstCaseIds);
data.put("refs", "Ref:Regression Suite");
System.out.println(data.toString());
// Post with URL and payload in Map format.
Object response_Post_addRun = client.sendPost("add_run/" + project_id, data);
System.out.println("Response of response_Post-AddRun-->" +
response_Post_addRun.toString());
JSONObject obj = new JSONObject(response_Post_addRun.toString());
int run_Id = obj.getInt("id");
TEST_RUN_ID = run_Id;
System.out.println("Added Run ID -->" + run_Id);
}
public static void addResultForTestCase(String testCaseId, int status, String
screenshotPath) throws IOException, APIException, ParseException {
client.setUser(TESTRAIL_USERNAME);
client.setPassword(TESTRAIL_PASSWORD);
HashMap data = new HashMap();
data.put("status_id", status);
data.put("comment", "Executed Selenium TestFramework.");
Object response_Post_add_result_for_case = client
.sendPost("add_result_for_case/" + TEST_RUN_ID + "/" + testCaseId + "",
data);
System.out.println("response_Post-->" +
response_Post_add_result_for_case.toString());
JSONObject obj = new JSONObject(response_Post_add_result_for_case.toString());
int result_id = obj.getInt("id");
System.out.println("result_id->" + result_id);
System.out.println("screenshotPath-->" + screenshotPath);
client.sendPost("add_attachment_to_result/" + result_id, screenshotPath);
}
***********Reporting***********
@Override
public void onTestSuccess(ITestResult tr) {
System.out.println("onTestSuccess");
String screenshotPath = ".//" + screenshotName + ".png";
File screenshotTRPath = new File(System.getProperty("user.dir") + "/Reports/" +
screenshotName + ".png");
System.out.println("screenshotPath-->" + screenshotPath);
System.out.println("screenshotTRPath-->" + screenshotTRPath.toString());
///TestRail API
try {
addResultForTestCase(currentTestCaseId, TEST_CASE_PASSED_STATUS,
screenshotTRPath.toString());
} catch (IOException | APIException | ParseException e) { e.printStackTrace();
}
}
