TestRail

Guide on the Ketryx integration with TestRail

1. Introduction

Ketryx provides seamless integration with TestRail, enhancing compliance and streamlining the management of test cases and requirements. This integration is purpose-built to

• align your TestRail workflows with industry standards such as ISO 13485, IEC 62304, and ISO 14971,

• synchronize data between Ketryx and TestRail to maintain consistent electronic records,

• provide traceability from requirements to test cases and test results, and

• enforce workflows that incorporate electronic approvals and audit trails for robust compliance.

This guide offers step-by-step instructions for setting up and utilizing the integration, and answers to frequently asked questions.

2. Setting up the integration

To integrate Ketryx with TestRail, please ensure you have administrative permissions on both platforms. With the required permissions in place, proceed with the following steps:

• Note: the initial connection should be made from a service account (e.g. [e-mail]), which should have TestRail administrator privileges. It should NOT be someone's personal account, even if that person is an administrator.

• Rationale: Whoever sets up the initial connection to Ketryx via the API will be credited with making the changes that Ketryx makes in TestRail. The biggest problem this creates is the danger that once the original person who sets up Ketryx leaves the company, typically, that person's IT credentials are revoked. In turn, this would revoke Ketryx's access to TestRail and cause the Ketryx/TestRail integration to stop working. (No data would be lost, but this is very annoying and potentially difficult to troubleshoot.)

1. Log into TestRail.

2. Make sure Enable API is active on the Administration->API page.

3. Navigate to your settings and API KEYS.

4. Create a new API KEY.

5. Log in to Ketryx.

6. In Ketryx, navigate to the organization connections page click Add TestRail connection.

7. Provide the required TestRail credentials (e.g., API key, instance URL and email) to establish the connection.

8. Confirm the integration in Ketryx and map TestRail projects to corresponding Ketryx projects.

After setting up the connection, test cases and results in TestRail can be seamlessly linked to requirements and other items within Ketryx.

3. Frequently Asked Questions

3.1. Q: How does Ketryx synchronize TestRail data?

Ketryx synchronizes the following data between the platforms:

• Test Cases: Test cases in TestRail are imported into Ketryx as Test Cases (TEST_PROTOCOL) items.

• Test Executions: Test executions in Ketryx are generated based on the Run + Result combination in TestRail, creating an item of type TEST_EXECUTION in Ketryx.

• Test Plans: Test plans in TestRail are imported into Ketryx as Test Plan (TEST_PLAN) items. These items incorporate all the relationships established in TestRail by the runs and can be manually managed within Ketryx. Alternatively, they can be automatically synchronized if the setting Automatically update release test plan based on external test plan(s) is enabled.

3.2. Q: Can I customize how Ketryx maps TestRail fields?

Yes. Ketryx provides advanced settings for TestRail that enable you to define how TestRail fields map to Ketryx items. For instance, you can configure TestRail’s Template or Result fields to specific Ketryx attributes.

3.3. Q: What happens to deleted test cases or results in TestRail?

When a test case or result is deleted in TestRail:

• It is marked as deleted in Ketryx but remains linked for traceability.

• No further updates are synchronized unless the test case or result is reactivated in the case of soft deletion.

3.4. Q: How do I handle changes in TestRail schema (e.g., custom fields)?

If you add custom fields in TestRail, you can update the Ketryx configuration to recognize them using the TestRail field type mapping setting.

3.5. Q: How do I handle versions?

There are multiple ways to handle versioning between Ketryx and TestRail. These options are prioritized in descending order:

1. Custom Field Mapping: Create a custom field in TestRail and map it to introducedInVersion in Ketryx.

2. Version Field: Use the default version field in TestRail results.

3. Parent Milestone: If the milestone name in TestRail matches the Version number pattern, it will be used as the version.
4. Plan Name: If all the above options fail, Ketryx will attempt to use the plan name, matching it with the Version Number Pattern.

3.6. Q: Which types of TestRail projects are supported?

Ketryx supports the following types of TestRail projects:

• Single Repository for All Cases (Recommended): This is fully supported without additional configuration and offers seamless synchronization.

• Single Repository with Baseline Support: Supported with custom configuration to map baselines effectively to Ketryx structures.

These options ensure compatibility and flexibility when integrating different TestRail project setups with Ketryx.

4. TestRail Configuration

4.1. Overview

The TestRail settings in Ketryx enable users to configure how TestRail-specific fields map to Ketryx items. This mapping ensures accurate data synchronization between the two systems, maintaining traceability and compliance. These configurations are crucial for integrating test cases, test executions, and test plans into Ketryx workflows seamlessly.

The configuration includes four primary mapping types:

1. TestRail field type mapping: Matches TestRail fields to Ketryx attributes for test cases, test executions, and test plans.

2. TestRail status mapping: Aligns TestRail statuses with Ketryx item states.

3. TestRail result mapping: Converts TestRail test result statuses to Ketryx result states.

4. TestRail work item relations mapping: Maps TestRail fields to create relational links in Ketryx with items in external systems.

4.2. TestRail field type mapping

The TestRail field type mapping defines how TestRail fields (e.g., custom fields) are translated into Ketryx attributes for various entities such as test cases, test executions, and test plans.

All custom fields from TestRail must be prefixed with custom_ in the configuration.

Example configuration

{
  "TEST_PROTOCOL": {
    "DEFAULT": {
      "refs": "refs",
      "description": "custom_description",
      "introducedInVersion": "custom_prd_ver",
      "steps": "custom_steps_separated",
      "expectedBehavior": "custom_expected"
    },
    "My Custom Template": {
      "refs": "refs",
      "description": "custom_description_for_steps",
      "introducedInVersion": "custom_version",
      "steps": "custom_steps",
      "expectedBehavior": "custom_expected_behavior"
    }
  },
  "TEST_EXECUTION": {
    "description": "custom_execution_desc",
    "introducedInVersion": "custom_exec_prd_ver",
    "steps": "custom_step_results",
    "expectedBehavior": "custom_expected_exec"
  }
}

• TEST_PROTOCOL: Maps fields for test case templates and default configurations.

• TEST_EXECUTION: Maps fields for test execution data.

Refer to Fields for a complete list of built-in fields available in Ketryx.

4.3. Mapping Fields to Extra field names

It is possible to map fields to Extra field names. To do this, assign a label key in the TestRail field type mapping and use the same key in the Extra field names setting.

Example:

TestRail field type mapping:

{  
  // ...
  "TEST_EXECUTION": {
    "description": "custom_execution_desc",
    "introducedInVersion": "custom_exec_prd_ver",
    "steps": "custom_step_results",
    "expectedBehavior": "custom_expected_exec",
    "My custom field to be tracked": "custom_tr_field"
  }
}

Extra field names:

["My custom field to be tracked"]

4.4. TestRail Status Mapping

The TestRail status mapping aligns TestRail status names with Ketryx item states such as OPEN, REOPENED, IN_PROGRESS, RESOLVED, or CLOSED.

Example configuration

{
  "Backlog": "OPEN",
  "Done": "RESOLVED"
}

• Backlog in TestRail maps to Open in Ketryx.

• Done in TestRail maps to Resolved in Ketryx.

Refer to Status mapping for a complete list of Ketryx statuses.

4.5. TestRail Result Mapping

The TestRail result mapping converts TestRail test result statuses to Ketryx result states like PASSED, FAILED, or PASS_WITH_EXCEPTION.

Example configuration

{
  "passed": "PASS",
  "failed": "FAIL"
}

Passed in TestRail maps to Pass in Ketryx. Failed in TestRail maps to Fail in Ketryx.

Refer to testResult for a complete list of Ketryx result states.

4.6. TestRail work item relations mapping

The TestRail work item relations mapping links TestRail fields with items in external systems (e.g., Jira, Git, ...). This ensures test cases and executions are correctly related to development or work item-tracking items.

Example configuration

[
  {
    "sourceField": "my_custom_field",
    "relationType": "AFFECTS",
    "targetSystem": "Jira"
  }
]

• sourceField: The field in TestRail to map.

• relationType: The type of relation in Ketryx (e.g., AFFECTS, RELATES_TO).

• targetSystem: The external system where the target item resides (e.g., Jira, TestRail, Git).

Refer to relationType for a complete list of Ketryx relation types.

---

For more detailed instructions and advanced configuration options, please refer to the Ketryx Integration Settings section of the documentation.