MAN-07 Traceability
Manual for the Ketryx traceability matrix and its traceability mechanism
1. Introduction
This manual describes the setup and processes for interpreting, maintaining and verifying the traceability matrix of a Ketryx project.
1.1. Terms and definitions
RTM: Requirements Traceability Matrix.
Specification: A Ketryx specification item, namely Software Item- and Hardware Item Specifications. A specification fulfills a Requirement with relevant details for the concrete implementation
Requirement: A Ketryx requirement item. A Requirement may have a requirement type assigned to make them more specific, such as Use case requirement types
Use case / Use case Requirement: A Ketryx requirement item with requirement type assigned to "Use case".
Test Case: A Ketryx Test Case item. Depending on its relations to other requirements and specifications, a Test Case may be represented as a verification or validation test
Test Execution: Either a Ketryx Test Execution or a Jira Xray Test Execution item. A test execution holds a test result for a given Test Case — pass, pass (with exception), fail, fail (accepted)
Risk: A Ketryx risk item. A risk may be controlled by a Requirement, Specification or Test Case item
Verification Test: A Ketryx Test Case item that tests at least one Specification item. A verification test may also be a validation test at the same time
Validation Test: A Ketryx Test Case item that tests at least one Requirement. A validation test may also be a verification test
Fulfilled / failed traceability: A traceability matrix and its corresponding rows may be considered fulfilled if all the traceability checks are successful for the given set of items. Vice-versa, it is considered failed if any of the traceability constraints have been violated
Traceability error: A traceability error represents a violation of the traceability constraints according to the traceability mode and is commonly presented in red
Traceability warning: A traceability warning represents a less severe violation of the traceability constraints, such as a missing item approval, or an uncontrolled risk related to a traced item
Traceability mode: The algorithm used to determine the fulfillment state of a given traceability matrix. The traceability mode also has an impact on what columns are displayed
Traceability row: A row in the calculated RTM based on a certain primary item, and its traces to requirements, use-cases, specifications, verification and validation tests.
Traceability check enforcement (also called "full traceabilty"): An enforced traceability check means that a given traceability matrix needs to be fulfilled for a final approval
Design input: A Requirement item that leads to the creation of a design output
Design output: A Specification item that results from a design input
Risk: A Risk item that represents a particular risk in the system
Risk control: An item for mitigating a potential risk. Risk controls are marked with a "RC" short name.
2. Traceability Modes
The traceability mode dictates certain traceability behavior and rulesets in the traceability matrix that are derived from the Ketryx project schema configuration.
2.1. System-Requirement and Specification Mode
Enabled with Ketryx project schema: Software design and development, Software and hardware design and development, Full schema including Post-Market Surveillance
The System-Requirement and Specification mode is the strictest mode available in Ketryx. The following constraints need to be satisfied for the traceability matrix to be considered fulfilled:
Each detected Use case Requirement requires at least one related Requirement (either another Use case or any other Requirement type)
Each Requirement requires at least one Software Item or Hardware Item Specification, and a Test Case (= validation test)
Each Specification requires at least one Test Case (= verification test)
Each Test Case requires at least one manual and/or automated Test Execution. Depending on the project configuration, a manual Test Execution may be mandatory
Each Test Case needs to evaluate to the following test result (based on its manual/automated Test Executions): Passed, Passed with Exception, Failed (accepted)
All Use cases, Requirements, Specifications, Test Cases, Test Executions and related Risks need to be in a controlled state
2.2. Requirement-Validation-Only Mode
Enabled with Ketryx project schema: Requirement validation
The Requirement-Validation-Only mode was designed for (Software) systems with a primary focus on tracing Requirements to corresponding Test Cases and their subsequent Test Executions. Software/Hardware Item Specifications may be part of the requirements and specification architecture, but are not considered in the traceability check of the resulting traceability matrix.
The following constraints need to be satisfied for the traceability matrix to be considered fulfilled:
Each detected Use case Requirement requires at least one related Requirement (either another Use case or any other Requirement type)
Each requirement requires at least one Test Case (= validation test)
Each Test Case requires at least one manual and/or automated Test Execution. Depending on the project configuration, a manual Test Execution may be mandatory
Each Test Case needs to evaluate to the following test result (based on its manual/automated Test Executions): Passed, Passed with Exception, Failed (accepted)
All Use cases, Requirements, Specifications, Test Cases, Test Executions and related Risks need to be in a controlled state
3. Traceability Page
The Traceability (RTM) page can be accessed via the Traceability navigation item within the project sidebar and will display the RTM for a given version.
The page displays a paginated set of all traceability rows for a given primary item (Use cases, Design inputs, Design outputs, Verification, Validation, etc). Primary items represent a distinct item in the system that are traced to its corresponding relations.
3.1. Test Case Categorization
A Test Case may test a Requirement and/or a Specification item. Depending on the tested item, a Test Case may be qualified as a Verification test (testing a Specification), Validation test (testing a Requirement/Use Case), or both (testing a Specification and Requirement/Use Case).
Important: Due to legacy reasons, in Jira, a Test Case item will allow members to set a test type. Please note that the traceability page will ignore this Jira field and will only consider the item's trace relations when categorizing into Verification and Validation tests. In future versions of Ketryx, the test type field will not distinct between Verification/Validation tests but rather focus on the actual test kind (Regression, Unit, Integration, System, etc).
3.2. Excluded Test Cases
A Test Case may be excluded from a test plan within Ketryx' test management page. A Test Case not included or explicitly excluded from a version's test plan will be marked as "Not included in test plan" in the traceability page. It doesn't require any Test Execution, nor a Test result, and doesn't need to be controlled for the RTM to be fulfilled.
3.3. Automated Tests
Test results may be tracked through automated CI builds, and may be testing Requirements, Software/Hardware Item Specifications, or Test Cases. If the automated test is related to a Test Case, it will be displayed within the Test Case's cell's "Automated tests" list.
If the automated test is testing a Requirement or Specification item, the test will be displayed as a standalone cell marked with an AT abbreviation. This separation is important — automated tests linked to a Test Case cannot be explicitly approved until an equivalent manual Test Execution has been created. If the Ketryx project is configured to enforce manual tests on automated tests, the Traceability page will notify the user that a manual test execution is missing for a given Test Case.
3.4. Showing item traces
A traceability row may contain multiple items that represent different traces within the different columns. A member may highlight individual traces for a given item by clicking the Tracing button. It will highlight all items that are directly related to the selected item.
Note: An item may have an indirect relation to a Use case via a parent of a parent relationship. In this case, the tracing function will only highlight Use cases that are directly related to the selected item.
3.5. Using the Traceability Check filters
Ketryx offers pre-defined traceability check metrics to allow members to quickly solve RTM errors and get to a releasable state.
Selecting a traceability check will activate the corresponding filter, showing all the rows that contain a cell matching the filter condition.
The following traceability checks are available (depending on the traceability mode):
Use cases covered by a design input: Filters for all rows containing at least one Use case that is not covered by a design input (Requirement)
Design input covered by a design output: Filters for all rows containing at least one design input (Requirement) without a proper relation to a design input (Specification). This check will not be available in the Requirement-only mode
Verification, design outputs covered by Tests: Filters for all rows containing design inputs (Specification) without a Test Case relation
Validation, design inputs covered by Tests: Filters for all rows containing design outputs (Requirement) without a Test Case relation
Test Executions created within test plan: Filters for all rows containing Test Cases missing a Test Execution or a corresponding test result
Failing tests: Filters for all rows containing Test Cases with failing Test Executions
3.6. Fixing Traceability Errors
While verifying a version's RTM, errors will occur that require to be fixed for the RTM to be approvable for a release.
The following list gives a list of errors and potential fixes:
Requirement missing: The erroneous Use case has no child Requirement. Fix: Create or identify the relevant Requirement and add the erroneous Use case to its "Parent requirements" field
Specification missing: The erroneous design input (Requirement) has no fulfilling Specification item defined. Fix: Create or identify the relevant Software/Hardware Item Specification and set its Fulfilled requirements field to the erroneous Requirement
Validation test case missing: The erroneous design input (Requirement) is not tested by any Test Case. Fix: Create or identify an existing Test Case and add the erroneous Requirement to its Tested items field
Verification test case missing: The erroneous design output (Specification) is not tested by any Test Case. Fix: Create or identify an existing Test Case and add the erroneous Specification to its Tested items field
Test execution missing: The erroneous Test Case is missing a Test Execution. Fix: Create a Test Execution item with its Test being executed field set to the erroneous Test Case. In case Jira Xray is being used, create a corresponding Test Execution for the erroneous Test Case via the Xray UI.
Manual test execution missing: An automated test doesn't have a corresponding manual test execution created. Fix: In the Test management page, find the automated test, select the item and create a Test Execution for the selection. This will create a dedicated manual Test Execution based on the result of the automated Test Execution, so it is ready to be approved
Test result missing: The erroneous Test Execution is missing a test result. Fix: Open the erroneous Test Execution item in Jira and set the Test result field to the appropriate value.
Test execution failed: The erroneous Test Execution has failed. Fix: Make sure the Test Execution has a Test result value set to either Passed, Passed with Exception, or in case the relevant configuration has been enabled, approve the Test Execution with all relevant approval steps to mark it as Failed (accepted)
Approval missing: The erroneous item is not fully controlled and requires approvals from the relevant parties. Fix: Approve the erroneous item with all relevant approval steps (if configured, including the owner)
3.7. Dangling items
The traceability matrix makes sure that all item relation constraints are fulfilled, but based on the selected primary column, there may be independent items that may not correspond to the primary item type (so-called "dangling items").
Dangling items are listed in separate rows at the first page of the Traceability table (one row per column type), with the primary column being empty.
Please note that the dangling item rows will also contain its direct item relations, e.g. Verification Test Cases or other entities like directly linked Use cases.
Note: Depending on the selected primary item it is very likely dangling items will be displayed. As long as they don't yield any traceability error, they are considered safe from a traceability fulfillment perspective.
3.8. Approving the Traceability matrix on the Traceability page
Note: This functionality needs to be enabled via the Advanced Settings (Traceability configuration).
The Traceability page allows users belonging to any steps of the Release approval type to approve a fulfilled traceability matrix for a selected version via a dedicated status message. When approving the traceability matrix, the given version's Traceability Matrix release document will be (re-)generated and approved.
4. Requirements Tree Page
All Requirements and Software/Hardware items may be shown in hierarchical order on the Traceability >> Requirements tree page.
This page allows members to review the Requirement architecture from the highest level (Use Case / Intended Use) to the lowest level (System Requirements and related Software / Hardware Items).
Each row will show the following information:
Title of the Requirement/Software Item/Hardware item
Jira link
Specification relations (for Requirements)
Risk relations for introduced risks
Test case relations (including the test execution state for a given version)
The aggregated status for a given row's sub-rows
Note: When the project scheme is set to Requirement validation, the page will not show software / hardware items as separate rows, and will not enforce specifications to be fully controlled. Specifications may still be shown in the Specifications column for each Requirement.
4.1. Reviewing the Requirements Tree Page
Similarly to the Traceability page, the requirements tree page gives an overview of the overall validation / verification test statuses of a version's requirements and specifications, including the state of pending test executions and excluded tests.
A test status may be marked as missing or failed and requires further action according to the procedures defined in WI-04 Test Case and WI-05 Test Execution to allow a transition into a fulfilled state (denoted as all passed).
When clicking on a Tests pill, a popup will show all the relevant Test Cases and their latest corresponding Test Execution (including its approval and test result state).
5. Download the Traceability Matrix Document
The Traceability Matrix Document can either be generated and downloaded as a release document (as described in MAN-02 Software Release Process, Section 3.9), or generated on demand for a selected version on the Traceability and Requirement tree page. The latter document can be generated either as a CSV or Excel file and will contain the following data:
Use case id: Unique identifier for a Use case related to the given Requirement
Use case title: The title of the Use case related to the given Requirement
Requirement ID: Unique identifier of the Requirement that's being traced
Requirement title: The title of the Requirement that's being traced
Requirement type: The requirement type of the Requirement that's being traced (e.g. System, Technical, Safety, etc.)
Requirement URL: A permanent url to the given Requirement record on Ketryx
Note: The generated document aims to replicate the Traceability page content with primary column set to Design input as closely as possible as the format permits. If a Requirement has relations to multiple Use cases, there will be multiple rows of the same Requirement for each Use case.
6. Configuration
Ketryx allows the following customization for the Traceability feature and behavior
6.1. Enable/Disable Traceability check enforcement
Requires manage project permissions.
For the desired project, navigate to the Settings page
Under Release controls, enable or disable Require full traceability (enabled by default)
If full traceabily has been disabled, the version release page will not block a release if the traceability matrix hasn't been fulfilled.
Important: Please be aware that this setting will allow members to approve a release artifact that may be not be compliant with current regulations. It was designed to be an escape hatch, in case there's a special need to finalize an incomplete RTM for important reasons.
6.2. Customize the Traceability Matrix document
The Traceability Matrix document can be configured to omit certain columns and rename column names.
Refer to the Advanced Settings (Requirements Traceability Matrix) page for details.
6.3. Customize the Traceability Matrix columns
Ketryx allows full customization for the displayed columns, column relationships / traces, traceability checks, including their corresponding error status messages, and more.
Refer to the the Traceability Configuration page for more details.
Last updated
