In today’s landscape of digital adoption and the rapid growth of software technologies, many domains leveraging technology are within regulated industries. However, with the introduction of more technology comes the need for more software—and more software testing. This article will touch on the unique attributes, challenges, and considerations of software testing within these regulated domains.
Defining “regulated” industries
While many industries have specific guidelines and domain nuances, we will refer to “regulated” industries as those that are governed by overarching regulatory compliance standards or laws.
These governance standards in most cases impact the depth, agility, and overall Software Development Lifecycle (SDLC) on how these standards are developed into requirements and then validated.
Below is a sampling of some of these domains:
- Healthcare
- Manufacturing
- Banking/Finance
- Energy
- Telecommunications
- Transportation
- Agriculture
- Life sciences
Unique requirements
Common characteristics that teams will likely encounter when analyzing the software quality/testing requirements in these environments include:
- Implementation of data privacy restriction laws (like HIPAA)
- Detailed audit history/logging of detailed system actions
- Disaster recovery and overall data retention (like HITRUST)
- High standards for traceability and auditing “readiness”
- Government compliance and/or oversight (like the Food and Drug Administration / FDA)
These common regulatory requirements are critical for planning and executing testing and establishing a quality of recording artifacts essential to supporting auditing and traceability.
Testing considerations & planning
Many testers and their teams are now being proactive in using paradigms such as shift-left to get early engagement during the SDLC. As part of early requirements planning through development and testing, specialized considerations should be taken within these regulated industries.
Requirements & traceability
- The use of a centralized test repository for both manual and automation test results is critical
- Tests and requirements should be tightly coupled and documented
- Product owners and stakeholders should be engaged in user acceptance testing and demos to ensure compliance
- Test management platforms should be fully integrated with a requirement tracking platform, such as Jira
Image: The TestRail Jira integration is compatible with compliance regulations and flexible enough to integrate with any workflow, achieving a balance between functionality and integration.
Once teams have solidified a process for defining and managing requirements and traceability, it becomes imperative to ensure that the quality of test records is not only accessible but also restricted to those who require it.
This controlled access is crucial, particularly in auditing situations, where the accuracy and reliability of test records may play a critical role. This approach for access controls is commonly referred to as the “least privilege” principle.
Image: With TestRail Enterprise role-based access controls, you can delegate access and administration privileges on a project-by-project basis
Test record access controls
- Limit test management record access to the minimum required for team members
- Ensure only current active team members have test record access
- Implement a culture of peer reviews and approval to promote quality and accurate tests
Image: TestRail Enterprise teams can implement a test case approval process that ensures test cases meet organizational standards.
As test cases and test runs are created manually or using test automation integrations like the TestRail CLI, it is important to maintain persistent audit logging of these activities. Within regulated industries, audit requirements and “sampling” may require investigation of the history and completeness of a given test that was created and executed against a requirement.
Image: TestRail Enterprise’s audit logging system helps administrators track changes across the various entities within their TestRail instance. With audit logging enabled administrators can track every entity in their installation.
Audit history
It’s important to maintain a log that allows viewing of historical data on test case creation and execution. This supports audit readiness for requirements validation traceability.
Lastly, as teams focus on the development, testing, and delivery of software, we have to be mindful of disaster recovery and data retention of the artifacts we create.
In the same thought process as disaster recovery of a given system under test, the quality of records for testing and release must persist to support compliance requirements and audits. Although centralized test management platforms with integrated restore capabilities are preferred, various tools and processes can be used to achieve this.
Image: TestRail Enterprise’s configurable backup and restore administration features enable administrators to specify a preferred backup time window, see when the last backup was completed, and restore the last backup taken.
Self-assessments & internal auditing
For all teams that are iterating on engineering, testing, and overall SDLC improvements, it’s important to dedicate time to perform self-assessments.
Self-assessments in the context of software testing and quality in regulated environments can be a highly effective tool for identifying process gaps and shortcomings.
Self-assessment/audit evaluation criteria
Examples of critical areas to include in your self-assessments or audit readiness exercises include:
- Having full traceability via linkage of all tests to the corresponding requirements artifact (such as a Jira issue or defect)
- Tests that have been planned and executed are linked to a given release event/designation
- Failed tests for a given release or sprint are linked to a defect artifact (such as a Jira defect)
Once a self-assessment or internal audit is performed, ensure that the team collects actionable information such as improvements to requirements traceability or more detailed disaster recovery documentation that can be used to improve the overall SDLC with a focus on core compliance best practices and standards.
Bottom line
Additional considerations and requirements must be made across the SDLC when operating teams within regulated industries. The early inclusion of these additional requirements with all team members is critical to ensuring compliance and overall success in audits and other regulatory assessments.
Key takeaways
- Focus on traceability, ensure linkage of tests to requirements
- More focus on security and access controls testing
- Centralize all test artifacts in a repository with backups/data retention
- Plan and execute disaster recovery validation
Watch the Testing In Regulated Industries webinar on the TestRail Youtube channel for more information on the unique challenges and characteristics of software testing in regulated industries!
