Automate the Tool Qualification Process for Safety-Critical Software

Safety-critical software development standards, such as DO-178C (aerospace), ISO 26262 (automotive), IEC 62304 (medical), and EN 50128 (functional safety), require that manufacturers prove that the tools they are using to develop their software provide correct and predictable results. The process of providing such evidence is known as tool qualification. While it’s a necessary process, tool qualification is often a tedious and time-consuming activity that many organizations fail to plan for.

The end deliverable is proof in the form of documentation, but there’s more to the qualification process than just delivering a big pile of static documentation. In this post, we’ll dive into how to use Parasoft’s Qualification Kit for C/C++test, which includes a convenient tool wizard that brings automation into the picture and reduces the time and effort required for tool qualification.

First: Ensure Your Development Tool Is Certified

Tool qualification needs to start with tool selection, ensuring you are using a development tool that is certified by an organization such as TÜV SÜD. This will significantly reduce the effort when it comes to tool qualification. Parasoft C/C++test is certified by TÜV SÜD for functional safety according to IEC 62304, EN 50128, IEC 61508, and ISO 26262 standards for both host-based and embedded target applications, paving the way for a streamlined qualification of static analysis, unit testing, and code coverage requirements for functional safety-critical standards.

Second: Streamline the Tool Qualification Process

Traditionally, tool qualification has meant significant manual labor, testing, and documenting development tools to satisfy a certification audit. However this documentation-heavy process requires manual interpretation and completion, and as such, is time-consuming and prone to human error.

We solve this problem by bringing automation to this process with our Tool Qualification Kit. Increasing productivity and reducing human error through automation has always been at the core of what we do at Parasoft, so we’ve applied this same mission to the Tool Qualification process for Parasoft C/C++test. Our Tool Qualification Kit walks the user through an intuitive workflow to dramatically reduce the required effort.

Benefits of using the tool qualification kits include:

• Automatically reduce the scope of qualification to only the parts of the tool in use
• Automate tests required for qualification as much as possible
• Handle any manual tests as eloquently as possible and integrate results alongside automated tests
• Automatically generate audit-ready documentation that reports on exactly what’s being qualified, not more or less!

Qualify Only What You Use

Why make developers sift through qualification material for DO-178C when they’re working on an automotive project requiring ISO 26262?

What happens if the team is only using parts of Parasoft’s C/C++test solution?

There should be no need to do any extra work for qualifying capabilities not used in your development products. Reducing the scope of testing, reporting, and documentation is a key way to reduce the qualification workload.

The example below shows a user selecting the only use case of C/C++test being used (static analysis) to check compliance with the MISRA C 2023 standard, as part of ISO 26262 qualification. The tool then selects only the parts of the qualification kit needed for this function. There are five functional safety standards supported for qualification.

Users can deselect options to reduce the scope further as required for their project. Upon selection, only tests and documentation are used and provided from this point forward.

Leverage Test Automation and Analytics

If there’s one advantage to qualifying test automation tools, it’s that the tools can be used to automate their own testing. The tool qualification requires that tests be run and results verified as outlined in the qualification kits we provide. Automating this as much as possible is key to making it as painless as possible. Even manual tests, which are inevitable for any development tool, are handled as efficiently as possible. Step-by-step instructions are provided and results are entered and stored as part of the qualification record.

Parasoft C/C++test collects and stores all test results from each build, and tests run as they do for any type of project. These results are brought into the test status wizard in the Parasoft Tool Qualification Kit to provide a comprehensive overview of the results:

Centralizing, aggregating, and automating the qualification process greatly reduces manual tracking of compliance progress.

Addressing Known Defects

Every development tool has known bugs and any vendor that doesn’t document them isn’t doing your compliance effort any favors. But there’s more to dealing with known defects than just documenting them. Tool qualification requires that you provide proof that these defects are not affecting the results used in the project. For each known defect, you must provide a mitigation and document it to the satisfaction of the certifying auditor.

It’s incumbent on the tool vendor to automate the handling of known defects as much as possible. After all, we’re asking customers to deal with our bugs as part of their workload! The Parasoft C/C++test qualification kit includes a wizard to automate the recording of mitigation for known defects as shown below:

Automate the Documentation You Need

The result of tool qualification is documentation and lots of it. Every test executed with results, every known defect with mitigation, manual test results, and exceptions are all recorded and reported. Qualification kits from other vendors can be just documentation alone, and without automation, documenting compliance is very tedious.

Instead, using the Qualification Kits for C/C++test, the critical documents are generated for you as part of the workflow.

• Tool classification report. Determines the qualification needed and presents the maximum safety level classification for C/C++test based on the use cases selected by the user.
• Tool qualification plan. Describes how C/C++test is going to be qualified for use in a safety-relevant development project.
• Tool qualification report. Demonstrates that C/C++test has been qualified according to the tool qualification plan.
• Tool safety manual. Describes how C/C++test should be used safely, for example, compliance with safety standards like ISO 26262, IEC 61508, and so on, in safety-critical projects

In each of these documents, only the documentation required for the tool features in use is generated, since the scope of the qualification was narrowed down at the beginning of the project. Automation and narrowing the scope of qualification greatly reduces the documentation burden.

Conclusion

Tool qualification for safety-critical software projects in automotive, aerospace, railway, and functional safety industries is a burdensome process. Reducing the scope of qualification and automating as much of the process as possible is key to reducing the workload and tedium of qualification. The Parasoft Tool Qualification Kit for C/C++test makes this as painless and efficient as possible to greatly reduce the burden on our customers, so they can get back to delivering high-quality products for their customers.

“MISRA”, “MISRA C” and the triangle logo are registered trademarks of The MISRA Consortium Limited. ©The MISRA Consortium Limited, 2021. All rights reserved.