Reducing the Risk and Cost of Achieving Compliant Software

Compliance in software development may mean slightly different things to different organizations. The differences in attaining compliant software likely relate to the marketplace where they sell their products. They range from rigorous development for safety-critical software to enterprise applications that require security and privacy capabilities for handling sensitive data.

What Is Compliant Software?

Standards and regulations emerge in different industries and are put in place to protect customers and users from harm. That harm could manifest itself as physical danger or financial loss of credentials and sensitive personal information. Often these standards are a reaction to events that resulted in injury or major data breaches.

Safety, Security, & Privacy Standards

In terms of safety, there are several standards some of which include:

• DO-178C (aircraft avionics)
• ISO 26262 (automotive software)
• EN 50128 (railway systems)
• IEC 62304 (medical)
• IEC 61508 (industrial automation)

There are also standards for security and privacy such as GDPR for general data protection/privacy and PCI-DSS for secure handling of credit card information.

Meeting Requirements of the Standards

For most standards, compliance demonstrates in various ways that the software you’re deploying meets the requirements of the standard. The development process produces documentation, reports, and other artifacts that show compliance. It keeps audits in mind and can include demonstrations and acceptance testing.

The amount of time and cost for compliance depends on the level of rigor the industry standard requires. These standards define the design and development guidelines to follow based on the safety or security impact that the software has on the end user. Failure of the product can lead to death or injury in severe cases. For example, ISO 26262 defined for levels of safety integrity level (A, B, C, D), where ASIL A is the lowest safety risk or impact, and D the highest.

Safety Integrity Level

The probability of occurring plus the severity of the risk and controllability determines where a product fits on the safety integrity level. Controllability is the ability of the driver to mitigate the risk.

The following table shows the severity, exposure, and controllability translated to the various ASIL in ISO 26262.

EN 50128 uses the terms software safety integrity levels (SSIL) with similar criteria. Both the ISO 26262 and EN 50128 standards derive their definitions from IEC 61508, which is an overarching standard for functional safety.

These SSIL levels are also based on probability of failure and the severity of the failure on life and limb. As expected, the level or rigor increases as the ASIL/SIL level requires and compliance costs rise with it. The level of scrutiny increases with risk. As such, there’s an impact on software development cost and schedule, which development teams can improve with automation.

Risks & Costs Associated With Achieving Software Compliance

The biggest risk to software organizations is software failure that can cause death, injury, or other type of harm that opens the gates to liability. To help cut this risk, software organizations should comply with safety and security standards. Organizations must demonstrate that they made all reasonable engineering choices to ensure the safety and security of the user/customer/operator. Creating safer and secure software requires a commitment to the spirit and the letter of industry standards.

Rework, Retesting, & Redocumenting

Discovering significant safety and security issues late in the software lifecycle requires significant rework, retesting, and re-documenting. It’s expensive and takes time to resolve. It also points to a possible failure in the system and engineering process.

It’s critical to shift left compliance to as early as possible in the software development lifecycle (SDLC). Parasoft recommends shifting left for all quality, security, and safety practices. To this end, our recommendations include the right tools, processes, and automation to help reduce software risk and increase compliance without negatively impacting cost and schedule.

Lacking a Process Suited to the Application Space

Frequently, the adoption of culture and process that suits the application space is lacking in many organizations. Safety-critical software, for example, requires a high level of rigor in software development that implies higher quality throughout.

Lacking the Right Tools & Automation

Organizations aligning to safety and security requirements carry a heavy workload that involves providing an audit trail of compliance for each software stage and process. Along with usual software development, the additional work includes:

• Tracing requirements to code and tests.
• Generating reports of failures, code coverage, remediation, and deviations.

In most cases, coding standards are applicable, for example MISRA for ISO 26262. So, teams must work to correct deviations from the standards and measure compliance.

Functional tests are necessary to prove that products meet requirements. This is in addition to safety and security requirements tests. All the tests require traceability and audit documentation.

Combined, the increased level of traceability, testing, strict development standards, and documentation significantly increase development costs for products. It makes the risk of compliance issues downstream even more daunting.

The key to reducing the workload and tedium of these processes is automation.

Best Practices for Successful Software Compliance

By automating the most repetitive and tedious processes, software development tools help reduce the workload and errors of compliance processes. In addition, tools are better at keeping track of the project history and relating results against requirements, software components, tests, and recorded deviations.

Tools also help enforce good practices such as coding standards, unit testing, code coverage, traceability, and more. Teams can adopt tools early in the life cycle to help shift left the compliance workload.

Here are four recommended best practices to follow to successfully deliver compliant software:

• Thorough assessment and documentation. Put in place a quality management system with well-documented processes, procedures, and responsibilities for achieving quality policies and objectives for the entire organization. This is a primary step in ensuring compliance.
• Implement clear policies and procedures. Adopt a software development process with clear policies and procedures that work for your organization. Agile scrum is hugely popular and comes with clear policies and procedures that allow teams to self-manage, learn from experience, and adapt to change.
• Utilize tools and automation. Use certified software tools that have a successful history of being used for attaining software compliance to industry functional safety standards. Tools that offer automation and remediate the introduction of human error, provide an audit trail, and reduce labor costs.
• Regular audits and monitoring. Create reporting, such as a viewable dashboards or documentation with analytics on the progress, health, and quality of your software project. Reporting that includes an assessment or audit on critical areas with high risk that need to be addressed as soon as possible

Reducing Risks & Costs of Compliance With Automation

Parasoft tools help reduce the risks and costs of compliance. Additionally, teams can satisfy verification and validation requirements with comprehensive reporting and tool integrations.

Functional Safety Standards Compliance With Parasoft Tools

Developing safety-critical software is no easy feat. Parasoft helps ease the burden by offering a broad range of development testing and verification tools. Here are some of those proven verification methods, which are designed specifically for functional safety-critical applications:

• Coding standards compliance analysis
• Data and control flow analysis
• Unit testing
• Application monitoring
• Workflow components

When performing unit testing or runtime memory monitoring, all components required to build the test binary, including the test cases for tested code stimulation and stubs, are expressed in the form of source code and can be versioned and inspected. This approach is superior to other tools in which the stimulation for tested code is sent at runtime from the host and requires additional conversions before assigning to the variables in memory. Parasoft C/C++test removes this unnecessary layer and assures that the state of the memory before executing tests is constructed in the same way as in the production systems.

C/C++test combines multiple testing technologies integrated into one tool. Its comprehensiveness translates into increased developer efficiency when developing applications with functional safety requirements. Developers can focus on their core activities without having to learn, integrate, and qualify several tools. Tool integrators or architects don’t have to spend time implementing interfaces between tools for exchanging data and generating uniform reports. Instead, they get all of it from Parasoft out of the box

Assembly Code Coverage

Parasoft’s Assembly Coverage Tool (ASMTool) satisfies the executable object code recommendations set out by C Level A. ASMTool can generate structural coverage reports from executable object code with little effort.

Structural coverage is reported with easily reviewable insight into compiler-generated code that is not directly traceable to source code statements. It also tracks switch statements, providing full insight into the test execution paths at the machine language instruction level.

ASMTool also supports collecting structural coverage from both the software integration process (target hardware testing) and desktop development environment (Parasoft C/C++test for unit testing).

Coding Standard Compliance

Coding standards play an important role in many safety and security standards. In most cases, the only practical way to enforce compliance with a coding standard like AUTOSAR C++14 (used in automotive systems) is with a static analysis tool. Parasoft helps organizations perform static analysis, unit testing, structural code coverage, requirements traceability, and other testing methods recommended by the standard.

Parasoft C/C++test supports mapped checkers for the coding standards, so developers don’t need to dereference warnings back to the standards. The proven tool provides testing functions, configurable contexts, and reporting mechanisms that assist with advanced software testing methodologies. The standards that Parasoft directly supports include:

• MISRA
• AUTOSAR
• CERT
• CWE
• ISO 26262
• DO-178C
• EN 50128
• IEC 61508
• IEC 62304
• DISA ASD STIG
• OWASP
• PCI DSS
• UL 2900

Software development teams can achieve compliance and save time with Parasoft C/C++test. Teams have a practical way to prevent, expose, and correct errors in their software with automatically generated reports and dashboards. They can prepare the documentation required for audits with the help of automated tool qualification.

Teams can customize Parasoft’s interactive reporting system to the requirements imposed by whichever rules they use, like AUTOSAR C++ 14, HIC++, MISRA C/C++, and so on, and enable an efficient daily workflow. Developers can check the compliance of their code without leaving their IDEs and integrate the scanning process into CI builds on servers.

Parasoft also supports security-focused standards such as SEI CERT. It helps organizations detect security vulnerabilities in their code and manage the process of achieving compliance with the standard. The high accuracy of the code checkers, low level of false alarms, and sophisticated compliance reporting consistent with the CERT standard help minimize related costs and overhead.

Parasoft’s unique compliance reporting provides a dynamic view of the development process. Interactive reports and customized dashboards provide different views into the results of CERT compliance status and use the wording and categorizations that are defined by the standard, to make it easier to understand.

Centralized Reporting and Document Generation

Combining several test automation techniques results in a lot of quality information. This data is useful, but the sheer volume makes it difficult to gain insight into the quality of your deliverables and make decisions about the release or changes in the development process.

To support decision making, it’s beneficial to aggregate the data into a centralized view of quality. Graphical dashboards provide a top-level view of testing progress. Correlating quality data to your requirements and user stories provides true bidirectional traceability so you can assess the business risk and impact of gaps in your testing practices.

Parasoft’s reporting and analytics dashboard aggregates data from across all the different testing practices, including static analysis, like AUTOSAR or MISRA coding standard compliance, code coverage and traceability from unit testing, API testing, and system-level functional validation. This provides a centralized view of quality that’s a real-time audit of your process and visibility into the status of compliance with the required standard. This analysis also generates the documentation needed to demonstrate compliance with an auditor.

Just as examples, let’s look at reporting from Parasoft C/C++test and Parasoft DTP for MISRA and SEI CERT.

Compliance Reporting for MISRA

Parasoft C/C++test provides dedicated reporting for documenting compliance to MISRA C. A dashboard on the Parasoft web portal provides at-a-glance views on the current state of the project, such as the one here:

Each of these dashboard widgets is linkable to a more detailed view, containing detailed violation reports, files, and source code.

From here, you can automatically create the reports needed to document MISRA compliance as outlined in MISRA Compliance 2020: Achieving Compliance With MISRA Coding Guidelines. Automating these reports is a big time-saver, greatly reducing the amount of manual work required to document project compliance.

Compliance Reporting for SEI CERT C

Although the SEI CERT C standard doesn’t require specific compliance reports, it does require a project to document conformance to the rulesets, such as L1, L2, and fully compliant. Parasoft C/C++test includes a dedicated dashboard for CERT C conformance, that looks like this:

Team leads can use this dashboard view to dig deeper into specific areas of concern and assign tasks to developers to increase compliance over time. Viewing the results in context of the risk assessment framework used by the coding standard itself (for instance, seeing specific violations of L1 guidelines), significantly streamlines the process. Automating this reporting reduces the amount of analysis that team leads and architects need to perform in order to achieve CERT C compliance.

Tool Qualification

Tool qualification is a mandated process for safety-critical software development. Parasoft’s Qualification Kits for Parasoft C/C++test automate the process of creating the supporting documentation required for tool qualification of static analysis, unit testing, and coverage requirements, reducing both the potential for human error and time taken to perform tool qualification.

Reducing the risk and cost of tool qualification includes the following:

• Automatically generate compliance documentation. Parasoft automates the creation of documentation required for qualifying Parasoft C/C++test for use in safety-critical industries by walking the user through an intuitive workflow and generating documentation that’s critical for demonstrating tool qualification.
• Reduce the scope of what’s needed to qualify the tool. The first step in Parasoft’s efficient workflow is selecting the specific use cases and capabilities of Parasoft C/C++test being used in the software project, to reduce the overall scope of what needs to be qualified and streamline the qualification process.
• Automatically execute tests. Although the process of tool qualification cannot be 100% automated, the unique workflow of Parasoft’s qualification kits makes it as painless as possible, reducing manual testing efforts and executing automated tests for selected use cases.

Balancing Risk & Rewards in Software Compliance

Building safety and security-critical software requires rigorous processes that are time-consuming and expensive to implement and for good reason: the failure of such software can have drastic consequences. Industry standards are put into place to help guide companies to build high-quality software to ensure safety and security in developed products. However, meeting these industry standards is costly. That cost lies in both the development process and the artifacts required to document the engineering process. Organizations can reduce costs with the right applications of tools and modern software techniques.

Parasoft provides the right tools and support for processes that help reduce the risk and costs of software compliance with a proven, unified tool suite that reduces the “friction” caused by using multiple tools from multiple vendors. An integrated solution also greatly simplifies tool qualification, a critical aspect of compliance.

Having an integrated platform allows for unique data analysis collected from multiple tools at various stages of development to focus resources on the most important goals. This centralized view of compliance, quality, and security is key to making sound decisions and greatly reducing the guesswork in achieving compliance.

“MISRA”, “MISRA C” and the triangle logo are registered trademarks of The MISRA Consortium Limited. ©The MISRA Consortium Limited, 2021. All rights reserved.