ISO 26262 Software Compliance in the Automotive Industry

As part of most C and C++ software development processes, regression testing is done after changes are made to the software. These tests determine if the new changes impact the existing operation of the software.

Regression tests are necessary, but they only indicate that recent code changes have not caused tests to fail. There’s no assurance that these changes will work. In addition, the nature of the changes that motivate the need to do regression testing can go beyond the current application and include changes in hardware, operating system, and operating environment.

Regression Testing in Safety Critical Software

In safety-critical C and C++ software development, validation is critical in proving correct functionality, safety, and security. Tests are needed to confirm any changes to the application to ensure functionality and to verify there are no unforeseen impacts on the rest of the system.

If a test case that previously passed now fails, a potential regression has been identified. New functionality could be the cause of the failure. If so, the test case may need to be updated with consideration to those changes in input and output values.

Regression testing of embedded systems also includes the execution of:

• Integration test cases
• System test cases
• Performance test cases
• Stress test cases and more

All previously created test cases may need to be executed to ensure that no regressions exist and that a new dependable software version release is constructed. This is critical because each new software system or subsystem release is built or developed upon. If you do not have a solid foundation the whole thing can collapse.

Parasoft DTP supports the creation of regression testing baselines as an organized collection of tests and will automatically verify all outcomes. These tests run automatically regularly to verify whether code modifications change or break the functionality captured in the regression tests.

If any changes are introduced, these test cases will fail to alert the team to the problem. During subsequent tests, DTP will report tasks if it detects changes to the behavior captured in the initial test.

How to Decide What to Regression Test?

The key challenge with regression testing is determining what parts of an application to test. It is common to default to executing all regression tests when there’s doubt on what impacts recent code changes have had—the all-or-nothing approach.

For large C and C++ software projects, this becomes a huge undertaking and drags down the productivity of the team. This inability to focus testing hinders much of the benefits of iterative and continuous processes, potentially exacerbated in embedded software where test targets are a limited resource.

A couple of tasks are required here.

• Identify which tests need to be re-executed.
• Focus the testing efforts (unit testing, automated functional testing, and manual testing) on validating the features and related code that are impacted by the most recent changes.