Software Development Process for Safety-Critical Systems

When it comes to designing and programming software for things like autonomous driving cars, you have to consider safety a top priority. This applies to much more than futurist ideas for transportation. In fact, it applies to many items millions of people use every single day. Airplanes and rail lines. An MRI machine. Your own car’s turn signal or windshield wiper.

All of these complex systems (and many more) depend on safety-critical software developed with as many failure scenarios as possible in mind to mitigate the consequences. But what sets this kind of development apart from others? The challenges, best practices, and safety standards involved entail far bigger stakes and some friction with modern methodologies. So, what’s the deal with safety-critical software development?

What Is Safety-Critical Software?

Something considered safety-critical software is usually an embedded software application specifically designed for systems that, in the event of a failure, measures exist to prevent injury and the loss of life. This means that any such software requires verification, validation, and reliability to be baked into every step of the development life cycle.

What Are Examples of Safety-Critical Devices?

One example of a safety-critical system would be a pacemaker or other health and medical devices. Other types include advanced car safety features like a backup camera or lane assist systems. More complex examples are aircraft flight controls and nuclear systems.

How Is the Development of Safety-Critical Software Different?

Safety-critical systems require any related software to adhere to rigid standards of safety during development. Safety standards are regularly updated, as well, to address issues and vulnerabilities as they arise. This requires software developers of safety-critical systems to keep abreast and remain agile when it comes to development.

In addition, the goal of this kind of development is not speed nor even features, but safety. During the life cycle, developers must consider as many real-world failure scenarios as possible on top of adhering to safety standards within their industry. This also requires robust testing, validation, and verification.

Software Development in Industries With High Safety Requirements

While industries such as medical devices and nuclear systems might jump to mind, many more utilize rigorous safety standards. Industries, where failures can lead to catastrophic loss, must rely on safety standards that facilitate the safest software possible. This includes automotive and aerospace, of course. However, this also applies to industries such as rail and even cybersecurity or financial systems.

What Are the Industries Where Safety Critical Software Is Used Extensively?

As mentioned earlier, both the automotive and aerospace industries heavily rely on safety-critical systems to operate. Things such as ABS brakes, the ever-developing autonomous driver, and even simple lights all play into the safety of vehicles concerning software development. In aerospace, developers must consider safety standards across a variety of systems from oxygen and air recycling to engine combustion and telemetry communication.

But other industries also heavily employ high safety requirements such as healthcare, transportation, nuclear energy, and defense. Failure could mean the loss of human life, property damage, or environmental damage—even on a small scale. That’s unacceptable when it comes to software development in these industries.

What Are the Standards & Regulations Governing the Development of Safety-Critical Software?

The International Organization for Standardization (ISO), established in 1947, sets safety standards across many industries with the input of more than 160 members from many countries. Along with other bodies, the ISO works constantly to develop and re-evaluate their safety standards across thousands of systems in many industries.

Each industry has its own bodies that practice oversight and regulation of safety-critical devices and software. For instance, in the automotive industry, you have the international ISO 26262 standard for functional safety and ISO/SAE 21434 standard for automotive security. For medical devices, you have IEC 62304 and IEC 62443 for security. There are also coding standards like MISRA and AUTOSAR C++ 14 that are known to be applied to a variety of other industries like aerospace, military, rail, and medical. Other security standards for various types of applications include CERT, CWE, and OWASP.

Essentially, there are distinct gold standards developers must follow within each industry – and some that cross industries. These standards are constantly being updated based on the latest technology and ways to exploit that technology.

Best Practices for Software Development in Safety-Critical Systems

There are five essential pillars for requirements in safety-critical software development.

1. Unambiguous. In documentation as in the code itself, there needs to be agreement on what requirements mean without interpretation or guesswork.
2. Traceable. Programmers and managers should be able to trace requirements up and down when reviewing code or during the test process.
3. Repeatable. Having the program and software return expected requirement results is the bare minimum. The devices should operate as requirements, while also returning expected data and output during any kind of failure.
4. Consistent. There shouldn’t be conflicting requirements characteristics when it comes to timing, performance, prioritization, etc. Terminology is uniform and consistent in documentation, as well, to avoid confusion.
5. Testable. Well-developed safety-critical programs will meet requirements across test types.

Additionally, there are another five best practices to keep in mind when developing software for safety-critical systems.

1. Maintain accurate records and documented history trail or traceability.
2. Educate programmers on best practices, methodology, and the importance of documentation.
3. Work with the most appropriate and best coding standards for your project.
4. Keep code complexity down as complex code in complex systems is difficult to test which could lead to more errors.
5. Develop the software with the end user in mind.

What Are the Potential Consequences of Errors in Safety-Critical Software Operation?

The consequences for safety-critical errors are not something as simple as a light bulb not working when it should. The errors can range from a battery getting too hot during operation to something such as catastrophic airplane engine failure. There can also be a human toll, damage to property and the environment, financial losses, or even threats posed to national security.

That’s why following the above best practices is not just advised, but mandatory. Achieving regulatory approval is not a situation where “good enough” will work.

Factors to Keep in Mind When Developing Software for Safety-Critical Systems

Remember: the goal of safety-critical software development is not speed or features, but safety. This is the most critical aspect to remember when coding for safety standards. Safety-critical software development failure in these systems can lead to things like the NASA Mars Climate Orbiter entering the Martian atmosphere too quickly and too low, causing destruction. Issues like this come down to several important factors that teams must keep in mind during the development life cycle to mitigate safety-critical software failure.

In addition to adhering to safety requirements and formal methods, keep these things in mind:

• Consider the software architecture because a good design can facilitate easier upgrading.
• Safety concerns, mitigating hazards, and ensuring regulation adherence will make your project more complex.
• A fail-safe design means that your device and/or software can fail in a safe or expected and predictable manner.
• Only a specific level of danger and failure are permissible.
• Safety-critical software development can be performed using various development lifecycle methodologies like Spiral, Waterfall, Agile, and in tandem.
• Go farther than you think you need to in terms of testing your software and then keep going.
• Practical field data is required to improve safety-critical system software.

Empowering Safety-Critical Software Development With Advanced Tools & Technologies

A big pitfall developers can avoid when it comes to tools for safety-critical software development is certification. Using a tool that is certified by the appropriate organization can offer peace of mind. Certified tools lead to achieving compliance reliably. However, compliant software requires the right development methodologies to avoid prolonged development timelines and costs. Also, that’s where the shift-left mindset comes into play.

A great example of Agile methodologies is DevOps CI/CD and Scrum co-existing in parallel, removing silos, promoting communication, enabling productivity, and automating verification and validation.

Maintaining quality and affordability have become priorities and unique challenges for many development teams. An example is software test automation built into an automotive CI/CD DevOps deployment. CI/CD pipelines offer continuous testing that can reduce project costs and reduce project timelines. Even with adhering to rigorous safety standards like ISO 26262, automated testing brings added code coverage, security, and actionable data to the table.

Arming your developers with the best tools for the job can only make their job more straightforward. This, in turn, can lead to more streamlined documentation, consistency, and delivering your product on time and on budget—or even ahead of time and under budget.

The Role of Parasoft in Software Development Process for Safety-Critical Systems

Parasoft delivers a variety of solutions across many industries when it comes to developing safety- and security-critical software. These complex systems require consistent, unambiguous documentation, actionable test data, and tools that empower developers.

Unit testing, requirements-based testing, regression testing, security testing, and integration testing all need to include requirements traceability and repeatability. Automated and TÜV SÜD certified solutions like C/C++test can reduce the workload on developers and testers alike while improving code quality, coverage, and overall product quality.

Our tools directly support many safety standards including, but not limited to:

• ISO 26262
• IEC 62304
• DO-178C
• IEC 61508
• UL 2900
• DISA ASD STIG
• OWASP
• MISRA
• AUTOSAR
• CERT
• CWE
• PCI DSS

Tools like C/C++test allow teams to detect defects early, automate industry standard compliance, and test intelligently. A great real-world example of how Parasoft solutions integrate into safety-critical systems is how a U.S. Department of Defense contractor in the aerospace and defense industry leveraged a DevOps pipeline to enhance code coverage, testing thoroughness, and sustainability.

How Is Safety-Critical Software Evolving in Response to Industry Trends?

As with all things, the visible trend everyone can see when developing safety-critical systems is the rising cost of doing business. In addition, the complexity is ever-growing, as well. This means that developers will need to be more diligent with documentation and managers will have to be willing to educate their teams as they need to with regards to safety standards, and on a regular basis to ensure up-to-date knowledge.

However, when it comes to how the software itself will have to respond to industry trends, the waters get a bit murkier. We can thank the Internet of things, machine learning, and AI for that—in both beneficial and burdensome ways. While having more things to be able to speak to each other or even adapt on an as-needed basis to different scenarios is useful, this can lead to additional security vulnerabilities. Planning for the inevitable exploitation of these new threat vectors makes already complex systems in software engineering even more complex. Test automation has started to move into AI and machine learning to fill this gap before it becomes too wide.

Conclusion

Edsger W. Dijkstra gave us the key to modern software development with this quote:

“Those who want really reliable software will discover that they must find means of avoiding the majority of bugs to start with…”

After all, preventing problems from the outset is easier than fighting fires as they pop up. This aspect of Agile methodology is a game changer when it comes to safety-critical software development. The right tools can make the process even more streamlined.

Building a solid code architecture, maintaining consistent and systematic language for documentation, and having continuous and appropriate testing can help your team meet compliance standards.