Comprehensive System Testing for Airborne Software

See what API testing solution came out on top in the GigaOm Radar Report. Get your free analyst report >>

Software System Testing

System testing tests the system as a whole. Once all the components are integrated, the entire system is tested rigorously to verify that it meets the specified functional, safety, security, and other nonfunctional requirements.

DO-178C specifies both software and hardware/software integration testing. In terms of the software development aspect of airborne systems, this aligns with the concept of “system testing” for the purposes here. There are many more aspects of system and flight testing of airborne systems that aren’t covered here.

Section 6.4.3 a Requirements-Based Hardware/Software Integration Testing focuses on the operation of the software on the target hardware environment. The aim is to validate high-level requirements. It’s also important to point out that nonfunctional requirements must be tested, and Section 6.4.2.1 requires normal range tests to demonstrate normal operation of the software alongside Section 6.4.2.2, which requires robustness test cases. These are tests that use abnormal data ranges that fall outside expected values for inputs to demonstrate the system can handle them without failure.

Photo showing a wide view of a jet bridge at the airport filled with parked commercial airplanes, baggage tugs, and ramp-ups.

This type of testing in safety-critical software is performed by a specialized testing team. System testing falls within the scope of black box testing. As such, it shouldn’t require any knowledge of the inner design of the code or logic.

An important distinction with system level testing is that the system is tested in an environment that is close to the production environment where the application will be deployed. At this stage, specific safety functions are verified, and system wide security testing is run.

Service Level Testing of Airborne Systems

Airborne systems may have connectivity into larger systems that, as an example, collect and analyze status and flight data. Any sort of communication bus or network must be tested for data integrity, security, and confidentiality. System testing needs to include these environments for complete validation.

Instead of viewing system quality in terms of meeting individual component requirements, the scope is broadened to consider the quality of the services provided. Testing at the service level ensures nonfunctional requirements are met. For example, performance and reliability are difficult to assess at the device level or during software unit testing. Service-based testing can simulate the operational environment of a device to provide realistic loads.

Security is a growing concern in airborne systems. Cyberattacks are possible in modern systems and likely originate from the network itself by attacking the exposed APIs. Service based testing can create simulated environments for robust security testing, either through fuzzing (random and erroneous data inputs) or denial-of-service attacks.

Virtual Test Environment & Service Level Testing

A real test lab requires the closest physical manifestation of the environment in which a system is planned to work. Even in the most sophisticated lab, it’s difficult to scale to a realistic environment. A virtual lab fixes this problem.

Virtual labs evolve past the need for hard-to-find (or nonexistent) hardware dependencies. They use sophisticated service virtualization with other key test automation tools.

Service Virtualization

Service virtualization simulates all of the dependencies needed by the device under test in order to perform full system testing. This includes all connections and protocols used by the device with realistic responses to communication. For example, service virtualization can simulate an enterprise server backend with which a system under test communicates. Similarly, virtualization can control and simulate a dependent system, like patient information, in a realistic manner.

Service & API Testing

This testing drives the system under test in a manner that ensures the services and APIs it provides perform flawlessly. These tests can be manipulated via the automation platform to run performance and security tests as needed.

Runtime Monitoring

This detects errors in realtime on the system under test and captures important trace information.

Test Lab Management & Analytics

Once virtualized, an entire lab setup can be replicated as needed, providing overarching control of the virtual labs. Test runs can be automated and repeated. Analytics provide the necessary summary of activities and outcomes.

Parasoft SOAtest & Virtualize for Service Level Testing of Airborne Software

Developers can build integrations earlier, stabilize dependencies, and gain full control of their test data with Parasoft Virtualize. Teams can move forward quickly without waiting for access to dependent services that are either incomplete or unavailable. Companies can enable partners to test against their applications with a dedicated sandbox environment.

Parasoft SOAtest delivers fully integrated API and web service testing tools that automate end-to-end functional API testing. Teams can streamline automated testing with advanced functional test creation capabilities for applications with multiple interfaces and protocols.

SOAtest and Virtualize are well suited for network-based, system-level testing of various types, including the following:

Comprehensive protocol stack that supports HTTP, MQTT, RabbitMQ, JMS, XML, JSON, REST, SOAP, and more.
Security and performance testing during integration and system testing with integration into the existing CI/CD process.
End-to-end testing that combines API, web, mobile, and database interactions into virtual test environments.