Discover TÜV-certified GoogleTest with Agentic AI for C/C++ testing!
Get the Details »
WEBINAR
As AI moves into safety-critical systems, it changes some of the fundamental assumptions behind traditional safety engineering. An AI system can operate exactly as designed and still produce an unsafe result because of limitations in its data, capabilities, or operating conditions.
ISO/PAS 8800 provides a framework for addressing these challenges. Developed for AI used in road vehicles, it works alongside ISO 26262 and SOTIF to address AI-specific risks across the entire safety lifecycle—from requirements, data, and model development to verification, deployment, and operational monitoring.
Watch this webinar to learn how ISO/PAS 8800 approaches AI safety at the system level and how organizations can build the evidence needed to demonstrate that AI-enabled systems are acceptably safe.
What you’ll learn:
AI safety isn’t about proving that an AI model will never make a mistake. It’s about engineering the entire system so that AI limitations don’t become unacceptable safety risks.
With conventional software, unsafe behavior can often be traced to an identifiable fault or defect, such as incorrect code, a specification error, or a hardware failure. Engineers can identify the cause, correct it, and verify the change.
AI introduces another challenge. A pedestrian detection model might operate exactly as designed but fail to recognize a person in heavy rain, unusual lighting, or another condition that was not adequately represented in its training data.
This distinction is central to ISO/PAS 8800. AI safety is not just about proving that a model is reliable. It is about controlling risk across the entire system.
That system may include:
If a vehicle fails to detect a pedestrian, the cause could be poor training data, a model limitation, a preprocessing defect, a sensor fault, or a problem in conventional software. Each cause requires a different response.
ISO/PAS 8800 was developed for AI used in production road vehicles, but its ideas can apply to other safety-critical industries. It does not replace ISO 26262 or SOTIF. Instead, it extends the safety process to account for AI-specific risks.
The standard allows teams to deviate from traditional verification methods when those methods do not fit AI technology. For example, static analysis and structural code coverage are valuable for C or C++ software, but they cannot be applied to a neural network in the same way.
A different strategy may involve:
This flexibility is not a free pass. Any deviation must be documented, justified, and supported by evidence showing that the safety objective is still being met.
A key idea in AI safety is that the model does not need to be perfect for the overall system to be safe. Rather than trying to prove that the AI will never make a mistake, the architecture should anticipate that it sometimes will.
For example, an AI component might incorrectly instruct a vehicle to travel at 80 miles per hour in a residential area. An independent speed-limit control can compare that command with map data and block the unsafe action.
Other protective measures include:
The goal is to detect, contain, and control unexpected behavior.
AI safety continues after development and deployment. ISO/PAS 8800 connects AI safety activities to the familiar safety life cycle: define requirements, design, verify, validate, deploy, and monitor.
In practice, the process is more iterative:
Connected systems make this feedback loop increasingly practical. Field information can reveal performance drops in certain weather or unusual situations. That information may lead to new training data, revised requirements, or a model update.
Over-the-air updates can support improvement, but safety-related changes must be controlled, traceable, verified, and supported by evidence before deployment.
With machine learning, the training, validation, and test data strongly influence system behavior. Incomplete, biased, incorrect, or poorly labeled data can create safety problems even when the model itself is implemented correctly.
The dataset needs its own lifecycle. From defining what the data needs to represent to maintaining it as new conditions are discovered in the field.
The dataset lifecycle should address a series of practical questions:
Traceability should connect dataset decisions to AI safety requirements. Teams also need to record known limitations and explain why the data is sufficient for the intended operating environment.
AI safety cannot be demonstrated with one accuracy number. A claim such as “the model is 99.9% accurate” is incomplete without knowing where, when, and how it was tested.
A stronger assurance argument combines:
Testing should move from the individual AI component to integrated functions and finally to the complete vehicle system. Simulation is especially useful because engineers can repeat difficult scenarios involving weather, construction, pedestrians, traffic, and unusual road layouts without the cost or danger of reproducing them on public roads.
AI safety analysis also needs to consider hazards that can emerge from complex system interactions, even when individual components have not failed. Techniques such as STPA (System-Theoretic Process Analysis), based on the STAMP model, can complement traditional approaches such as FMEA and fault tree analysis by examining unsafe control actions, interactions, and system behaviors that could lead to hazards.
AI-enabled systems still depend on conventional software. C and C++ code often handles preprocessing, monitoring, safeguards, communication, and fallback behavior. If those controls are expected to protect people when the AI is wrong, they must be highly dependable.
In many cases, this conventional software implements the deterministic safeguards that monitor, constrain, or override AI behavior. If we’re relying on those safeguards when the AI gets something wrong, we need strong evidence that the software itself is reliable.
Teams can use established practices such as:
AI can also assist with verification by suggesting fixes, generating tests, finding coverage gaps, and prioritizing risks. However, engineering judgment remains essential. People must review the results and decide whether the evidence is accurate and sufficient.
ISO/PAS 8800 recognizes that AI cannot be proven perfectly safe. Instead, it provides a way to identify AI-specific risks, manage uncertainty, and build a defensible safety argument.
The most important shift is to stop treating the AI model as the whole product. Safety depends on the full chain—from data and sensors to models, software, safeguards, and real-world monitoring. When that chain is engineered as one system, organizations have a better chance of building AI that is not only capable, but acceptably safe.
