Autonomy Is Forcing a Rethink of Hardware Verification and Validation

For most of the history of hardware certification, verification and validation was a tractable problem. You wrote a requirement, derived a test, executed the test, recorded pass or fail, and moved on. The underlying assumption — that a human operator was the adaptive, intelligent element in the loop — made deterministic test cases reasonable. The hardware just had to behave predictably. The human handled everything else.

Autonomous systems break that assumption completely.

When a hardware platform is carrying out decisions previously delegated to a trained human — approach and landing sequences in commercial aviation, lane-change arbitration in highway driving, threat engagement prioritization in defense — the test space is no longer bounded by hardware states alone. It is bounded by the full distribution of operational scenarios the machine might encounter, weighted by the consequences of failure in each. That is a fundamentally different problem than verifying that a brake actuator responds within 50 milliseconds of a commanded input.

The result is a collision between verification traditions built for deterministic hardware and the statistical, scenario-dependent behavior of autonomous functions. Every major sector deploying autonomous systems — aerospace, automotive, defense — is managing that collision right now, with incomplete standards, improvised tooling, and institutional habits that evolved over decades.

What Traditional Hardware V&V Was Built to Do

The deterministic test paradigm has served hardware engineering well. A requirement states a measurable output condition. An engineer constructs a test that exercises the relevant input domain. The test produces a binary verdict. Requirements traceability matrices link requirements to tests to results, and a completed RTM is evidence of coverage.

For physical hardware behavior — thermal limits, structural load margins, electromagnetic compatibility, power supply rejection ratios — this approach is rigorous and appropriate. Hardware physics does not depend on context in the way autonomous decisions do. A capacitor’s ESR does not change based on whether the vehicle is in a construction zone.

The limits of the deterministic model only became visible when autonomous functions started producing hardware outputs. Consider an autonomous emergency braking (AEB) system. The hardware — actuators, sensors, power electronics — still has deterministic specifications. But the decision to brake, and when, is now made by an algorithm whose behavior is a function of a nearly infinite input space: object classification confidence, sensor fusion weights, road surface estimates, relative velocity vectors, occlusion patterns. You cannot write a finite set of deterministic test cases that covers this adequately. You can verify the hardware responds correctly when commanded. You cannot verify, with traditional methods, that the command is always correct.

That distinction — hardware response verification versus behavioral decision verification — is where autonomy programs are getting stuck.

Scenario-Based Testing: Borrowed from Software, Now Entering Hardware Conversations

The software and systems safety communities had started wrestling with scenario-based testing long before autonomy reached hardware certification. Model-based testing, equivalence partitioning, combinatorial test design — these are established disciplines in software V&V that acknowledge you cannot execute every possible input combination and must instead reason about coverage structurally.

What is new is that these approaches are now appearing in hardware verification plans. The shift is being driven by the hardware-software integration point that autonomous systems represent. If a hardware subsystem — a lidar-inertial measurement unit, a compute-on-chip radar processor, a propulsion controller executing autonomous thrust vectors — can only be verified in the context of the operational scenarios the software will expose it to, then hardware V&V cannot be decoupled from scenario definition.

The practical consequence is that hardware test engineers are being asked to participate in scenario libraries that were previously considered software or systems problems. Worst-case scenario analysis, which traditionally meant worst-case environmental conditions for hardware, now also means worst-case operational contexts: edge cases in object classification, adversarial weather inputs, sensor degradation combinations that produce confident-but-wrong fused estimates.

This is not purely a testing scope expansion. It changes what a hardware requirement looks like. A traditional requirement is a testable assertion about a hardware output. A scenario-conditioned requirement is an assertion about system behavior across a distribution of contexts — and that requires different documentation structures, different traceability approaches, and different tooling than most hardware programs currently have.

Simulation as a Verification Artifact

Simulation has always been part of hardware development. Thermal models, structural FEA, electromagnetic field solvers — these are mature tools. What is changing is the evidentiary role of simulation in formal verification.

In the autonomous systems context, simulation is increasingly proposed as the primary mechanism for achieving scenario coverage that physical testing cannot reach at scale. A ground vehicle autonomy program cannot physically test 10 million scenario variants. A simulation environment can generate and evaluate them. The question regulators and certification authorities are grappling with is: under what conditions does simulation-generated evidence substitute for, or complement, physical test evidence?

The aerospace sector is furthest along on this question, largely because DO-178C and its derivatives (DO-331 for model-based development) have existing frameworks for model-verified software. The FAA’s ongoing work on machine learning in aviation — particularly through the EASA AI Roadmap and FAA’s own Concepts of Operation for Uncrewed Aircraft Systems — is generating guidance on simulation fidelity requirements, environment model validation, and the relationship between simulation coverage and physical test requirements. But none of it has reached the specificity that hardware certification engineers need. The guidance is directionally correct and operationally incomplete.

In automotive, the ISO 21448 (SOTIF — Safety of the Intended Functionality) standard explicitly incorporates scenario-based and simulation-based analysis as part of the safety argument. SOTIF was written precisely to address the gap between ISO 26262 (which handles systematic hardware failures) and the behavioral risks of autonomous systems operating in complex environments. It requires that programs define a scenario space, demonstrate coverage of known unsafe scenarios, and argue statistically about the residual risk of unknown scenarios. That is a meaningful framework, but it puts enormous weight on how the scenario space is defined and how coverage is measured — both areas where current tooling and practice are immature.

Defense autonomy programs, operating under MIL-STD and TEMP (Test and Evaluation Master Plan) frameworks, are handling this the most inconsistently. Individual program offices are developing their own simulation-based V&V approaches without consolidated guidance. The DoD’s Data-Driven Autonomy Verification (DDAV) initiative and related work through the Office of the Under Secretary of Defense for Research and Engineering have produced useful frameworks, but adoption is fragmented.

The Standards Gap Is Real and Specific

It is worth being precise about where the standards gap exists, because it is sometimes described as a general absence of guidance when the actual situation is more specific.

The structural integrity of hardware — how it fails physically — is well-covered. EMC, thermal, vibration, power quality: standards exist, test methods are specified, certification paths are clear. The gap is not in hardware environmental qualification.

The gap is in three specific areas:

Behavioral requirement completeness. Existing standards do not specify how to determine that a set of behavioral requirements for an autonomous function is sufficiently complete. For hardware, completeness is defined by physical performance bounds. For autonomous behavior, completeness would require coverage of the operational design domain — and current standards provide guidance that is conceptually correct but procedurally underspecified.

Simulation fidelity qualification. There is no agreed standard for how faithful a simulation environment must be before its outputs count as verification evidence for hardware or system certification. Fidelity thresholds, environment validation methods, and correlation requirements to physical test results are each being handled differently by different programs.

Statistical coverage adequacy. Traditional V&V has coverage measured as a binary: every requirement has a test, every test has passed. Statistical coverage — the fraction of the scenario distribution covered, at what confidence level, to what risk threshold — requires a different mathematical framework that existing standards do not specify for hardware certification.

These are not gaps that will be filled by minor updates to existing standards. DO-178C took years to revise into its current form. ISO 26262 second edition addressed significant gaps from the first. The regulatory bodies know this, and the guidance documents being produced — EASA’s AI Roadmap Level 1 and 2 guidance, the FAA’s MOSART framework for machine learning airworthiness, the draft updates to MIL-HDBK-516C — are early steps in a process that will take years to stabilize.

How Programs Are Adapting in the Interim

What are programs actually doing while the standards catch up?

The most common approach is a hybrid argument structure: deterministic hardware verification using traditional methods, supplemented by scenario-based and simulation-based evidence for autonomous behavior, with an explicit residual risk argument accepted by the certification authority or program office as a matter of engineering judgment.

This works when the certification authority is engaged early and participates in defining what evidence is acceptable. It fails — and programs have discovered this at high cost — when evidence strategies are developed in isolation and then presented to regulators who have different expectations.

The second practical adaptation is scenario library management as a first-class engineering artifact. Programs that are doing this well have moved scenario definitions into their requirements management infrastructure, with traceability from scenarios to test cases, to simulation runs, to coverage metrics. This is where the adequacy of the scenario library is visible, auditable, and improvable.

This traceability requirement is where legacy requirements management tooling struggles. Document-based RTMs were designed for one-to-one requirement-to-test relationships. Scenario-to-requirement relationships are many-to-many, conditional, and depend on parameter ranges rather than fixed values. Graph-based requirements management platforms — Flow Engineering being a current example — handle this structure natively because they model requirements and their relationships as a connected graph rather than a table. That architecture matters when a single behavioral requirement may be exercised by hundreds of scenario variants and needs coverage visibility across all of them.

The third adaptation is tighter integration between systems engineering and hardware V&V planning. In traditional hardware programs, the hardware test engineer received requirements derived from the system-level design and wrote test cases against them. In autonomous system programs, the V&V planning has to be upstream of the requirement derivation — you have to understand the scenario space before you can write testable hardware requirements that are meaningful in context.

Honest Assessment of Where This Lands

Autonomy has not made hardware V&V harder in the way that is sometimes implied — as if the physics of hardware has changed. Actuators still respond to commands. Processors still have thermal limits. Power supplies still have noise floors. Those verifications are unchanged.

What has changed is the upstream boundary of hardware V&V. The inputs that hardware must be verified against are no longer bounded by a human operator’s decision space. They are bounded by an algorithm’s decision space operating over a complex environment. Characterizing that boundary rigorously, and proving coverage of it, is a problem that the V&V community has the tools to address — but not yet the standards to require or the institutional consensus to agree on.

The programs that will navigate this most successfully are those treating the standards gap as a known engineering constraint to be managed explicitly, rather than waiting for regulatory clarity before building their V&V approach. That means early engagement with certification authorities, investment in scenario library infrastructure, and requirements management tooling that can represent the many-to-many relationships between behavioral requirements and verification evidence.

The standards will eventually catch up. They always do. But in aerospace, automotive, and defense programs running on five-to-ten-year development cycles, eventually is not a planning assumption. The engineering work has to proceed now, with the best available frameworks and explicit documentation of where engineering judgment is carrying the load.

That is uncomfortable. It is also the actual situation.