The Requirement Nobody Can Test

A spacecraft is designed for a 25-year operational life in geostationary orbit. One requirement reads: The propulsion subsystem shall maintain full thrust authority after 25 years of continuous operation in the GEO radiation environment.

No lab on Earth will run that test. No schedule accommodates it. No budget survives it. Yet the requirement is real, the system must be delivered, and a review board — or a government customer, or a safety regulator — will eventually ask for objective evidence that the requirement is closed.

This is not an exotic edge case. It is a routine problem in space, defense, nuclear, and automotive domains. Radiation tolerance requirements, extreme thermal cycling endurance, multi-decade life predictions, structural margins under mission-unique load combinations — all of them routinely exceed what any laboratory environment can replicate fully. Engineers who treat “can’t test it” as the end of the conversation are confusing one verification method with verification itself.

There are four verification methods. Test is one of them.

The Four Verification Methods

Every major systems engineering standard — MIL-STD-1916, ECSS-E-ST-10-02, NASA-STD-7009, ISO 15288 — recognizes the same four methods, even when the naming varies slightly.

Test is physical execution of the system or a representative article under controlled, measurable conditions, with direct measurement of the parameter in question. It is the most defensible method when it is achievable because it leaves the least interpretive gap between evidence and claim.

Analysis is the application of mathematical models, simulation, calculation, or logical argument to predict or bound system behavior. It is the correct method when the test environment cannot be replicated, when test would destroy the only available unit, or when the parameter cannot be measured directly.

Inspection is confirmation through visual or dimensional examination that a design feature, material, or configuration is present as specified. It applies to requirements like The unit shall not use any tin-whisker-prone pure-tin plating — requirements where presence or absence of a characteristic is the whole question.

Demonstration is functional operation under realistic but not fully controlled or instrumented conditions. It is weaker than test in metrology terms but sufficient for requirements like The crew shall be able to configure the system for safe mode within 60 seconds using only primary controls.

Each method is legitimate when applied to the right class of requirement. The error is applying test-thinking to every requirement and then treating untestable requirements as somehow unverifiable. They are not unverifiable. They require a different method, with a different evidence structure.

When Analysis Is the Right Answer

Analysis is the right verification method when one or more of the following conditions hold:

The test duration exceeds program life. A 25-year life requirement cannot wait 25 years. Accelerated life testing can partially substitute, but acceleration models themselves are analytical — which means you are doing analysis anyway, and the question is whether you are doing it explicitly and rigorously.

The environment cannot be reproduced. Deep-space radiation flux, true vacuum at 10⁻¹² torr with simultaneous UV and particle bombardment, combined multi-axis random vibration plus acoustic loading — these environments can be partially approximated in chambers, but never fully replicated. The gap between the test environment and the real environment must be closed analytically.

The system scale prevents physical test. A structural analysis of a large launch vehicle segment under flight loads may be partially demonstrated on coupon articles, but the full-scale integrated verification argument will always include finite element analysis. The coupon tests calibrate the model; the model closes the requirement.

Test would destroy the only available flight unit. When only one flight article exists and destructive testing would eliminate it, analysis backed by development test data on engineering model hardware is the standard approach.

Choosing analysis as the method is not the end of the work. It is the beginning of a more demanding documentation task.

Building a Credible Analysis-Based Verification Argument

A simulation output is not a verification closure. A finite element result, a radiation dose calculation, or a thermal margin prediction becomes a verified closure only when it is embedded in a documented argument that a technically qualified reviewer can independently evaluate.

That argument must contain five elements.

1. The requirement being closed, stated exactly. Not a paraphrase. The exact allocated requirement, including its quantitative threshold, applicable conditions, and any parent requirement it derives from. Ambiguity at this stage contaminates everything downstream.

2. The model or analysis method, with its governing assumptions. Every analysis rests on assumptions: material property values, environmental input spectra, boundary conditions, failure mode scope. These assumptions must be written down. A reviewer who cannot see the assumptions cannot evaluate the argument. Assumptions that are not documented will not be reviewed — and unreviewed assumptions are where verification failures hide.

3. The model validation and calibration record. How do you know the model is right? This question must be answered with data, not confidence. The validation record shows the test conditions under which the model’s predictions were compared to physical measurements, the agreement metrics, and the conditions under which the model is and is not considered valid. A model used outside its validation domain is an extrapolation, and that must be stated.

4. The uncertainty quantification and margin disposition. Analysis produces a predicted value with an uncertainty band. The requirement closes only when the predicted value, accounting for uncertainty, clears the requirement threshold by a margin that is itself justified. For structural margins, NASA and ESA publish standard factors. For novel analysis, the margin justification must be argued explicitly. “The model says 27 years, the requirement is 25, so we’re good” is not sufficient if the model uncertainty is ±5 years.

5. The independence and authority of the closure. Who performed the analysis? Who reviewed it? Who has authority to close the requirement on behalf of the project? An analysis that closes a safety-critical requirement and was reviewed only by its own author is not closed.

Heritage Data and Similarity Arguments

Some requirements are best verified by pointing to prior evidence from a previous program — a flight-proven component, a qualified material, a subsystem that has survived the relevant environment in service.

Heritage arguments are legitimate. They are used routinely in space and defense programs, and standards such as ECSS-E-ST-10-02C explicitly recognize similarity as a verification approach. But similarity is not identity, and a heritage argument carries specific obligations.

You must define what is the same. The component, its design revision, its manufacturing source, its lot traceability, its operational environment in the heritage program — all must be documented. Heritage that cannot be specifically identified is not heritage; it is anecdote.

You must define what is different. Every change between the heritage configuration and the current application must be identified. A part from a heritage program that has since changed its supplier, its substrate material, or its assembly process may not carry its prior qualification.

You must argue why the differences don’t matter. This is the actual substance of the similarity argument. If the new application’s radiation environment is 15% higher than the heritage program’s, you need an analysis showing that 15% delta is within the original margin. The similarity argument closes only when the delta is bounded and the bounding argument is documented.

Heritage arguments without these three elements are assertions, not evidence. A review board that accepts undocumented heritage claims is not doing its job, and an engineering team that submits them is building a verification record that will not survive scrutiny.

The Auditability Problem

Here is where many programs fail — not in the quality of the analysis, but in the accessibility of the verification argument.

A team of competent engineers produces a detailed thermal model, validates it against coupon test data, runs the mission lifetime simulation, documents the assumptions, and concludes the requirement is met. The analysis is sound. Then, six months later, a customer review board asks to see the closure evidence for that requirement. The analyst who ran the model has left the program. The model files are in a shared drive with three years of undifferentiated simulation runs. The requirement is in a DOORS database. The analysis report is in a document management system. The link between them exists only in the analyst’s memory.

This is not a tools failure. It is a process failure that tools can prevent.

Requirements management tools that support bidirectional traceability allow teams to link analysis artifacts — reports, model files, validation records, assumption logs — directly to the requirement record they close. When the closure record contains not just a “closed” status but a pointer to the specific version of the analysis report, the specific model validation test, and the specific reviewer who accepted the argument, the verification claim becomes auditable. Anyone with appropriate access can follow the chain from requirement to evidence without relying on institutional memory.

Flow Engineering is built on this model. It represents requirements and their verification evidence as nodes in a connected graph, so the relationship between a radiation tolerance requirement, the dose-depth analysis that closes it, the coupon test data that calibrates the model, and the heritage record that informs the margin assumption are all traversable links, not separate documents that must be manually reconciled. When a review board asks “show me the closure evidence for requirement PROP-087,” the answer is a structured artifact network, not a document search.

The practical value of this during reviews is significant. Programs that can respond to requirement-level questions with traceable artifact chains move through gate reviews faster, field fewer corrective action requests, and carry less risk of undiscovered gaps surfacing at delivery.

Flow Engineering also supports requirement-level rationale fields — meaning the reason a team selected analysis rather than test, and the specific analysis approach chosen, can be captured at the requirement record itself, not buried in a separate document that may or may not be retrieved in context.

Practical Starting Points

If your program has requirements that cannot be verified by direct test, the following practices will produce defensible closures.

Assign verification method at allocation. When a requirement is allocated to a subsystem, assign the verification method at the same time. If the method is analysis, document the reason. Don’t defer this to the verification phase.

Write the analysis plan before the analysis. The plan defines the model, the validation approach, the uncertainty method, and the acceptance criteria before results are known. Plans written after results are known are not plans; they are rationalizations.

Track assumption changes as configuration items. When an assumption in a closing analysis is revised — because a material property was updated, a load profile was refined, a heritage data source was reexamined — the verification closure must be re-evaluated. Assumption changes that don’t trigger closure re-evaluation are the source of latent verification gaps.

Make the heritage argument explicit and reviewable. Document the heritage item, the differences, and the difference argument as a structured artifact, not as a sentence in a report. A reviewer should be able to evaluate it without reading the entire analysis package.

Link everything to the requirement record. The closure is not the analysis. The closure is the requirement record, marked closed, with traceable pointers to every artifact that supports the closure claim.

Honest Summary

Verification by analysis is not a lesser form of verification. For a wide class of requirements — life prediction, radiation tolerance, extreme environment margins — it is the only technically honest approach available. What makes it credible is not the sophistication of the model, though that matters, but the rigor of the documented argument: explicit assumptions, validated models, bounded uncertainties, justified margins, and an auditable link between the analysis artifacts and the requirement they close.

Programs that treat untestable requirements as a paperwork exercise, producing analysis reports that exist to fill a compliance slot rather than to constitute a genuine argument, are accumulating risk that will surface during qualification, delivery, or operations. The argument is not auditable because the customer demands it. It is auditable because the requirement is real, and real requirements must be closed with real evidence — even when that evidence is analytical.