How Do You Write Testable Requirements for Reliability?

Somewhere in your requirements database, there is probably a requirement that reads something like this:

The system shall be highly reliable.

Everyone on the program knows it’s useless. The systems engineer who wrote it knows it. The test lead who has to verify it knows it. The customer who approved it suspects it. And yet, in program after program, requirements like this survive preliminary design review, critical design review, and sometimes qualification testing — at which point someone has to make a judgment call about whether the system passed.

This is not a tooling problem or a process problem at its root. It is a writing problem. Reliability requirements fail not because engineers don’t understand reliability, but because the conventions for writing testable reliability language are rarely taught explicitly. This article provides those conventions.

Why “Highly Reliable” Is Not a Requirement

A requirement exists to constrain the design space and to define an objective, binary pass/fail criterion for acceptance. “Highly reliable” does neither. It constrains nothing — a designer cannot make an architectural decision based on it — and it defines no acceptance criterion that a test team could evaluate without subjective interpretation.

The deeper problem is that reliability is not a single quantity. It is a family of related metrics, each of which captures a different aspect of system behavior, each of which is appropriate for different operational contexts, and each of which demands a different verification approach. Writing “highly reliable” is like writing “good performance” and expecting the propulsion team and the navigation team to derive independent, consistent requirements from it. They won’t.

To write a testable reliability requirement, you need to choose the right metric first.

The Three Metrics That Matter

Mean Time Between Failures (MTBF)

MTBF is the expected operating time between successive failures under a defined operating condition. It is the most commonly cited reliability metric and also the most commonly misused.

MTBF is appropriate when:

  • The system operates continuously or near-continuously
  • Failures are recoverable (the system is repaired and returned to service)
  • The failure rate is approximately constant (the useful-life portion of the bathtub curve)

A testable MTBF requirement looks like this:

The system shall demonstrate an MTBF of not less than 2,000 hours when operated in accordance with Operational Profile OP-3 (defined in Section 4.2), verified by reliability demonstration testing at a consumer’s risk of 10% and producer’s risk of 10% using a sequential test plan.

Every element of that sentence is load-bearing. The 2,000-hour threshold drives parts selection and redundancy architecture. The operational profile defines the stress environment under which the threshold applies — temperature cycling, vibration spectrum, duty cycle. The confidence terms (consumer’s and producer’s risk) define the test plan: how many hours must be accumulated, how many failures are acceptable. Remove any element and the requirement either becomes ambiguous or the test plan becomes indeterminate.

Mission Reliability

Mission reliability is the probability that a system completes a defined mission without mission-critical failure. It is the right metric when:

  • The mission has a defined duration and defined success criteria
  • Not all failures are equal — some failures are mission-critical, others are degraded-mode acceptable
  • The system is not necessarily repaired mid-mission

A testable mission reliability requirement:

The system shall achieve a mission reliability of not less than 0.95 for a 4-hour mission profile as defined in Mission Scenario MS-7, with 90% statistical confidence, verified by analysis using a validated reliability model with test-validated component failure rates.

Note the verification method is analysis, not test. For high-reliability, long-duration missions, direct test verification is often impractical — accumulating enough test hours to demonstrate 0.9999 reliability at 90% confidence would take centuries of test time. Analysis verified by component-level test data is the standard approach. The requirement must say this, or the test team will spend time trying to figure out how to verify it by test.

Availability

Availability is the fraction of time a system is in a functioning state. It combines reliability (how often failures occur) and maintainability (how quickly the system is restored after failure). The standard definition is:

Inherent Availability (Ai) = MTBF / (MTBF + MTTR)

where MTTR is mean time to repair.

Availability is the right metric for ground systems, infrastructure, and any system where the operational impact of downtime is the primary customer concern. A testable availability requirement:

The system shall achieve an operational availability (Ao) of not less than 0.995, measured over a rolling 90-day window during operational acceptance testing, using the maintenance concept defined in Maintenance Plan MP-2, excluding scheduled preventive maintenance downtime.

The exclusion clause is not a loophole — it is precision. Operational availability and inherent availability measure different things. Specifying which definition you are using and what downtime categories are included or excluded is part of writing the requirement, not a footnote to it.

The Four Components Every Reliability Requirement Needs

Regardless of which metric you choose, every reliability requirement must contain:

1. A specific, quantified metric. MTBF in hours, mission reliability as a probability, availability as a fraction. Not “high,” not “robust,” not “acceptable.”

2. A threshold value with direction. “Not less than 2,000 hours” is correct. “Approximately 2,000 hours” is not a requirement.

3. An operational profile reference. The metric is meaningless without the environment and duty cycle under which it applies. If your requirements document doesn’t have defined operational profiles, write them before writing the reliability requirements.

4. A confidence level and verification method. This is where most requirements fail. The confidence level determines the test plan. The verification method — test, analysis, demonstration, inspection — determines what artifacts the test team must produce at verification. Both must be stated in the requirement or in the associated verification cross-reference matrix, at time of writing.

Reliability Requirements vs. Reliability Goals

This distinction matters for program management, not just engineering.

A reliability goal is an internal planning target used to guide design and allocate reliability budgets across subsystems. Goals can be aspirational. A reliability goal of MTBF = 5,000 hours on a new platform drives engineers toward more conservative designs; it is not a contract threshold.

A reliability requirement is a contractual threshold. It defines pass/fail for acceptance. It constrains what the program will deliver and what the customer will accept. Requirements must be verifiable. Goals do not need to be.

The danger of conflating them: if you write a reliability goal into a requirements specification — because the goal sounds impressive and no one pushes back — you have just committed the program to either verifying an unachievable threshold or accepting a system that does not meet its stated requirement. Programs that confuse goals with requirements tend to discover the problem at system qualification, when it is expensive to fix.

The rule: goals live in reliability program plans and design-to specifications. Requirements live in system or subsystem specifications. They should be consistent, but they are not the same document and do not carry the same contractual weight.

How the Requirement Determines the Test Program

The test community has a saying: “You get what you specify.” This is never more true than for reliability.

Consider two requirements for the same system:

Requirement A: The system shall be highly reliable.

Requirement B: The system shall demonstrate an MTBF of not less than 500 hours at 80% lower confidence bound, verified by reliability demonstration testing using a time-terminated test plan with accumulated test hours as defined in MIL-HDBK-781A, Table V, operating under Operational Profile OP-1.

For Requirement A, there is no test plan. The test team must either invent one and hope the customer agrees, or declare verification by inspection against some subjective judgment.

For Requirement B, the test plan is determined before the test team writes a single test procedure. MIL-HDBK-781A Table V, given the MTBF threshold and the confidence level, specifies the required test hours and the maximum number of allowable failures. The test team’s job is execution and documentation, not interpretation.

The downstream cost of Requirement A is not just verification risk. It is program risk. A customer who accepts a system against Requirement A has accepted nothing — and both parties know it. When field reliability is poor, there is no contractual basis for remedy.

The labor required to write Requirement B — perhaps 30 additional minutes of work — eliminates months of potential verification dispute.

The Compounding Problem: Requirements That Survive Too Long

In most programs, reliability requirements are reviewed for technical adequacy during requirements reviews. But technical adequacy (is the threshold achievable?) is not the same as verifiability (can we prove the threshold was met?). A requirement can be technically reasonable and still be unverifiable as written.

The traditional process catches this late — sometimes at test planning, sometimes at formal qualification, occasionally in customer acceptance disputes. By that point, changing the requirement means a formal change request, a potential contract modification, and a test plan that may need to be redesigned.

How Modern Tools Address This at the Source

The pattern described above — untestable requirements surviving into late program phases — is largely a process failure enabled by tooling that doesn’t prompt engineers to think about verification at the time of writing.

This is where tools with explicit verification tagging change the dynamic. Flow Engineering, built specifically for hardware and systems engineering teams, treats verification method as a required attribute of every requirement, not an afterthought. When an engineer writes a requirement in Flow Engineering, they assign a verification method — test, analysis, demonstration, or inspection — at the same time. If a requirement cannot be tagged with a credible verification method, that gap is immediately visible in the traceability model.

For reliability specifically, this matters because it forces the question before the requirement is baselined: How will this be verified? A requirement tagged “verification method: TBD” is flagged rather than silently accepted. A requirement tagged “verification method: test” against an unquantified threshold creates an immediate inconsistency that surfaces in review dashboards, not in a qualification dispute six months later.

Flow Engineering’s graph-based model also makes it straightforward to link requirements to the operational profiles and test scenarios that define their context — the connection between a reliability requirement and the OP-3 operational profile it references is a live, navigable relationship, not a text cross-reference that gets stale when documents are revised.

The tool doesn’t write requirements for engineers. But it structures the writing process so that the attributes needed for verifiability — metric, threshold, operational context, verification method — are prompted rather than optional. Teams that adopt this practice consistently find fewer verification discrepancies at CDR and qualification because the requirement writing and test planning are conceptually unified from day one.

Practical Starting Points

If you are auditing existing reliability requirements for testability, apply this checklist to each one:

  • Does the requirement name a specific metric (MTBF, mission reliability, availability)?
  • Is the threshold quantified with a direction (not less than, not greater than)?
  • Is there a reference to an operational profile or operating condition?
  • Is a statistical confidence level or acceptable risk specified?
  • Is the verification method (test, analysis, demonstration, inspection) identified?
  • Is the verification method actually achievable within the program’s test resources and schedule?

Any requirement that fails more than two of these checks should be considered a draft, not a baselined requirement.

If you are writing new requirements, write the verification approach before you finalize the threshold. The threshold you can verify with your test resources may be different from the threshold you would write if verification were unconstrained. Better to know that before PDR.

The Honest Summary

“The system shall be highly reliable” is not a bad requirement because the author didn’t care about reliability. It is a bad requirement because nobody asked, at the time of writing, how it would be verified. The fix is not more rigor in the abstract — it is a concrete writing convention, applied consistently, that forces the four components of a verifiable reliability requirement into every statement before it is baselined.

The test program that verifies your reliability requirements is written in the requirements themselves. Write them accordingly.