up:: Migration Architecture MOC

Testing as the Control Surface

Testing as the control surface is the principle that, in a post-quantum migration, the thing that actually delivers security is validating the deployment, not shipping the new algorithm. Swapping RSA or ECC for ML-KEM or ML-DSA is a configuration change; whether that change protects anything is decided by whether the two endpoints still negotiate, whether the larger keys survive every buffer and field they pass through, whether the implementation leaks the key through a physical channel, and whether an attacker can quietly force the connection back to the old algorithm. Post-quantum cryptography introduces new failure modes that classical crypto never had, and none of them show up in “we deployed ML-KEM.” They show up in a test.

The short version:

1. Deploying the post-quantum algorithm is the claim. Testing is what turns the claim into a fact, so testing is where the real security control lives.
2. PQC breaks size assumptions that held for thirty years. ML-KEM and ML-DSA keys and signatures are kilobytes where RSA and ECC were tens of bytes, and that inflation breaks handshakes, buffers, certificate fields, and middleboxes that no one tests until it fails.
3. The failure modes only testing catches are interoperability and negotiation, oversized-message fragmentation, data-flow truncation, side-channel leakage, and silent downgrade. A correct algorithm passes all five only if the deployment is validated for them.
4. Formal validation programs exist for exactly this reason. CAVP and ACVP test that the algorithm computes correctly, CMVP tests the whole module, and ISO/IEC 17825 tests the physical implementation for leakage.
5. Testing has limits worth naming. A green test proves the paths you exercised, and the estate always has paths you didn’t, which is why layered evidence beats any single test.

Think of re-keying a building with a new master-key system. Cutting the new keys is trivial. What tells you the building is actually secure is walking every door with the new key in hand: does it turn, does the deadbolt still throw, does the fire door still release, can someone with the old key still get in. Nobody would call the job done because the locksmith delivered a box of correctly-cut keys. Testing as the control surface is that walk, applied to an organization’s cryptography, and it’s the step a migration skips at its own peril.

What does “testing as the control surface” mean?

It means the security of a post-quantum migration is produced at the testing step, not at the deployment step, so testing is the surface where the control actually acts. A control surface is the place where you steer the outcome. For classical cryptography that place felt like the algorithm choice, because a correctly-configured RSA or ECDSA endpoint almost always just worked with every other endpoint, and the keys were small enough to fit anywhere without thought. The move to post-quantum cryptography relocates the control surface, because the algorithm is standardized and well-studied, and the thing that now decides whether a deployment is safe is everything around the algorithm.

Two facts make this concrete:

1. The cryptographic move is close to solved. NIST finalized ML-KEM, ML-DSA, and SLH-DSA in August 2024, and the major libraries and protocols shipped them.
2. The deployment is the unsolved part, because post-quantum primitives are large and behave differently on the wire, and a running system has dozens of places where that difference can turn a mathematically-safe algorithm into a broken or insecure connection.

The gap between “the algorithm is correct” and “the deployment is secure” is exactly the gap that testing closes.

Source: NIST, “NIST Releases First 3 Finalized Post-Quantum Encryption Standards,” August 13, 2024, nist.gov.

This is why discovery and agility get you to the starting line and testing gets you across it. You can inventory every algorithm and make every one swappable, and you still don’t know the migration is safe until you’ve validated that the new algorithm works end to end, under real conditions, against a real adversary model, in your actual estate.

Why isn’t shipping the new algorithm the same as being secure?

Because the algorithm is the smallest and best-understood part of the system, and the failure modes live in the parts around it. A post-quantum algorithm can be implemented byte-for-byte correctly and still produce an insecure or non-functional deployment for reasons that have nothing to do with the math:

1. The two endpoints negotiate a classical algorithm instead of the post-quantum one, because one side’s configuration or a middlebox on the path quietly preferred the old suite.
2. The larger post-quantum key or signature overflows a buffer, truncates in a database column, or exceeds a certificate field sized for classical keys, so it corrupts or drops.
3. A network device that assumed handshakes stay small drops the oversized message, and the connection fails for users behind that device while working fine in the lab.
4. The implementation leaks the key through timing, power, or cache behavior, so the connection is confidential against the math and transparent against an attacker with side-channel access.
5. An attacker strips the post-quantum option from the negotiation and forces both sides back to quantum-vulnerable cryptography, and nothing in the logs looks wrong.

Every one of those is invisible to “we deployed ML-KEM” and visible to a test. That’s the whole argument. A migration measured by deployment reports a number that feels like progress and hides five ways the deployment can be unsafe. A migration measured by testing reports whether the thing actually holds.

What new failure modes does post-quantum cryptography introduce?

Post-quantum cryptography introduces five failure classes that classical deployments rarely hit, almost all of them downstream of one root cause: post-quantum keys and signatures are large. An ECDSA P-256 public key is 65 bytes and its signature is about 64; an ML-DSA-65 public key is 1,952 bytes and its signature is 3,309, and an ML-KEM-768 encapsulation key is 1,184 bytes with a 1,088-byte ciphertext. That is a one-to-two-order-of-magnitude jump landing inside protocols, buffers, and hardware that were tuned around the small classical sizes. The table below is the map of what breaks, and it doubles as the test matrix a migration validates against.

Failure mode	What actually goes wrong	Why a correct algorithm still fails	What testing catches it
Interoperability and negotiation	A post-quantum endpoint and a classical (or differently-configured post-quantum) endpoint fail to agree, or fall back to classical crypto	The two sides support different hybrid code points, extension versions, or algorithm identifiers, so the handshake mismatches	Cross-implementation interoperability testing against every counterpart in the estate
Oversized-message fragmentation	A handshake carrying a post-quantum key share splits across packets, and a middlebox drops or fails to reassemble it	Decades of network devices assumed a handshake fits in one small message; a 1KB+ key share crosses that assumption	At-scale testing across real network paths, beyond two clean endpoints
Data-flow truncation	A large key, ciphertext, or signature is silently truncated or rejected by a buffer, field, log, or storage column sized for classical crypto	Field widths, HSM object slots, and certificate structures were sized for tens of bytes, not thousands	Tracing the oversized value through every hop, store, and API boundary
Side-channel leakage	The implementation leaks the secret key through timing, power, EM, or cache behavior	Quantum-safe math says nothing about a leaky implementation; lattice schemes have secret-dependent steps	Side-channel validation (constant-time checks, leakage assessment, ISO/IEC 17825)
Silent downgrade	An attacker or a misconfiguration forces the connection back to a quantum-vulnerable algorithm without either party noticing	Negotiation prefers a classical suite, or a stripped extension removes the post-quantum option, and the session still succeeds	Downgrade and negotiation-policy testing with a minimum-floor check

Sources: NIST, “Module-Lattice-Based Key-Encapsulation Mechanism Standard,” FIPS 203, August 2024, FIPS 203; NIST, “Module-Lattice-Based Digital Signature Standard,” FIPS 204, August 2024, FIPS 204.

The through-line is that four of the five modes are size-driven and the fifth is implementation-driven, and none of them is an algorithm defect. They are deployment defects, which is precisely why the algorithm can be perfect and the migration still unsafe.

How does the oversized-message problem actually break a connection?

It breaks a connection when a post-quantum handshake grows past a size that some device on the path silently assumed it would never exceed, and that device drops the larger message. This is the most-cited real-world PQC deployment problem, and it fails in a way that lab testing between two clean endpoints never reveals.

The mechanics differ by transport, and both matter:

1. Over TCP/TLS, a hybrid key share carrying a post-quantum key pushes the first flight of the handshake past a kilobyte, and legacy middleboxes, load balancers, or inspection devices on the path can refuse or mangle the larger message. Cloudflare, testing hybrid post-quantum key exchange at internet scale, found sharp cliffs where connections failed once the handshake grew past certain sizes, caused by devices neither endpoint controls. Two perfectly-configured post-quantum endpoints still fail if a box between them won’t pass the bigger handshake.
2. Over QUIC, the problem is structural in the protocol. Before it has validated the client’s address, a QUIC server may send at most three times the bytes it received, an anti-amplification limit written into the transport standard. A client Initial packet has a 1,200-byte floor, so the server’s early budget is roughly 3,600 bytes, and a post-quantum certificate chain plus key material can blow through that budget, forcing extra round trips or outright failure.

Sources: Cloudflare, “Defending against future threats, Cloudflare goes post-quantum,” blog.cloudflare.com; IETF, “QUIC, A UDP-Based Multiplexed and Secure Transport,” RFC 9000 §8.1, rfc-editor.org.

The lesson the size cliffs teach is that a protocol is only as safe as the weakest device on its path, and you learn where that device is by testing across real paths rather than by confirming two endpoints agree in isolation. NIST’s own practice guide made this a first-class activity: SP 1800-38 Volume C is dedicated to interoperability and performance testing of draft post-quantum algorithms inside real protocols precisely so the industry finds these breaks in a lab instead of in production.

Source: NIST NCCoE, “Migration to Post-Quantum Cryptography, Volume C,” nccoe.nist.gov.

How do you test a post-quantum deployment?

You test it in four layers, because no single test sees every failure mode, and the layers stack from the algorithm outward to the adversary. The layers, in the order they build on each other:

1. Known-answer and functional testing (the algorithm computes correctly). A known-answer test feeds the implementation fixed inputs with pre-computed expected outputs and confirms they match, which is how you prove the primitive itself is right before you trust anything built on it. NIST operationalizes this through the Cryptographic Algorithm Validation Program and its Automated Cryptographic Validation Protocol (ACVP), where a validation server issues test cases and checks the responses. For ML-KEM, ACVP defines an Algorithm Functional Test that has the implementation generate a key pair from a supplied seed and validates the result against the server. These same known-answer tests run again inside a deployed module as power-up self-tests, a FIPS 140-3 requirement.
2. Interoperability testing (the two endpoints agree, at scale). You test each post-quantum implementation against every counterpart it has to talk to, and you do it across real network paths rather than only two clean endpoints, so the negotiation, the extension versions, the hybrid code points, and the oversized handshakes all get exercised against the middleboxes that will actually carry them.
3. Side-channel validation (the implementation doesn’t leak). You test that the running code is constant-time and doesn’t leak the key through timing, power, EM, or cache behavior, using constant-time analysis and leakage assessment. This is a first-class concern for post-quantum schemes because lattice designs have secret-dependent steps, and FIPS 203 itself requires implementations to destroy intermediate values because those values can compromise the key. The deeper mechanics live in Side-Channel Analysis.
4. Downgrade and negotiation-policy testing (the connection can’t be pushed back). You test that an attacker or a misconfiguration can’t force the endpoints onto a quantum-vulnerable algorithm and have the session still succeed, which means checking that a minimum-floor policy is enforced and that downgrade protections actually fire.

Sources: NIST, “Cryptographic Algorithm Validation Program,” csrc.nist.gov; NIST/IWG ACVP, “ACVP ML-KEM JSON Specification,” pages.nist.gov/ACVP; NIST FIPS 203, §3.3, FIPS 203.

The layers are cumulative. A known-answer test proves the primitive; it says nothing about whether two products interoperate. Interoperability testing proves the handshake completes; it says nothing about whether the implementation leaks. Side-channel validation proves the code doesn’t leak; it says nothing about whether an attacker can downgrade the negotiation. A deployment is only as validated as the weakest layer you skipped.

How does downgrade testing actually work?

Downgrade testing works by attempting the attack you’re defending against, forcing the negotiation toward the classical algorithm, and confirming the deployment either refuses or gets caught. A downgrade is dangerous exactly because a successful session is the normal outcome, so nothing looks wrong unless you specifically test for it.

The protocol layer gives you a concrete thing to verify. TLS 1.3 carries a downgrade-protection mechanism: a TLS 1.3-capable server that ends up negotiating an older version must set the last eight bytes of its server-random to a fixed sentinel value (the ASCII string DOWNGRD followed by a version marker), and a TLS 1.3 client that sees that sentinel while being told it’s on an older version must abort the handshake. Testing confirms both halves are live, that the server emits the sentinel and the client actually checks it, because implementations have shipped that failed to verify it and silently accepted downgrades.

Source: IETF, “The Transport Layer Security (TLS) Protocol Version 1.3,” RFC 8446 §4.1.3, rfc-editor.org.

For a post-quantum migration the same idea moves up a level: the test is whether an adversary who removes the post-quantum key-exchange option, or offers only classical suites, can drive both endpoints onto quantum-vulnerable cryptography with the session still completing. A crypto-agile system that can negotiate is only safe if its policy has a floor below which it refuses to go, and a negotiated system without that floor is downgradeable by construction.

What do the formal validation programs test?

The formal programs test three different things, and keeping them straight is the difference between “we ran a test” and “we validated the deployment.” They stack from the narrowest to the broadest:

1. ACVP tests the algorithm. The Cryptographic Algorithm Validation Program, driven by the Automated Cryptographic Validation Protocol, proves that one algorithm implementation produces correct outputs in isolation. It’s necessary and narrow: it says the primitive is right, and nothing about the module around it.
2. CMVP tests the module. The Cryptographic Module Validation Program validates the whole assembled module against FIPS 140-3, including its interfaces, key management, self-tests, and physical protection. CAVP is a prerequisite for CMVP, so the module can’t be validated until its algorithms are, and the CMVP certificate is the one a federal buyer actually checks.
3. ISO/IEC 17825 tests the physical implementation for leakage. ISO/IEC 17825:2016, “Testing methods for the mitigation of non-invasive attack classes against cryptographic modules,” specifies how a lab checks a module for side-channel leakage, built on the Test Vector Leakage Assessment (TVLA) statistical method. It feeds into FIPS 140-3’s non-invasive-security requirements through NIST SP 800-140F, which supplies the U.S. test metrics for that area.

Sources: NIST, “Cryptographic Algorithm Validation Program,” csrc.nist.gov; NIST, “Cryptographic Module Validation Program,” csrc.nist.gov; ISO/IEC 17825:2016, iso.org; NIST SP 800-140F, csrc.nist.gov.

The programs matter because they’re independent, repeatable, and federally recognized, which is what turns “we tested it ourselves” into evidence someone else can rely on. They also have a timing consequence for migration planning: a correct algorithm is not deployable in a FIPS-bound estate until it ships inside a CMVP-validated module, and that validation lags the code, sometimes by more than a year. The full mechanics of that lag are in FIPS 140-3.

What can testing not catch?

Testing catches the failure modes on the paths you exercise, and every real estate has paths you didn’t, so a passing test is evidence and not a guarantee. Naming the limits is part of using testing honestly:

1. Coverage gaps. A test proves the behavior of the configurations, endpoints, and network paths you actually ran. The vendor product you couldn’t instrument, the firmware you couldn’t reach, and the middlebox you didn’t route through are untested, and untested is not the same as safe.
2. The limits of leakage assessment. ISO/IEC 17825’s TVLA-based approach is the standardized method, and academic analysis has shown that passing it does not prove a module is free of all side-channel leakage, only that it survived a specific statistical test regime. A green side-channel result narrows the risk; it doesn’t zero it.
3. Point-in-time vs. living systems. A test validates the system as it was when you tested it. New versions, new endpoints, rotated policies, and vendor updates all move the deployment off its validated state, which is why testing is a cadence rather than an event, the same way a CBOM is a living document rather than a snapshot.
4. Test-environment realism. A lab that doesn’t reproduce the production network’s middleboxes, MTUs, and load will miss exactly the oversized-message failures that only appear at scale, which is the whole reason at-scale testing exists.

Source: Whitnall and Oswald, “A Critical Analysis of ISO 17825,” ASIACRYPT 2019, IACR ePrint 2019/1013.

The honest posture is that testing raises confidence and layered testing raises it further, and no amount of it substitutes for knowing your estate. That’s why testing rides on top of discovery rather than replacing it.

Common misconceptions

1. “We deployed ML-KEM, so we’re post-quantum secure.” Deployment is the claim; testing is the proof. A correct algorithm can still fail on interoperability, oversized messages, data-flow truncation, side-channel leakage, or silent downgrade, and only testing surfaces those.
2. “If it works in the lab, it works in production.” The most common PQC deployment failure is a middlebox on a real network path dropping an oversized handshake that two clean lab endpoints exchange without trouble. At-scale testing exists because lab success doesn’t predict path success.
3. “A known-answer test means the deployment is validated.” A known-answer test proves the primitive computes correctly. It says nothing about whether two products interoperate, whether the implementation leaks, or whether the connection can be downgraded. Those are separate tests.
4. “CAVP validation means we’re FIPS compliant.” CAVP validates one algorithm; the compliance gate is the whole-module CMVP certificate, and CAVP is only a prerequisite for it. A CAVP-tested algorithm in an unvalidated module is not FIPS 140-3 validated.
5. “Constant-time code passed review, so there’s no side-channel risk.” Constant-time review closes timing and cache leaks and does nothing about power and EM emissions, which need masking and hardware measures and independent leakage assessment. And passing the standardized leakage test narrows the risk without eliminating it.
6. “A successful handshake means there was no downgrade.” A downgrade’s whole danger is that the session succeeds normally. You only know the connection couldn’t be forced onto quantum-vulnerable crypto if you tested the downgrade and confirmed the floor held.

Questions people ask

Isn’t testing just QA? Why call it a security control? Because in a post-quantum migration the security outcome is decided at the testing step, not the deployment step. The algorithm is standardized and correct; whether the deployment protects anything depends on interoperability, size handling, leakage, and downgrade resistance, all of which are established by tests. That makes testing the surface where the control acts, not a box you check after the real work.

What’s the difference between a known-answer test and interoperability testing? A known-answer test feeds one implementation fixed inputs and checks it produces the expected outputs, proving the primitive is correct in isolation. Interoperability testing runs two independent implementations against each other across real paths to prove they negotiate and complete a handshake. The first validates the math; the second validates the deployment. You need both.

Why do post-quantum handshakes break connections that classical ones didn’t? Because post-quantum keys and signatures are kilobytes where classical ones were tens of bytes, and that size increase pushes handshakes past limits that network devices, buffers, and protocol budgets silently assumed would never be crossed. A device that drops the larger message breaks the connection even when both endpoints are configured perfectly.

Do I have to test against every device on the network path? In practice you test across representative real paths at scale rather than every device individually, because the failures cluster at middleboxes, load balancers, and inspection devices that assume small handshakes. The point NIST’s SP 1800-38 interoperability work makes is that finding these in a lab that mirrors production beats finding them when users can’t connect.

How do I test for a downgrade attack? You attempt the downgrade: strip or de-prioritize the post-quantum option, offer only classical suites, and confirm the deployment either refuses the connection or trips its protection. At the TLS 1.3 layer that includes verifying the downgrade sentinel is both emitted and checked; at the migration layer it means confirming a minimum-algorithm floor is enforced so a negotiated system can’t be pushed below it.

Does a FIPS 140-3 validation cover side-channel testing? Partly. FIPS 140-3 includes a non-invasive-security requirement area whose test methods draw on ISO/IEC 17825 and NIST SP 800-140F, so higher-assurance validations do assess leakage. A baseline software-module validation may not exercise it deeply, so side-channel resistance is worth confirming independently rather than assuming the certificate covers it.

Can testing ever prove my migration is fully secure? No, and it’s healthier to treat it as raising confidence than as a guarantee. A test validates the paths you exercised, and every estate has vendor products, firmware, and network paths you couldn’t fully reach. Layered testing plus current discovery is how you keep the untested surface small and known rather than large and invisible.

---

Everything here is the map, given freely. When your team needs its post-quantum deployment tested for real, interoperability across the estate, oversized-message handling, side-channel resistance, and downgrade protection validated against your actual systems, that’s the work I do, and there’s an alignment briefing for it.

Last verified 2026-07-09 · Maintained by Addie LaMarr, LaMarr Labs.