up:: In the Protocols MOC

Hybrid TLS vs KEMTLS

Post-quantum TLS has two design answers to the same problem, which is that post-quantum cryptography is byte-heavy and a TLS handshake is size-sensitive. The deployed answer keeps the TLS 1.3 handshake exactly as it is and swaps algorithms inside it, running hybrid key exchange for confidentiality and post-quantum signatures for authentication. KEMTLS instead redesigns the handshake so the server authenticates by decapsulating a KEM ciphertext rather than by signing the transcript.

The first shipped across the internet. The second is a peer-reviewed research proposal that measures materially better and was not adopted, and understanding why is the most useful thing in the comparison.

The short version:

• They operate on different planes, and conflating them is the first error. Hybrid key exchange protects confidentiality. KEMTLS changes how the server proves identity. A deployment running X25519MLKEM768 has done nothing about authentication.
• The deployed path changes algorithms, and KEMTLS changes structure. One swaps what goes in the existing slots. The other removes the handshake signature entirely.
• KEMTLS measures better on both axes that matter. The paper reports under half the bandwidth of a size-optimized post-quantum TLS 1.3 handshake, and almost 90% fewer server CPU cycles.
• Compatibility won over efficiency. Keeping the standard handshake and swapping a single algorithm is a far smaller change to a protocol the entire internet already speaks, which is why the ecosystem went with signatures.
• The byte problem is real and already biting. A classical X25519 key_share is 32 bytes and an X25519MLKEM768 share is 1,216 bytes, which pushes the ClientHello past a boundary a long tail of middleboxes was built assuming it would never cross.

Two ways to fit a much larger document into an envelope system designed decades ago for thin letters. The first keeps every envelope, form and sorting machine exactly as they are and accepts that some letters now need two envelopes, then discovers that certain old sorting machines jam on anything arriving in two parts. The second redesigns the form so the same information fits on one sheet again, which works beautifully and requires every post office in the world to retrain on a new form. The first shipped, because retraining every post office is the harder problem.

What is the difference between hybrid TLS and KEMTLS?

The difference is whether the handshake’s structure changes. Deployed post-quantum TLS keeps the TLS 1.3 handshake intact: the client and server negotiate a key-exchange group, the server presents a certificate, and the server signs the transcript to prove it holds the certificate’s private key. Post-quantum migration happens by putting new algorithms into those existing slots, a hybrid group for the key exchange and eventually a post-quantum signature for the proof.

KEMTLS removes the server’s handshake signature entirely. The certificate carries a long-term KEM public key rather than a signature-verification key, the client encapsulates a secret to it, and the server proves identity by successfully decapsulating. Authentication becomes a property of the derived keys rather than a discrete verification step.

What is deployed post-quantum TLS?

The deployed path covers confidentiality first through hybrid key exchange, which pairs a classical curve with ML-KEM inside a single negotiated group:

Group	Codepoint	Classical part	Post-quantum part	Recommended
X25519MLKEM768	0x11EC (4588)	X25519	ML-KEM-768	Yes
SecP256r1MLKEM768	0x11EB (4587)	secp256r1 (NIST P-256)	ML-KEM-768	No
SecP384r1MLKEM1024	0x11ED (4589)	secp384r1 (NIST P-384)	ML-KEM-1024	No

Source: RFC 10024, “Post-Quantum Traditional (PQ/T) Hybrid Key Agreement Mechanisms for TLS 1.3,” Standards Track, August 2026 (formerly draft-ietf-tls-ecdhe-mlkem), datatracker.ietf.org, §3, and the IANA TLS Supported Groups registry.

Authentication follows on a deliberately slower track, through post-quantum signatures such as ML-DSA carried in composite certificates and the post-quantum signature RFCs. A 2026 deployment commonly runs X25519MLKEM768 for its keys while presenting a fully classical certificate.

What is KEMTLS?

KEMTLS is a redesign of the TLS 1.3 authenticated handshake published as “Post-Quantum TLS Without Handshake Signatures,” which substitutes key-encapsulation mechanisms for signatures in server authentication. The certificate is still issued and signed by a certificate authority in the usual way, so the expensive post-quantum signature moves out of the live handshake and up into the certificate chain, computed once at issuance rather than on every connection.

The motivation is a size asymmetry the paper states directly: among post-quantum candidates, signature schemes generally have larger public key and signature sizes than the public key and ciphertext sizes of KEMs. Authenticating by encapsulation therefore carries a compact KEM ciphertext where authenticating by signing carries a large post-quantum signature.

Source: P. Schwabe, D. Stebila, T. Wiggers, “Post-Quantum TLS Without Handshake Signatures,” ACM CCS 2020, ePrint 2020/534.

Hybrid TLS vs KEMTLS at a glance

Dimension	Deployed post-quantum TLS 1.3	KEMTLS
What changes	The algorithms inside existing handshake slots	The structure of the authenticated handshake
Server authentication	A post-quantum or hybrid signature over the transcript	KEM encapsulation to a KEM key in the certificate
What the server does per handshake	Signs the transcript	Decapsulates a client ciphertext
Authentication style	Explicit. The client verifies a signature and it passes or fails on the spot	Implicit. Derived keys only agree if the server decapsulated correctly
What the certificate carries	A signature-verification key	A long-term KEM public key
Where the signature cost is paid	On every connection	Once, at certificate issuance
Confidentiality mechanism	Hybrid key exchange, X25519MLKEM768 and siblings	A KEM, in the same handshake
Handshake bandwidth	Grows with post-quantum signature size	Under half a size-optimized post-quantum TLS 1.3 handshake
Server CPU	Signature generation per handshake	Almost 90% fewer server CPU cycles, per the paper
Standardization	Standardized handshake, with hybrid groups in an IETF draft and IANA codepoints assigned	A peer-reviewed research proposal, not an IETF standard
Deployed today	Yes, on by default across browsers and CDNs	No
Main advantage	Reuses a protocol the whole internet already speaks	Cuts handshake bytes and server work substantially
Main obstacle	ClientHello size against ossified middleboxes	Requires every TLS implementation and certificate authority to adopt a new structure

How do they actually differ?

1. One is an algorithm substitution and the other is a protocol change. Swapping a signature algorithm inside TLS 1.3 touches cipher negotiation and certificate contents while leaving the message flow identical. KEMTLS changes what the server does to prove identity, which changes the state machine, the security proof and every implementation.

2. Explicit and implicit authentication fail differently. In standard TLS the client runs a verification that returns pass or fail at a known point. In KEMTLS there is no such check, and a wrong server simply derives different keys, so the failure surfaces later in the handshake as a key mismatch. That is provably secure and it is a different debugging and operational experience.

3. The certificate contents diverge. Deployed TLS puts a signature-verification key in the leaf certificate. KEMTLS puts a KEM public key there, which means certificate authorities, certificate profiles, trust stores and every validation library have to understand a certificate type that authenticates by encapsulation.

4. They pay the post-quantum signature cost at different times. Deployed TLS pays it on every handshake, which is why post-quantum signature size is the dominant concern in TLS sizing work. KEMTLS pays it once at issuance, moving the cost off the hot path entirely.

5. Their obstacles sit in different places. The deployed path’s obstacle is external, in middleboxes neither endpoint controls. KEMTLS’s obstacle is internal to the ecosystem, requiring coordinated adoption across implementations and certificate authorities, which is the harder of the two coordination problems.

Where do they agree?

1. On the diagnosis. Both exist because post-quantum authentication is byte-heavy, and both are engineered responses to that single fact.

2. On keeping certificate authorities in the loop. KEMTLS does not remove signatures from the trust chain. Certificate authorities still sign leaf certificates in the usual way, so the chain above the leaf is unchanged.

3. On using a KEM for confidentiality. Both derive session secrets from ML-KEM-class encapsulation, and they differ on whether a second mechanism proves identity.

4. On the underlying standards. Both build on the NIST post-quantum standards rather than on new primitives, so the algorithm layer is settled in each case.

5. On what neither fixes. Neither addresses the certificate hierarchy above the leaf, so a handshake protected by either still chains up through intermediates and roots that need their own migration.

Why does the ClientHello size matter so much?

Because the ML-KEM key inflates the first message the client sends past a boundary the internet quietly hardened around. A classical X25519 key_share is 32 bytes and an X25519MLKEM768 share is 1,216 bytes, and for most of TLS’s history the ClientHello fit inside a single network packet. A long tail of middleboxes, load balancers and inspection appliances was built assuming it always would, and some of that equipment mishandles or drops a ClientHello it cannot buffer.

This is protocol ossification, where an ecosystem calcifies around an assumption that later stops being true. Cloudflare, testing hybrid at internet scale, found sharp cliffs where connections failed once the handshake grew past specific sizes, which is evidence that particular middleboxes cannot process handshakes beyond a fixed length. The weak point is a third box on the path that neither endpoint controls.

The standards left an escape hatch built from the negotiation itself. A client can advertise a hybrid group while sending only a small classical key_share, and a server wanting hybrid responds with a HelloRetryRequest asking for the full share on a second round trip. That trades a round trip for surviving the middleboxes.

Source: Cloudflare, “Defending against future threats: Cloudflare goes post-quantum”.

Why did the ecosystem choose signatures over KEMTLS?

Compatibility, and the reasoning is worth stating plainly because KEMTLS measures better. Keeping the standard TLS 1.3 handshake and swapping only the signature algorithm is a much smaller change to a protocol the entire internet already speaks. It deploys through machinery that already exists: negotiation, certificate profiles, and the libraries and CDNs that ship them.

KEMTLS asks for a new handshake structure, a new certificate type and coordinated adoption across implementations and certificate authorities simultaneously. That is a coordination problem rather than an engineering one, and coordination problems at internet scale are where better designs usually lose.

KEMTLS remains valuable as the clearest statement of the tradeoff, and as a design worth revisiting for constrained settings where handshake byte count dominates everything else.

Common misconceptions

1. “Hybrid key exchange makes my TLS connection post-quantum.” It protects confidentiality. The server still proves identity with a certificate signed by a classical algorithm in most 2026 deployments, and a quantum computer breaks those signatures too.

2. “KEMTLS is the standard for post-quantum TLS.” It is a peer-reviewed research proposal that was not adopted by the IETF. Production systems keep the TLS 1.3 handshake and migrate the algorithms inside it.

3. “KEMTLS removes certificate authorities.” It moves the signature out of the live handshake and up into the certificate chain, where a certificate authority still signs the leaf at issuance.

4. “The two are alternatives for the same job.” Hybrid key exchange addresses confidentiality and KEMTLS addresses server authentication, so the comparison is KEMTLS against post-quantum signatures rather than against the hybrid group.

5. “Post-quantum TLS fails because the algorithms are slow.” The dominant obstacle is size rather than speed, specifically the ClientHello crossing a packet boundary that ossified middleboxes cannot handle.

6. “If KEMTLS is better, it will eventually win.” Better measurements have not been the deciding factor. The deployed path won on requiring less coordinated change, and that advantage grows rather than shrinks as the installed base migrates.

Questions people ask

Is KEMTLS deployed anywhere in production? No. It is published and peer-reviewed at a major academic venue and was not adopted as an IETF standard, and production systems use the standard TLS 1.3 handshake with post-quantum or hybrid algorithms inside it.

How much better is KEMTLS? The paper reports a size-optimized KEMTLS handshake requiring under half the bandwidth of a size-optimized post-quantum TLS 1.3 handshake, and a reduction of almost 90% in server CPU cycles, because the server decapsulates rather than signs.

What does hybrid key exchange actually protect? The confidentiality of the session, which is the harvest-now-decrypt-later-urgent half, because recorded traffic becomes decryptable retroactively once a capable quantum computer exists. Authentication is a separate plane on a slower timeline.

Which hybrid group should be running? X25519MLKEM768 is the one the draft calls the most practical choice and the one shipped on by default across browsers and CDNs. The P-curve variants exist for organizations bound by a profile such as CNSA 2.0 that specifies the NIST curves.

Why does a bigger handshake break connections? Because the ClientHello grows past a single packet, and a long tail of middleboxes was built assuming it never would. Some of that equipment drops a handshake it cannot buffer, and neither endpoint controls the offending device.

What is implicit authentication? Authentication that is a property of the derived keys rather than a discrete verification step. In KEMTLS the client encapsulates a secret to the server’s KEM key, and the session keys only match if the server decapsulated with the correct private key.

Does my server signature need to be post-quantum today? It is the slower track by design, because signature forgery requires a live quantum computer while recorded traffic is exposed retroactively. The certificate ecosystem of authorities, trust stores and hardware modules is years behind the key-exchange side.

Should a cryptographic inventory track these separately? Yes. A CBOM that treats a key-exchange-migrated server as finished will record it as done while its identity is still classically forgeable, so the two planes have to be inventoried apart.

Could KEMTLS still be adopted later? It remains a design worth revisiting for constrained settings where handshake byte count dominates everything else. Broad adoption would need the same coordinated change across implementations and certificate authorities that kept it out the first time.

---

The map is free and I keep it that way. When the question becomes which of these planes an actual estate has migrated and what its handshakes really negotiate, that’s the work I do at LaMarr Labs.

Go deeper

• TLS 1.3 Hybrid Key Exchange and KEMTLS for each approach in full
• X25519MLKEM768 and the TLS hybrid named groups for the deployed groups
• Composite Certificates and Dual Signatures for the authentication track
• PQC Performance and Size Overhead for the byte problem across protocols
• The Two-Lane Split for why confidentiality ships before authentication
• In the Protocols MOC for applied post-quantum cryptography across protocols

Last verified 2026-08-10 · Maintained by Addie LaMarr, LaMarr Labs.