Key encapsulation mechanism

The difference between a public-key encryption scheme and a KEM is that a public-key encryption scheme allows a sender to choose an arbitrary message from some space of possible messages, while a KEM chooses a short secret key at random for the sender.^[1][2][3]

The sender may take the random secret key produced by a KEM and use it as a symmetric key for an authenticated cipher whose ciphertext is sent alongside the encapsulation to the receiver. This serves to compose a public-key encryption scheme out of a KEM and a symmetric-key authenticated cipher in a hybrid cryptosystem.^[1][2][3][5]

Most public-key encryption schemes such as RSAES-PKCS1-v1_5, RSAES-OAEP, and Elgamal encryption are limited to small messages^[6][7] and are almost always used to encrypt a short random secret key in a hybrid cryptosystem anyway.^[8][9][5] And although a public-key encryption scheme can conversely be converted to a KEM by choosing a random secret key and encrypting it as a message, it is easier to design and analyze a secure KEM than to design a secure public-key encryption scheme as a basis. So most modern public-key encryption schemes are based on KEMs rather than the other way around.^[10][5]

A KEM consists of three algorithms:^{[1][2][3][11][12]}

1. Key generation, ${\displaystyle ({\mathit {pk}},{\mathit {sk}}):=\operatorname {Gen} ()}$, takes no inputs and returns a pair of a public key ${\displaystyle {\mathit {pk}}}$ and a private key ${\displaystyle {\mathit {sk}}}$.
2. Encapsulation, ${\displaystyle (k,c):=\operatorname {Encap} ({\mathit {pk}})}$, takes a public key ${\displaystyle {\mathit {pk}}}$, randomly chooses a secret key ${\displaystyle k}$, and returns ${\displaystyle k}$ along with its encapsulation ${\displaystyle c}$.
3. Decapsulation, ${\displaystyle k':=\operatorname {Decap} ({\mathit {sk}},c')}$, takes a private key ${\displaystyle {\mathit {sk}}}$ and an encapsulation ${\displaystyle c'}$, and either returns an encapsulated secret key ${\displaystyle k'}$ or fails, sometimes denoted by returning ${\displaystyle \bot }$ (called "bottom").

In the asymptotic setting of theoretical cryptography, the algorithms are all probabilistic polynomial-time in a security parameter ${\displaystyle \lambda }$, and the length of the secret key ${\displaystyle k}$ is a function of the security parameter ${\displaystyle \lambda }$.^[1][2]

In practical cryptography, the secret key ${\displaystyle k}$ is usually of a fixed length for each algorithm. For example, ML-KEM always uses 256-bit secret keys,^{[4]: § 3.3, p. 16} while the algorithms in RFC 9180 vary between 256-, 384-, and 512-bit secret keys;^{[5]: § 7.1} secret keys of arbitrary length can be derived from ${\displaystyle k}$ by a key derivation function.^{[13]: § 5.3 [5]}

Decapsulation can fail because its input ${\displaystyle c'}$ is not an encapsulation ${\displaystyle c}$ returned by Encap, but has been tampered with or maliciously crafted. KEMs which report failure by a distinguished symbol ${\displaystyle \bot }$ (implemented in practice by returning an error code or raising an exception) are said to use explicit rejection. A KEM may instead return a random secret key in this event, or a secret key derived pseudorandomly from ${\displaystyle c'}$ under the key ${\displaystyle sk}$; this is called implicit rejection.^{[14]: § 5.3, pp. 76–78 [12]}

A KEM is correct if, for any key pair ${\displaystyle ({\mathit {pk}},{\mathit {sk}})}$ generated by ${\displaystyle \operatorname {Gen} }$, decapsulating an encapsulation ${\displaystyle c}$ returned by ${\displaystyle (k,c):=\operatorname {Encap} ({\mathit {pk}})}$ with high probability yields the same key ${\displaystyle k}$, that is, ${\displaystyle \operatorname {Decap} ({\mathit {sk}},c)=k}$.^{[2][3][11][12]}

Security of a KEM is quantified by its indistinguishability against adaptive chosen-ciphertext attack, IND-CCA, which is loosely how much better an adversary can do than a coin toss to tell whether, given a random key and an encapsulation, the key is encapsulated by that encapsulation or is an independent random key.^{[2][3][11][12][1]}

Specifically, in the IND-CCA game:

1. The key generation algorithm is run to generate ${\displaystyle ({\mathit {pk}},{\mathit {sk}}):=\operatorname {Gen} ()}$.
2. ${\displaystyle {\mathit {pk}}}$ is revealed to the adversary.
3. The adversary can query ${\displaystyle \operatorname {Decap} ({\mathit {sk}},c')}$ for arbitrary encapsulations ${\displaystyle c'}$ of the adversary's choice.
4. The encapsulation algorithm is run to randomly generate a secret key and encapsulation ${\displaystyle (k_{0},c):=\operatorname {Encap} ({\mathit {pk}})}$, and another secret key ${\displaystyle k_{1}}$ is generated independently at random.
5. A fair coin is tossed, giving an outcome ${\displaystyle b\in \{0,1\}}$.
6. The pair ${\displaystyle (k_{b},c)}$ is revealed to the adversary.
7. The adversary can again query ${\displaystyle \operatorname {Decap} ({\mathit {sk}},c')}$ for arbitrary encapsulations ${\displaystyle c'}$ of the adversary's choice, except for ${\displaystyle c}$.
8. The adversary returns a guess ${\displaystyle b'\in \{0,1\}}$, and wins the game if ${\displaystyle b=b'}$.

The IND-CCA advantage of the adversary is ${\displaystyle \left|\Pr[b'=b]-1/2\right|}$, that is, the probability beyond a fair coin toss at correctly distinguishing an encapsulated key from an independently randomly chosen key.

A key encapsulation mechanism can be used together with an authenticated symmetric cipher to construct a public-key encryption scheme for arbitrary messages. The security requirement for the symmetric cipher, called a data encapsulation mechanism or DEM, is indistinguishability against chosen-ciphertext attack for a single message encrypted by the sender.^[15][11][16]

Given a secure KEM with algorithms Gen/Encap/Decap, and a secure DEM ${\displaystyle E_{k}(m)}$, the following hybrid public-key encryption scheme is also secure against adaptive chosen-ciphertext attack in the public-key setting:^{[1][2]: § 7.2, Theorem 7.3 [13]: § 6.2.1}

• Key generation: Same as the KEM.
• To encrypt a message ${\displaystyle m}$ for a public key ${\displaystyle {\mathit {pk}}}$:
  1. Let ${\displaystyle (k,c):=\operatorname {Encap} ({\mathit {pk}})}$.
  2. Let ${\displaystyle \sigma :=E_{k}(m)}$.
  3. Send ${\displaystyle (c,\sigma )}$ as the ciphertext.
• To decrypt a ciphertext ${\displaystyle (c',\sigma ')}$ with private key ${\displaystyle {\mathit {sk}}}$:
  1. Let ${\displaystyle k':=\operatorname {Decap} ({\mathit {sk}},c')}$, or fail if it fails.
  2. Return the message ${\displaystyle E_{k'}^{-1}(\sigma ')}$, or fail if it fails.

Note that—as with any public-key encryption on its own—this does not authenticate the sender: anyone with the public key can send a message to a recipient with the private key. Other cryptography, such as digital signatures, must be used in a protocol for a sender to prove its identity to the receiver.^[17]

The use of an authenticated symmetric cipher is nevertheless required in this anonymous public-key encryption scheme to meet IND-CCA security. If an unauthenticated cipher were used, secure only against chosen-plaintext attack (IND-CPA), an adversary could selectively modify a message through its ciphertext in transit, which not only fails IND-CCA on a technicality^[18] but also can compromise confidentiality in practice as in EFAIL.^[19]

A KEM can also be used in an authenticated key agreement protocol such as TLS with forward secrecy for an online session, by having the client and server generate KEM key pairs and exchange signed encapsulations using those key pairs, which they then erase at the end of the session.^[13]

Different KEMs rely on different mathematical problems for their security. For example, the security of Rabin-KEM relies on the difficulty of integer factorization^[11], which has been studied for centuries, but is known to be vulnerable to quantum computers capable of running Shor's algorithm. In contrast, the security of ML-KEM relies on the difficulty of learning with errors,^[4] which has only been studied for decades, but is not known to be vulnerable even to an adversary with a Shor-capable quantum computer.

A KEM combiner is a scheme for combining two KEMs, KEM₁ and KEM₂ with respective encapsulation algorithms KEM₁.Encap and KEM₂.Encap and so on, into a combined KEM which is secure if either KEM₁ or KEM₂ is secure.^[20]

A KEM that combines a quantum-vulnerable KEM such as DH-KEM using X25519 with a post-quantum KEM such as ML-KEM is sometimes called a hybrid,^[21][10][22] not to be confused with a hybrid cryptosystem which combines public-key cryptography with symmetric-key cryptography.

Traditional RSA encryption, with ${\displaystyle t}$-bit moduli and exponent ${\displaystyle e}$, is defined as follows:^[23][24][25]

• Key generation, ${\displaystyle ({\mathit {pk}},{\mathit {sk}}):=\operatorname {Gen} ()}$:

1. Generate a ${\displaystyle t}$-bit semiprime ${\displaystyle n}$ with ${\displaystyle 2^{t-1}<n<2^{t}}$ at random satisfying ${\displaystyle \gcd(e,\lambda (n))=1}$, where ${\displaystyle \lambda (n)}$ is the Carmichael function.
2. Compute ${\displaystyle d:=e^{-1}{\bmod {\lambda }}(n)}$.
3. Return ${\displaystyle {\mathit {pk}}:=n}$ as the public key and ${\displaystyle {\mathit {sk}}:=(n,d)}$ as the private key. (Many variations on key generation algorithms and private key formats are available.^[26])

• Encryption of ${\displaystyle (t-1)}$-bit message ${\displaystyle m}$ to public key ${\displaystyle {\mathit {pk}}=n}$, giving ${\displaystyle c:=\operatorname {Encrypt} ({\mathit {pk}},m)}$:

1. Encode the bit string ${\displaystyle m}$ as an integer ${\displaystyle r}$ with ${\displaystyle 0\leq r<n}$.
2. Return ${\displaystyle c:=r^{e}{\bmod {n}}}$.

• Decryption of ciphertext ${\displaystyle c'}$ with private key ${\displaystyle {\mathit {sk}}=(n,d)}$, giving ${\displaystyle m':=\operatorname {Decrypt} ({\mathit {sk}},c')}$:

1. Compute ${\displaystyle r':=(c')^{d}{\bmod {n}}}$.
2. Decode the integer ${\displaystyle r'}$ as a bit string ${\displaystyle m'}$.

This naive approach is totally insecure. For example, since it is nonrandomized, it cannot be secure against even known-plaintext attack—an adversary can tell whether the sender is sending the message ATTACK AT DAWN versus the message ATTACK AT DUSK simply by encrypting those messages and comparing the ciphertext.

Even if ${\displaystyle m}$ is always a random secret key, such as a 256-bit AES key, when ${\displaystyle e}$ is chosen to optimize efficiency as ${\displaystyle e=3}$, the message ${\displaystyle m}$ can be computed from the ciphertext ${\displaystyle c}$ simply by taking real number cube roots, and there are many other attacks against plain RSA.^[23][24] Various randomized padding schemes have been devised in attempts—sometimes failed, like RSAES-PKCS1-v1_5^[23][27][28]—to make it secure for arbitrary short messages ${\displaystyle m}$.^[23][24]

Since the message ${\displaystyle m}$ is almost always a short secret key for a symmetric-key authenticated cipher used to encrypt an arbitrary bit string message, a simpler approach called RSA-KEM is to choose an element of ${\displaystyle \mathbb {Z} /n\mathbb {Z} }$ at random and use that to derive a secret key using a key derivation function ${\displaystyle H}$, roughly as follows:^[15][8][16]

• Key generation: As above.
• Encapsulation for a public key ${\displaystyle {\mathit {pk}}=n}$, giving ${\displaystyle (k,c):=\operatorname {Encap} ({\mathit {pk}})}$:

1. Choose an integer ${\displaystyle r}$ with ${\displaystyle 0\leq r<n}$ uniformly at random.
2. Return ${\displaystyle k:=H(r)}$ and ${\displaystyle c:=r^{e}{\bmod {n}}}$ as its encapsulation.

• Decapsulation of ${\displaystyle c'}$ with private key ${\displaystyle {\mathit {sk}}=(n,d)}$, giving ${\displaystyle k':=\operatorname {Decap} ({\mathit {sk}},c')}$:

1. Compute ${\displaystyle r':=(c')^{d}{\bmod {n}}}$.
2. Return ${\displaystyle k':=H(r')}$.

This approach is simpler to implement, and provides a tighter reduction to the RSA problem, than padding schemes like RSAES-OAEP.^[15]

Traditional Elgamal encryption is defined over a multiplicative subgroup of the finite field ${\displaystyle \mathbb {Z} /p\mathbb {Z} }$ with generator ${\displaystyle g}$ of order ${\displaystyle q}$ as follows:^[29][30]

• Key generation, ${\displaystyle (pk,sk):=\operatorname {Gen} ()}$:

1. Choose ${\displaystyle x\in \mathbb {Z} /q\mathbb {Z} }$ uniformly at random.
2. Compute ${\displaystyle y:=g^{x}{\bmod {p}}}$.
3. Return ${\displaystyle {\mathit {sk}}:=x}$ as the private key and ${\displaystyle {\mathit {pk}}:=y}$ as the public key.

• Encryption of a message ${\displaystyle m\in \mathbb {Z} /p\mathbb {Z} }$ to public key ${\displaystyle {\mathit {pk}}=y}$, giving ${\displaystyle c:=\operatorname {Encrypt} ({\mathit {pk}},m)}$:

1. Choose ${\displaystyle r\in \mathbb {Z} /q\mathbb {Z} }$ uniformly at random.
2. Compute: $${\displaystyle {\begin{aligned}t&:=y^{r}{\bmod {p}}\\c_{1}&:=g^{r}{\bmod {p}}\\c_{2}&:=(t\cdot m){\bmod {p}}\end{aligned}}}$$
3. Return the ciphertext ${\displaystyle c:=(c_{1},c_{2})}$.

• Decryption of a ciphertext ${\displaystyle c'=(c'_{1},c'_{2})}$ for a private key ${\displaystyle {\mathit {sk}}=x}$, giving ${\displaystyle m':=\operatorname {Decrypt} ({\mathit {sk}},c')}$:

1. Fail and return ${\displaystyle \bot }$ if ${\displaystyle (c'_{1})^{(p-1)/q}\not \equiv 1{\pmod {p}}}$ or if ${\displaystyle (c'_{2})^{(p-1)/q}\not \equiv 1{\pmod {p}}}$, i.e., if ${\displaystyle c'_{1}}$ or ${\displaystyle c'_{2}}$ is not in the subgroup generated by ${\displaystyle g}$.
2. Compute ${\displaystyle t':=(c'_{1})^{x}{\bmod {p}}}$.
3. Return ${\displaystyle m':=t^{-1}c'_{2}{\bmod {p}}}$.

This meets the syntax of a public-key encryption scheme, restricted to messages in the space ${\displaystyle \mathbb {Z} /p\mathbb {Z} }$ (which limits it to message of a few hundred bytes for typical values of ${\displaystyle p}$). By validating ciphertexts in decryption, it avoids leaking bits of the private key ${\displaystyle x}$ through maliciously chosen ciphertexts outside the group generated by ${\displaystyle g}$.

However, this fails to achieve indistinguishability against chosen-ciphertext attack. For example, an adversary having a ciphertext ${\displaystyle c=(c_{1},c_{2})}$ for an unknown message ${\displaystyle m}$ can trivially decrypt it by querying the decryption oracle for the distinct ciphertext ${\displaystyle c':=(c_{1},c_{2}g)}$, yielding the related plaintext ${\displaystyle m':=mg{\bmod {p}}}$, from which ${\displaystyle m}$ can be recovered by ${\displaystyle m=m'g^{-1}{\bmod {p}}}$.^[29]

Traditional Elgamal encryption can be adapted to the elliptic-curve setting, but it requires some way to reversibly encode messages as points on the curve, which is less trivial than encoding messages as integers mod ${\displaystyle p}$.^[31]

Since the message ${\displaystyle m}$ is almost always a short secret key for a symmetric-key authenticated cipher used to encrypt an arbitrary bit string message, a simpler approach—called Elgamal-KEM or DH-KEM—is to derive the secret key from ${\displaystyle t}$ and dispense with ${\displaystyle m}$ and ${\displaystyle c_{2}}$ altogether, as a KEM, using a key derivation function ${\displaystyle H}$:^[1][5]

• Key generation: As above.
• Encapsulation for a public key ${\displaystyle {\mathit {pk}}=y}$, giving ${\displaystyle (k,c):=\operatorname {Encap} ({\mathit {pk}})}$:

1. Choose ${\displaystyle r\in \mathbb {Z} /q\mathbb {Z} }$ uniformly at random.
2. Compute ${\displaystyle t:=y^{r}{\bmod {p}}}$.
3. Return ${\displaystyle k:=H(t)}$ and ${\displaystyle c:=g^{r}{\bmod {p}}}$ as its encapsulation.

• Decapsulation of ${\displaystyle c'}$ with private key ${\displaystyle {\mathit {sk}}=x}$, giving ${\displaystyle k':=\operatorname {Decap} ({\mathit {sk}},c')}$:

1. Fail and return ${\displaystyle \bot }$ if ${\displaystyle (c')^{(p-1)/q}\not \equiv 1{\pmod {p}}}$, i.e., if ${\displaystyle c'}$ is not in the subgroup generated by ${\displaystyle g}$.
2. Compute ${\displaystyle t':=(c')^{x}{\bmod {p}}}$.
3. Return ${\displaystyle k':=H(t')}$.

When combined with an authenticated cipher to encrypt arbitrary bit string messages, the combination is essentially the Integrated Encryption Scheme. Since this KEM only requires a one-way key derivation function to hash random elements of the group it is defined over, ${\displaystyle \mathbb {Z} /p\mathbb {Z} }$ in this case, and not a reversible encoding of messages, it is easy to extend to more compact and efficient elliptic curve groups for the same security, as in the ECIES, Elliptic Curve Integrated Encryption Scheme, or RFC 9180 DHKEM(...) instances.

• Public-key encryption
• Ciphertext indistinguishability
• IES—Integrated Encryption Scheme
• OAEP—Optimal Asymmetric Encryption Padding
• Hybrid cryptosystem