... Loading ...

SSD Secure Disclosure

Disclosing vulnerabilities responsibly since 2007

SSD Advisory – OpenSSH Pre-Auth XMSS Integer Overflow

Vulnerability Summary
The following advisory describes a Pre-Auth Integer Overflow in the XMSS Key Parsing Algorithm in OpenSSH.


An independent Security Researcher, Adam “pi3” Zabrocki, has reported this vulnerability to SSD Secure Disclosure program.

Affected Systems
OpenSSH version 7.7 up to the latest one (8.0) supporting XMSS keys (compiled with a defined WITH_XMSS macro).
Nevertheless, the bug is only there when OpenSSH is compiled via a compiler with data model ILP64, LLP64 or ILP32 (e.g. any 32 bits systems).

Vendor Response
The vulnerability was fixed in OpenSSH version 8.1

Vulnerability Details


OpenSSH is a free version of the SSH connectivity tools which technical users of the Internet rely on. Users of telnet, rlogin, and ftp may not realize that their password is transmitted across the Internet unencrypted, but in fact it is.
OpenSSH encrypts all traffic (including passwords) to effectively eliminate eavesdropping, connection hijacking, and other attacks. Additionally, OpenSSH provides secure tunneling capabilities and several authentication methods, and supports all SSH protocol versions.

OpenSSH supports several signing algorithms (for authentication keys) which can be divided in two groups depending on the mathematical properties they exploit:

  • DSA and RSA, which rely on the practical difficulty of factoring the product of two large prime numbers
  • ECDSA and Ed25519, which rely on the elliptic curve discrete logarithm problem

Elliptic curve cryptography (ECC) algorithms are a more recent addition to public key cryptosystems. One of their main advantages is their ability to provide the same level of security with smaller keys, which makes for less computationally intensive operations (i.e. faster key creation, encryption and decryption) and reduced storage and transmission requirements.

OpenSSH 7.7 add experimental support for PQC (Post Quantum Cryptography) XMSS keys (Extended Hash-Based Signatures) is designed to work in Post Quantum area. The code is not compiled in by default at this time.


The eXtended Merkle Signature Scheme (XMSS) is the latest stateful hash-based signature scheme. It has the smallest signatures out of such schemes and comes with a multi-tree variant that solves the problem of slow key generation.
Moreover, it can be shown that XMSS is secure, making only mild assumptions on the underlying hash function. Especially, it is not required that the cryptographic hash function is collision-resistant for the security of XMSS.

In contrast to traditional signature schemes, the signature schemes used in XMSS are stateful, meaning the secret key changes over time. If a secret key state is used twice, no cryptographic security guarantees remain. In consequence, it becomes feasible to forge a signature on a new message.

The Vulnerability

Integer Overflow vulnerability causing a memory corruption bug was found during process of parsing XMSS private key. This process requires taking into account previously saved “state”, if available. A function responsible for handling XMSS saved “states” is prone to a memory corruption via integer overflow vulnerability:

If an attacker generates a state where ‘aadlen’ + ‘encrypted_len’ is bigger than INT_MAX, then it is possible to successfully pass verification at [1].
Additionally, if ‘authlen’ + ‘encrypted_len’ is also bigger than INT_MAX, then an integer overflow at [2] results in allocating a smaller buffer than desired.

Immediate overflow can happen at [3] where cipher_crypt() is called. This function operates as follows:

  • Copy ‘aadlen’ bytes (without en/decryption) from ‘src’ to ‘dest’.
  • Theses bytes are treated as additional authenticated data for authenticated encryption modes.
  • En/Decrypt ‘len’ bytes at offset ‘aadlen’ from ‘src’ to ‘dest’.
  • Use ‘authlen’ bytes at offset ‘len’+’aadlen’ as the authentication tag.
  • This tag is written on encryption and verified on decryption.

Because of the nature of cipher_crypt() function, it is possible to overflow a lot of useful data before a crash, because ‘copy’ is not being done in a 1 single shot but chunk-by-chunk during crypto operation. Nevertheless, it is not a trivial task.

Vectors of Attack

Any OpenSSH functionality which can parse private XMSS key is vulnerable. E.g. If ‘sshd’ daemon is configured to use an XMSS host key that is malformed, it will crash upon any attempt to connect to this server:

If someone configures ‘authorized_key’ to use XMSS public key and keep private key to be able to connect to the server, ‘ssh’ client will crash:

If someone tries to add this key to ssh-agent, it will crash too:

From the real world cases, many hosting services allow the user to upload user’s own pair of private/public keys for the authentication. Then it can be automatically consumed when creating new VMs. They need to parse it so an attacker can upload a malicious key and cause vulnerability during parsing process. Examples include Azure Key Vault, Amazon Key Management Service (KMS) or Cloud Key Management Service by Google Cloud. Any Key management service might be a vector of the attack.

Print Friendly, PDF & Email

3 thoughts on “SSD Advisory – OpenSSH Pre-Auth XMSS Integer Overflow

Leave a Reply