Analysis of CVE-2020-1730: DoS in libssh
CVE-2020-1730: libssh - denial of service when handling AES-CTR (or DES) ciphers
libssh is a multiplatform C library implementing the SSHv2 protocol on client and server side. Recently a security advisory was released for “Denial of service vulnerability” in libssh.
Important information from advisory -
- Possible DoS in client and server when handling AES-CTR keys with OpenSSL
- Connection hasn't been fully initialized
- Issue arises in cleanup of ciphers when closing the connection
The typical libssh session looks like following diagram. This diagram is created for high level overview of process, so it may not be exactly precise.

Let’s go through each point mentioned in advisory -
1. Affected when handling AES-CTR keys with OpenSSL
The AES-CTR cipher related information is defined in libcrypto.c. The exploitation is possible when OPENSSL AES support is enabled and OpenSSL EVP interface is not supported.
For this analysis I used aes128-ctr mode and a connection from client-to-server.
libcrypto.c
704 #else /* HAVE_OPENSSL_EVP_AES_CTR */
705 {
706 .name = "aes128-ctr",
707 .blocksize = 16,
708 .ciphertype = SSH_AES128_CTR,
709 .keysize = 128,
710 .set_encrypt_key = aes_ctr_set_key,
711 .set_decrypt_key = aes_ctr_set_key,
712 .encrypt = aes_ctr_encrypt,
713 .decrypt = aes_ctr_encrypt,
714 .cleanup = aes_ctr_cleanup
715 },
2. Connection hasn't been fully initialized
To trigger this issue, another condition is that connection should not be fully initialized. The typical SSH connection session looks like -

To trigger this condition, connection needs to be disconnected in between session key exchange so that it’s not fully initialized. The session key exchange is handled by ssh_handle_key_exchange().This function internally calls multiple interesting functions.
ssh_handle_key_exchange -> ssh_handle_packets_termination -> ssh_handle_packets -> ssh_timeout_elapsed
server.c
630 int ssh_handle_key_exchange(ssh_session session) {
631 int rc;
632 if (session->session_state != SSH_SESSION_STATE_NONE)
633 goto pending;
634 rc = ssh_send_banner(session, 1);
635 if (rc < 0) {
636 return SSH_ERROR;
637 }
638
639 session->alive = 1;
640
641 session->ssh_connection_callback = ssh_server_connection_callback;
642 session->session_state = SSH_SESSION_STATE_SOCKET_CONNECTED;
643 ssh_socket_set_callbacks(session->socket,&session->socket_callbacks);
644 session->socket_callbacks.data=callback_receive_banner;
645 session->socket_callbacks.exception=ssh_socket_exception_callback;
646 session->socket_callbacks.userdata=session;
647
648 rc = server_set_kex(session);
649 if (rc < 0) {
650 return SSH_ERROR;
651 }
652 pending:
653 rc = ssh_handle_packets_termination(session, SSH_TIMEOUT_USER,
654 ssh_server_kex_termination,session);
655 SSH_LOG(SSH_LOG_PACKET, "ssh_handle_key_exchange: current state : %d",
656 session->session_state);
657 if (rc != SSH_OK)
658 return rc;
659 if (session->session_state == SSH_SESSION_STATE_ERROR ||
660 session->session_state == SSH_SESSION_STATE_DISCONNECTED) {
661 return SSH_ERROR;
662 }
663
664 return SSH_OK;
665 }
The ssh_handle_packets_termination() keeps polling the current session for an event and call the appropriate callbacks.
session.c
635 /**
636 * @internal
637 *
638 * @brief Poll the current session for an event and call the appropriate
639 * callbacks.
640 *
641 * This will block until termination function returns true, or timeout expired.
642 *
643 * @param[in] session The session handle to use.
644 *
645 * @param[in] timeout Set an upper limit on the time for which this function
646 * will block, in milliseconds. Specifying SSH_TIMEOUT_INFINITE
647 * (-1) means an infinite timeout.
648 * Specifying SSH_TIMEOUT_USER means to use the timeout
649 * specified in options. 0 means poll will return immediately.
650 * SSH_TIMEOUT_DEFAULT uses blocking parameters of the session.
651 * This parameter is passed to the poll() function.
652 *
653 * @param[in] fct Termination function to be used to determine if it is
654 * possible to stop polling.
655 * @param[in] user User parameter to be passed to fct termination function.
656 * @return SSH_OK on success, SSH_ERROR otherwise.
657 */
658 int ssh_handle_packets_termination(ssh_session session,
659 int timeout,
660 ssh_termination_function fct,
661 void *user)
662 {
3. Cleanup cipher information
During session, if connection ended abruptly, the termination function fct(user) = -2 is returned and polling is stopped.
session.c
......
688 while(!fct(user)) {
689 ret = ssh_handle_packets(session, tm);
690 if (ret == SSH_ERROR) {
691 break;
692 }
693 if (ssh_timeout_elapsed(&ts,timeout)) {
694 ret = fct(user) ? SSH_OK : SSH_AGAIN; <-- SSH_AGAIN = -2
695 break;
696 }
697
698 tm = ssh_timeout_update(&ts, timeout);
699 }
700
701 return ret;
702 }
Then function ssh_handle_key_exchange() returns without a success.
ssh_handle_key_exchange(session) != SSH_OK
When connection is closed abruptly and key exchange is failed, application kills the session by calling ssh_disconnect(session).
client.c
652 /**
653 * @brief Disconnect from a session (client or server).
654 * The session can then be reused to open a new session.
655 *
656 * @param[in] session The SSH session to use.
657 */
658 void ssh_disconnect(ssh_session session) {
659 struct ssh_iterator *it;
660 int rc;
.........
699 if (session->next_crypto) {
700 crypto_free(session->next_crypto);
701 session->next_crypto = crypto_new();
702 if (session->next_crypto == NULL) {
703 ssh_set_error_oom(session);
704 }
705 }
As the name suggests, function cyrpto_free() starts cleaning of cryptographic parameters used in session.
wrapper.c
146 void crypto_free(struct ssh_crypto_struct *crypto)
147 {
148 size_t i;
149
150 if (crypto == NULL) { <-- #we need to initiate KX to satisfy
151 return;
152 }
153
154 ssh_key_free(crypto->server_pubkey);
155
156 cipher_free(crypto->in_cipher); <-- #cryptographic information from other party
157 cipher_free(crypto->out_cipher);
crypto->in_cipher contains ciphers information received from another party during key exchange. When the connection is closed without successful key exchange, in crypto->in_cipher some cryptographic material remains uninitialized.
wrapper.c
130 static void cipher_free(struct ssh_cipher_struct *cipher) {
131 ssh_cipher_clear(cipher);
132 SAFE_FREE(cipher);
133 }
The cleanup function is called on this uninitialized cipher information.
(gdb) print session->next_crypto
$8 = (struct ssh_crypto_struct *) 0x40a120
150 if (crypto == NULL) {
(gdb) print crypto
$9 = (struct ssh_crypto_struct *) 0x40a120
156 cipher_free(crypto->in_cipher);
(gdb) print crypto->in_cipher
$10 = (struct ssh_cipher_struct *) 0x411870
131 ssh_cipher_clear(cipher);
(gdb) print cipher
$13 = (struct ssh_cipher_struct *) 0x411870
(gdb) print cipher->aes_key
$14 = (struct ssh_aes_key_schedule *) 0x0
125 if (cipher->cleanup != NULL) {
(gdb) print cipher->cleanup
$15 = (void (*)(struct ssh_cipher_struct *)) 0xb7f8c07f <aes_ctr_cleanup>
(gdb) print cipher->aes_key
$16 = (struct ssh_aes_key_schedule *) 0x0
wrapper.c
107 void ssh_cipher_clear(struct ssh_cipher_struct *cipher){
......
125 if (cipher->cleanup != NULL) {
126 cipher->cleanup(cipher);
127 }
128 }
The cleanup function which is defined with specific cipher i.e. aes128-ctr is called.
libcrypto.c
704 #else /* HAVE_OPENSSL_EVP_AES_CTR */
705 {
706 .name = "aes128-ctr",
....
714 .cleanup = aes_ctr_cleanup
715 },
The cryptographic material used for the session contains sensitive information. To clean this sensitive information explicit_bzero() function is called. This function guarantees that compiler optimizations will not remove the erase operation if the compiler deduces that the operation is “unnecessary”.
libcrypto.c
643 static void aes_ctr_cleanup(struct ssh_cipher_struct *cipher){
644 explicit_bzero(cipher->aes_key, sizeof(*cipher->aes_key));
645 SAFE_FREE(cipher->aes_key);
646 }
#include <string.h>
void
explicit_bzero(void *b, size_t len);
The explicit_bzero() variant behaves the same, but will not be removed by a compiler’s dead store optimization pass, making it useful for clearing sensitive memory such as a password.
433 /* Set N bytes of S to 0. The compiler will not delete a call to this
434 function, even if S is dead after the call. */
435 extern void explicit_bzero (void *__s, size_t __n) __THROW __nonnull ((1));
It is assumed that the block which is going to be cleaned i.e. void*__s is not a null pointer. In our scenario exactly the same case triggers, the connection is not fully initialized, so value of cipher->aes_key is null pointer.
126 cipher->cleanup(cipher);
(gdb) x/10xw cipher->aes_key
0x0: Cannot access memory at address 0x0
When explicit_bzero() function tries to clean the “cipher->aes_key” information, the result is “segmentation fault” error:
aes_ctr_cleanup (cipher=0x411870) at /home/developer/code-review/libssh-0.8.8/src/libcrypto.c:644
644 explicit_bzero(cipher->aes_key, sizeof(*cipher->aes_key));
Program received signal SIGSEGV, Segmentation fault.
__memset_ia32 () at ../sysdeps/i386/i686/memset.S:77
77 in ../sysdeps/i386/i686/memset.S
Test environment
OS: Debian GNU/Linux 10 (buster)
Server: examples/ssh_server_fork.c , removed fork() for this analysis.
Server-logs:
[2020/05/05 16:32:41.639034, 2] ssh_pki_import_privkey_base64: Trying to decode privkey passphrase=false
[2020/05/05 16:32:41.639321, 2] ssh_pki_openssh_import: Opening OpenSSH private key: ciphername: none, kdf: none, nkeys: 1
[2020/05/05 16:32:41.639674, 2] ssh_pki_import_privkey_base64: Trying to decode privkey passphrase=false
[2020/05/05 16:32:41.639929, 2] ssh_pki_openssh_import: Opening OpenSSH private key: ciphername: none, kdf: none, nkeys: 1
[2020/05/05 16:32:41.640235, 2] ssh_pki_import_privkey_base64: Trying to decode privkey passphrase=false
[2020/05/05 16:32:41.640570, 2] ssh_pki_openssh_import: Opening OpenSSH private key: ciphername: none, kdf: none, nkeys: 1
[2020/05/05 16:33:44.119994, 3] ssh_socket_pollcallback: Received POLLOUT in connecting state
[2020/05/05 16:33:44.120028, 3] callback_receive_banner: Received banner: SSH-2.0-libssh_0.8.8
[2020/05/05 16:33:44.120035, 1] ssh_server_connection_callback: SSH client banner: SSH-2.0-libssh_0.8.8
[2020/05/05 16:33:44.120040, 1] ssh_analyze_banner: Analyzing banner: SSH-2.0-libssh_0.8.8
[2020/05/05 16:33:44.120060, 3] packet_send2: packet: wrote [len=708,padding=6,comp=701,payload=701]
[2020/05/05 16:33:44.120080, 3] ssh_socket_unbuffered_write: Enabling POLLOUT for socket
[2020/05/05 16:33:45.968933, 3] ssh_packet_socket_callback: packet: read type 20 [len=588,padding=4,comp=583,payload=583]
[2020/05/05 16:33:45.968954, 3] ssh_packet_process: Dispatching handler for packet type 20
[2020/05/05 16:33:45.968974, 3] ssh_packet_kexinit: The client supports extension negotiation. Enabled signature algorithms: SHA512
[2020/05/05 16:33:45.968993, 2] ssh_kex_select_methods: Negotiated curve25519-sha256,ecdsa-sha2-nistp256,aes128-ctr,aes256-ctr,hmac-sha2-256,hmac-sha2-256,none,none,,
[2020/05/05 16:33:45.968999, 3] crypt_set_algorithms_server: Set output algorithm aes256-ctr
[2020/05/05 16:33:45.969004, 3] crypt_set_algorithms_server: Set HMAC output algorithm to hmac-sha2-256
[2020/05/05 16:33:45.969008, 3] crypt_set_algorithms_server: Set input algorithm aes128-ctr
[2020/05/05 16:33:45.969013, 3] crypt_set_algorithms_server: Set HMAC input algorithm to hmac-sha2-256
[2020/05/05 16:33:47.811206, 1] ssh_socket_exception_callback: Socket exception callback: 1 (0)
[2020/05/05 16:33:47.811227, 1] ssh_socket_exception_callback: Socket error: disconnected
[2020/05/05 16:33:47.811255, 3] ssh_handle_key_exchange: ssh_handle_key_exchange: current state : 9
Socket error: disconnected
Segmentation fault