OpenSSL HollowByte Flaw Allows Denial-of-Service via 11-Byte Requests

OpenSSL shipped a fix for the HollowByte vulnerability in June, a flaw that allows attackers to trigger a denial-of-service condition by sending specially crafted, 11-byte Transport Layer Security (TLS) requests. This vulnerability, disclosed by Okta's Red Team, causes unpatched OpenSSL servers to allocate up to 131 KB of memory for a message that never arrives. On glibc systems tested by Okta, this allocated memory remains unavailable until the affected process is restarted, effectively freezing server operations.

Okta's Red Team, responsible for discovering and naming the HollowByte bug, reported that the fix was integrated into OpenSSL without a dedicated Common Vulnerabilities and Exposures (CVE) identifier, an official advisory, or a specific mention in the changelog. This lack of explicit documentation means that many system administrators may be unaware of the vulnerability and the existence of the patch. The team's analysis indicates that the vulnerability lies within the handling of TLS messages, where an 11-byte input can trigger an excessive memory allocation, leading to resource exhaustion.

The implications of HollowByte are significant for any server running an unpatched version of OpenSSL that accepts TLS connections. A successful exploitation could render critical services unavailable, impacting businesses and users relying on those services. The ease of exploitation, requiring only a small, precisely crafted request, makes it a potent tool for denial-of-service attacks. Okta's disclosure aims to raise awareness and encourage prompt patching of affected systems, emphasizing the importance of diligent security practices even when vulnerabilities are not publicly advertised with standard identifiers.