-----BEGIN PGP SIGNED MESSAGE-----
AUSCERT External Security Bulletin Redistribution
HAProxy 1.8+ HTTP/2 HPACK Decoder Vulnerability Fixed
3 April 2020
AusCERT Security Bulletin Summary
Operating System: UNIX variants (UNIX, Linux, OSX)
Impact/Access: Execute Arbitrary Code/Commands -- Existing Account
Denial of Service -- Existing Account
CVE Names: CVE-2019-9506
- --------------------------BEGIN INCLUDED TEXT--------------------
HAProxy 1.8+ HTTP/2 HPACK Decoder Vulnerability Fixed
Daniel Corbett Daniel Corbett | Apr 2, 2020 | TECH | 0 comments
Security researcher Felix Wilhelm, working with Google's Project Zero, has
disclosed a critical vulnerability in HAProxy's HTTP/2 HPACK decoder in
versions 1.8 and above.
Wilhelm has disclosed critical vulnerabilities in other popular products such
as Xen, Hyper-V, IBM GPFS, and FireEye's MPS. Project Zero is a team of
security analysts employed by Google that is tasked with finding zero-day
vulnerabilities. The project was announced in July 2014 and has discovered many
vulnerabilities from a wide range of vendors including Apple, Cloudflare, and
Microsoft. Wilhelm intends to do his own write up around this bug and publish
at a later date so at this point we are unaware of how he has discovered the
issue and can only speculate.
In many cases, bugs such as this one are discovered through fuzzing, static
analysis, and manual code reviews. Fuzzing is an automated software testing
technique that involves providing invalid, unexpected, or random data as inputs
to a computer program. The program is then monitored for exceptions such as
crashes, failing built-in code assertions, or potential memory leaks.
Traditionally this is done using open source tools such as American Fuzzy Lop
(AFL) and Google's tool clusterfuzz. However, even with such tools available,
fuzzing is no easy task, especially when it comes to targeting a threaded,
event-driven, non-blocking engine such as HAProxy. Static analysis is the
analysis of software that is performed without actually executing the program.
According to Felix's paper Tracing Privileged Memory Accesses to Discover
Software Vulnerabilities, techniques that are used mainly by compilers for
performing safe optimizations of source code can also be used to discover
security vulnerabilities. Static analysis can also be quite tedious as it
involves crawling through each discovered item and eliminating false positives
from the report and testing the severity of each. In short, while there are
tools and processes available to help discover vulnerabilities in software it
is certainly not an easy task and takes countless hours of dedication,
validation and a tremendous effort to locate valid bugs in modern software.
* 24 March 18:37 UTC: Felix Wilhelm contacts Willy Tarreau, HAProxy's lead
developer, privately with a detailed report and a reproducer.
* 24 March 18:50 UTC: Tarreau confirms receipt and starts looking at the
* 25 March 8:31 UTC: Tarreau informs distros about an upcoming H2/HPACK
* 27 March 16:38 UTC: Tarreau proposes a candidate fix to Wilhelm
* 27 March 17:10 UTC: Tarreau notifies distros about possible workarounds
* 29 March 08:32 UTC: Tarreau sends the candidate patches to distros and
proposes a coordinated release date for April 2nd 13:00 UTC
* 30 March 11:38 UTC: all distros have already agreed on the CRD
* 30 March 16:46 UTC: Wilhelm confirms the fix
* 31 March 05:59 UTC: CVE ID assigned by the Debian team
* 2 April 13:00 UTC: new packages published, making the vulnerability public
As mentioned above, Wilhelm discovered a critical vulnerability in HAProxy's
HTTP/2 HPACK decoder that can be exploited to cause an out-of-bounds memory
write potentially leading to corruption of data, a crash, or code execution.
Specifically, the vulnerable code was located in HAProxy's HPACK table
management code. HAProxy's lead developer, Willy Tarreau, has provided a
detailed outline within the commit message explaining how this vulnerability
came to be.
The HPACK header table is implemented as a wrapping list inside a contiguous
area. Header names and values are stored from right to left while indexes are
stored from left to right. When there's no more room to store a new one, we
wrap to the right again, or possibly defragment it if needed. The condition to
use the right part (called the tailroom) or the left part (called the headroom)
depends on the location of the last inserted header. After wrapping happens,
the code is forced to stick to the tailroom by pretending there's no more
headroom, so that the size fit always fails. The problem is that nothing
prevents from storing a header with an empty name and empty value, resulting in
a total size of zero bytes, which satisfies the condition to use the headroom.
Doing this in a wrapped buffer results in changing the front header index and
causing miscalculations on the available size and the addresses of the next
headers. This may even allow an overwrite to some parts of the index, opening
up the possibility to perform arbitrary writes into a 32-bit relative address
This patch fixes the issue by making sure the headroom is considered only when
the buffer does not wrap, instead of relying on the zero size.
This vulnerability has been assigned CVE-2020-11100.
Writing to arbitrary memory locations may sometimes be used to grant special
permissions, modify data or even trick the program into following different
code paths, particularly when the program's stack is overwritten. Attempts to
exploit this bug may lead to a denial of service by crashing the HAProxy
process. The risk of using it to read or modify data in memory within the
program is low. This bug only allows for reaching memory areas within 32 bits
from the beginning of the HPACK table, which doesn't allow it to touch the
stack on 64-bit systems. In addition, the likelihood that it is used to
overwrite a known location is extremely low because the address of each
connection's HPACK table depends on the thread that accepted the connection and
on the traffic that precedes this connection, all of which are not under the
attacker's control. Even in a lab setting, when sending the attack as the very
first request, some extra uncertainty happens because modern operating systems
randomize functions and variables addresses via a mechanism called ASLR
(Address Space Layout Randomization) which provides random addresses to
functions, variables, and libraries.
Affected Versions & Remediation
The following is the list of affected versions, fixed version, and potential
workarounds where available. It is recommended to upgrade immediately if you
are using any of these with HTTP/2 enabled.
Important note: If you are using HAProxy 2.0 or 2.1 you must upgrade or apply
the relevant mitigations where available as these versions default to HTX which
automatically upgrades HTTP/1 to HTTP/2.
Affected Versions Fixed Versions Workarounds (where possible)
1.8r1 1.0.0-186.251 HAProxy 1.8.25+
HAProxy Enterprise Enterprise 1.8r2 Remove npn h2 or alpn h2 from the bind
1.8r2 2.0.0-190.714 2.0.0-205.1048+ lines
HAProxy 1.9.0 HAProxy 1.9.15+
HAProxy Enterprise Enterprise 1.9r1 Remove npn h2, alpn h2, or proto h2 from
1.9r1 1.0.0-197.290 1.0.0-213.948+ the bind lines
HAProxy ALOHA 11.0.0 11.0.8+
HAProxy 2.0.0 HAProxy 2.0.14+
HAProxy Enterprise Enterprise 2.0r1 Disable HTX by adding no option
2.0r1 1.0.0-204.260 1.0.0-220.698+ http-use-htx and remove npn h2, alpn h2,
219.645 or proto h2 from the bind lines
HAProxy ALOHA 11.5.0 11.5.4+
HAProxy 2.1.0 HAProxy 2.1.4+
HAProxy NO WORK AROUND AVAILABLE! MUST UPGRADE!
HAProxy Enterprise Enterprise 2.1r1
2.1r1 1.0.0-217.0 1.0.0-221.93+
HAProxy Enterprise customers can follow the below upgrade documentation to
ensure you are on the latest version:
* HAProxy Enterprise 1.8r2: https://www.haproxy.com/documentation/hapee/1-8r2
* HAProxy Enterprise 1.9r1: https://www.haproxy.com/documentation/hapee/1-9r1
* HAProxy Enterprise 2.0r1: https://www.haproxy.com/documentation/hapee/2-0r1
* HAProxy Enterprise 2.1r1: https://www.haproxy.com/documentation/hapee/2-1r1
Optionally you can contact our support for help.
For ALOHA 10.0 and above customers we recommend that you reach out to support
for assistance on obtaining the latest image.
HAProxy Edge has been automatically upgraded prior to announcement.
If you have any questions at all about this please feel free to reach out
directly to our support team.
We would like to really thank Felix Wilhelm for his responsible disclosure and
his timely help in addressing this issue. Felix also kindly agreed to slightly
delay the publication of his full analysis of this bug to help protect users
while they deploy the necessary updates on their systems. Full details will be
released on April 20th, but users are urged to upgrade and are reminded that
attackers will figure out the issue by themselves without waiting for the full
- --------------------------END INCLUDED TEXT--------------------
You have received this e-mail bulletin as a result of your organisation's
registration with AusCERT. The mailing list you are subscribed to is
maintained within your organisation, so if you do not wish to continue
receiving these bulletins you should contact your local IT manager. If
you do not know who that is, please send an email to firstname.lastname@example.org
and we will forward your request to the appropriate person.
NOTE: Third Party Rights
This security bulletin is provided as a service to AusCERT's members. As
AusCERT did not write the document quoted above, AusCERT has had no control
over its content. The decision to follow or act on information or advice
contained in this security bulletin is the responsibility of each user or
organisation, and should be considered in accordance with your organisation's
site policies and procedures. AusCERT takes no responsibility for consequences
which may arise from following or acting on information or advice contained in
this security bulletin.
NOTE: This is only the original release of the security bulletin. It may
not be updated when updates to the original are made. If downloading at
a later date, it is recommended that the bulletin is retrieved directly
from the author's website to ensure that the information is still current.
Contact information for the authors of the original document is included
in the Security Bulletin above. If you have any questions or need further
information, please contact them directly.
Previous advisories and external security bulletins can be retrieved from:
Australian Computer Emergency Response Team
The University of Queensland
Internet Email: email@example.com
Facsimile: (07) 3365 7031
Telephone: (07) 3365 4417 (International: +61 7 3365 4417)
AusCERT personnel answer during Queensland business hours
which are GMT+10:00 (AEST).
On call after hours for member emergencies only.
-----BEGIN PGP SIGNATURE-----
-----END PGP SIGNATURE-----