Protect yourself against future threats.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 =========================================================================== AUSCERT External Security Bulletin Redistribution ESB-2020.1189 HAProxy 1.8+ HTTP/2 HPACK Decoder Vulnerability Fixed 3 April 2020 =========================================================================== AusCERT Security Bulletin Summary --------------------------------- Product: haproxy Publisher: HAProxy Operating System: UNIX variants (UNIX, Linux, OSX) Impact/Access: Execute Arbitrary Code/Commands -- Existing Account Denial of Service -- Existing Account Resolution: Patch/Upgrade CVE Names: CVE-2019-9506 Original Bulletin: https://www.haproxy.com/blog/haproxy-1-8-http-2-hpack-decoder-vulnerability-fixed/ - --------------------------BEGIN INCLUDED TEXT-------------------- HAProxy 1.8+ HTTP/2 HPACK Decoder Vulnerability Fixed Daniel Corbett Daniel Corbett | Apr 2, 2020 | TECH | 0 comments [security-update] Security researcher Felix Wilhelm, working with Google's Project Zero, has disclosed a critical vulnerability in HAProxy's HTTP/2 HPACK decoder in versions 1.8 and above. Wilhelm has disclosed critical vulnerabilities in other popular products such as Xen, Hyper-V, IBM GPFS, and FireEye's MPS. Project Zero is a team of security analysts employed by Google that is tasked with finding zero-day vulnerabilities. The project was announced in July 2014 and has discovered many vulnerabilities from a wide range of vendors including Apple, Cloudflare, and Microsoft. Wilhelm intends to do his own write up around this bug and publish at a later date so at this point we are unaware of how he has discovered the issue and can only speculate. In many cases, bugs such as this one are discovered through fuzzing, static analysis, and manual code reviews. Fuzzing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. Traditionally this is done using open source tools such as American Fuzzy Lop (AFL) and Google's tool clusterfuzz. However, even with such tools available, fuzzing is no easy task, especially when it comes to targeting a threaded, event-driven, non-blocking engine such as HAProxy. Static analysis is the analysis of software that is performed without actually executing the program. According to Felix's paper Tracing Privileged Memory Accesses to Discover Software Vulnerabilities, techniques that are used mainly by compilers for performing safe optimizations of source code can also be used to discover security vulnerabilities. Static analysis can also be quite tedious as it involves crawling through each discovered item and eliminating false positives from the report and testing the severity of each. In short, while there are tools and processes available to help discover vulnerabilities in software it is certainly not an easy task and takes countless hours of dedication, validation and a tremendous effort to locate valid bugs in modern software. Timeline * 24 March 18:37 UTC: Felix Wilhelm contacts Willy Tarreau, HAProxy's lead developer, privately with a detailed report and a reproducer. * 24 March 18:50 UTC: Tarreau confirms receipt and starts looking at the issue * 25 March 8:31 UTC: Tarreau informs distros about an upcoming H2/HPACK vulnerability * 27 March 16:38 UTC: Tarreau proposes a candidate fix to Wilhelm * 27 March 17:10 UTC: Tarreau notifies distros about possible workarounds * 29 March 08:32 UTC: Tarreau sends the candidate patches to distros and proposes a coordinated release date for April 2nd 13:00 UTC * 30 March 11:38 UTC: all distros have already agreed on the CRD * 30 March 16:46 UTC: Wilhelm confirms the fix * 31 March 05:59 UTC: CVE ID assigned by the Debian team * 2 April 13:00 UTC: new packages published, making the vulnerability public Vulnerability Overview As mentioned above, Wilhelm discovered a critical vulnerability in HAProxy's HTTP/2 HPACK decoder that can be exploited to cause an out-of-bounds memory write potentially leading to corruption of data, a crash, or code execution. Specifically, the vulnerable code was located in HAProxy's HPACK table management code. HAProxy's lead developer, Willy Tarreau, has provided a detailed outline within the commit message explaining how this vulnerability came to be. The HPACK header table is implemented as a wrapping list inside a contiguous area. Header names and values are stored from right to left while indexes are stored from left to right. When there's no more room to store a new one, we wrap to the right again, or possibly defragment it if needed. The condition to use the right part (called the tailroom) or the left part (called the headroom) depends on the location of the last inserted header. After wrapping happens, the code is forced to stick to the tailroom by pretending there's no more headroom, so that the size fit always fails. The problem is that nothing prevents from storing a header with an empty name and empty value, resulting in a total size of zero bytes, which satisfies the condition to use the headroom. Doing this in a wrapped buffer results in changing the front header index and causing miscalculations on the available size and the addresses of the next headers. This may even allow an overwrite to some parts of the index, opening up the possibility to perform arbitrary writes into a 32-bit relative address space. This patch fixes the issue by making sure the headroom is considered only when the buffer does not wrap, instead of relying on the zero size. This vulnerability has been assigned CVE-2020-11100. Potential Impact Writing to arbitrary memory locations may sometimes be used to grant special permissions, modify data or even trick the program into following different code paths, particularly when the program's stack is overwritten. Attempts to exploit this bug may lead to a denial of service by crashing the HAProxy process. The risk of using it to read or modify data in memory within the program is low. This bug only allows for reaching memory areas within 32 bits from the beginning of the HPACK table, which doesn't allow it to touch the stack on 64-bit systems. In addition, the likelihood that it is used to overwrite a known location is extremely low because the address of each connection's HPACK table depends on the thread that accepted the connection and on the traffic that precedes this connection, all of which are not under the attacker's control. Even in a lab setting, when sending the attack as the very first request, some extra uncertainty happens because modern operating systems randomize functions and variables addresses via a mechanism called ASLR (Address Space Layout Randomization) which provides random addresses to functions, variables, and libraries. Affected Versions & Remediation The following is the list of affected versions, fixed version, and potential workarounds where available. It is recommended to upgrade immediately if you are using any of these with HTTP/2 enabled. Important note: If you are using HAProxy 2.0 or 2.1 you must upgrade or apply the relevant mitigations where available as these versions default to HTX which automatically upgrades HTTP/1 to HTTP/2. Affected Versions Fixed Versions Workarounds (where possible) HAProxy 1.8.0 1.8.24 HAProxy Enterprise 1.8r1 1.0.0-186.251 HAProxy 1.8.25+ 193.716 HAProxy HAProxy Enterprise Enterprise 1.8r2 Remove npn h2 or alpn h2 from the bind 1.8r2 2.0.0-190.714 2.0.0-205.1048+ lines 205.1000 ALOHA 10.5.13+ ALOHA 10.0.0 10.0.14 ALOHA 10.5.0 10.5.12 HAProxy 1.9.0 HAProxy 1.9.15+ 1.9.14 HAProxy HAProxy Enterprise Enterprise 1.9r1 Remove npn h2, alpn h2, or proto h2 from 1.9r1 1.0.0-197.290 1.0.0-213.948+ the bind lines 208.876 HAProxy ALOHA HAProxy ALOHA 11.0.0 11.0.8+ 11.0.7 HAProxy 2.0.0 HAProxy 2.0.14+ 2.0.13 HAProxy HAProxy Enterprise Enterprise 2.0r1 Disable HTX by adding no option 2.0r1 1.0.0-204.260 1.0.0-220.698+ http-use-htx and remove npn h2, alpn h2, 219.645 or proto h2 from the bind lines HAProxy ALOHA HAProxy ALOHA 11.5.0 11.5.4+ 11.5.3 HAProxy 2.1.0 HAProxy 2.1.4+ 2.1.3 HAProxy NO WORK AROUND AVAILABLE! MUST UPGRADE! HAProxy Enterprise Enterprise 2.1r1 2.1r1 1.0.0-217.0 1.0.0-221.93+ 221.38 HAProxy Enterprise customers can follow the below upgrade documentation to ensure you are on the latest version: * HAProxy Enterprise 1.8r2: https://www.haproxy.com/documentation/hapee/1-8r2 /upgrade/ * HAProxy Enterprise 1.9r1: https://www.haproxy.com/documentation/hapee/1-9r1 /upgrade/ * HAProxy Enterprise 2.0r1: https://www.haproxy.com/documentation/hapee/2-0r1 /upgrade/ * HAProxy Enterprise 2.1r1: https://www.haproxy.com/documentation/hapee/2-1r1 /upgrade/ Optionally you can contact our support for help. For ALOHA 10.0 and above customers we recommend that you reach out to support for assistance on obtaining the latest image. HAProxy Edge has been automatically upgraded prior to announcement. If you have any questions at all about this please feel free to reach out directly to our support team. Acknowledgements We would like to really thank Felix Wilhelm for his responsible disclosure and his timely help in addressing this issue. Felix also kindly agreed to slightly delay the publication of his full analysis of this bug to help protect users while they deploy the necessary updates on their systems. Full details will be released on April 20th, but users are urged to upgrade and are reminded that attackers will figure out the issue by themselves without waiting for the full disclosure. - --------------------------END INCLUDED TEXT-------------------- You have received this e-mail bulletin as a result of your organisation's registration with AusCERT. The mailing list you are subscribed to is maintained within your organisation, so if you do not wish to continue receiving these bulletins you should contact your local IT manager. If you do not know who that is, please send an email to auscert@auscert.org.au and we will forward your request to the appropriate person. NOTE: Third Party Rights This security bulletin is provided as a service to AusCERT's members. As AusCERT did not write the document quoted above, AusCERT has had no control over its content. The decision to follow or act on information or advice contained in this security bulletin is the responsibility of each user or organisation, and should be considered in accordance with your organisation's site policies and procedures. AusCERT takes no responsibility for consequences which may arise from following or acting on information or advice contained in this security bulletin. NOTE: This is only the original release of the security bulletin. It may not be updated when updates to the original are made. If downloading at a later date, it is recommended that the bulletin is retrieved directly from the author's website to ensure that the information is still current. Contact information for the authors of the original document is included in the Security Bulletin above. If you have any questions or need further information, please contact them directly. Previous advisories and external security bulletins can be retrieved from: https://www.auscert.org.au/bulletins/ =========================================================================== Australian Computer Emergency Response Team The University of Queensland Brisbane Qld 4072 Internet Email: auscert@auscert.org.au Facsimile: (07) 3365 7031 Telephone: (07) 3365 4417 (International: +61 7 3365 4417) AusCERT personnel answer during Queensland business hours which are GMT+10:00 (AEST). On call after hours for member emergencies only. =========================================================================== -----BEGIN PGP SIGNATURE----- Comment: http://www.auscert.org.au/render.html?it=1967 iQIVAwUBXoZ39GaOgq3Tt24GAQhEuQ/+PKaF+boecCzq4NFVHHbGNfDp18cepzkd nY/XTwypAlWu/XRIF+lIVAJ/+P13JMBc0FClxMaRJO9dlc3OqshsZ+5fenaT3Eto PHpkZY7vs7zkNqdyvuORU8isNU4l4qDdAG5fjZCY3o4WAdm4jk7MCJwiIMCvyzwh MMM3Y+Vlb1YmtfN25U0Tmk6NbuWfaqA97ofdGaKcC2cWqyRhoqEn5MUUv+O4msM8 RZ5pbPkRUWjm8jQuSK7/aIYe5jBy3vqicCcH9WRaFrWHPmxeEmMx2yltAdWGUoJI ro+MTmJAiaUZEXn8i0xl+Dn4ljQzuYzSwcnyKzRPN7QNY4rc9urVT4TQbC/fdP5m nQfghPLpowHiEDRZeKj3A5IfZxoVHhB9eMDkYIYT1THx2cBf2MALDeIV7TScyKZW WidM/iE8F3QyCuSQjeqQiFxlTZwzpuf+hHqmIY2KSQUEvv/IcGXgbCsdKsP34F3f 9MAptXDFOU5QlR869jTgZx47qH6MrbZwmgemfLsrEiqWPCeKRhD6QsqN9SaaT5m3 myKkZv+Yp5iH89M/4LmwsO9JB/zx9YsFKkO9HSCoN7omEHw5/X74tgWpKEWZSX7O dLVPzE6MWdxqkk2J47ECIZQgyAMnzyg94w0WZHNidk3htRSqBMtaUgCbOEWk8Hr3 Z6eijxB4ONI= =Q57t -----END PGP SIGNATURE-----