---
title:
  Security vulnerability in OTel Collector confighttp and configgrpc
  (CVE-2024-36129)
linkTitle: Collector vulnerability CVE-2024-36129
date: 2024-06-05
author:
  '[Juraci Paixão Kröhling](https://github.com/jpkrohling) (OpenTelemetry,
  Grafana Labs), [Pablo Baeyens](https://github.com/mx-psi) (OpenTelemetry,
  Datadog)'
cSpell:ignore: Baeyens configgrpc confighttp Miroslav Stampar zstd
---

On our path toward graduation, the OpenTelemetry project is currently undergoing
a security audit sponsored by the [CNCF](https://www.cncf.io/), facilitated by
[OSTIF](https://ostif.org/), and performed by
[7ASecurity](https://7asecurity.com/). During this process, we have received a
few ideas about things that we could do better, like using specific compiler
flags when preparing our
[OpenTelemetry Collector binaries](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).
On 31 May 2024, we received
[a more serious report](https://github.com/open-telemetry/opentelemetry-collector/security/advisories/GHSA-c74f-6mfw-mm4v):
a malicious user could cause a denial of service (DoS) when using a specially
crafted HTTP or gRPC request. The advisory was assigned the following CVE
identifier: CVE-2024-36129.

When sending an HTTP request with a compressed payload, the Collector would
verify only whether the compressed payload is beyond a certain limit, but not
its uncompressed version. A malicious payload could then send a
“[compressed bomb](https://en.wikipedia.org/wiki/Zip_bomb)”, causing the
Collector to crash.

Similarly, when sending a gRPC request using zstd compression, the decompression
mechanism would not respect the limits imposed by gRPC, also causing the
Collector to crash while decompressing the malicious payload.

A few business hours after the report, on 03 June 2024, Collector developers
were able to reproduce the report related to HTTP and worked together on a fix
that
[got merged the same day](https://github.com/open-telemetry/opentelemetry-collector/pull/10289).
Given the high score of this issue, we’ve decided to hold the release that would
have happened on the same day,
[completing it](https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.0)
on 04 June 2024 instead.

After the release, we got a confirmation that gRPC with zstd was also affected.
Within a few business hours of the confirmation, we worked on a fix that also
[got merged the same day](https://github.com/open-telemetry/opentelemetry-collector/pull/10323).
[We released v0.102.1](https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.1)
right after that.

## Am I affected?

You are affected by this vulnerability if you have an OpenTelemetry Collector
with one or more HTTP or gRPC receivers on a public port, such as the OTLP
Receiver with the “HTTP” or “gRPC” protocol enabled (typically on ports 4318 and
4317, respectively) AND the receiver has version 0.102.0 or below. The
vulnerability is exploitable only by attackers who can send payloads to your
HTTP/gRPC endpoint(s). This usually means that the port needs to be exposed to
the public internet or another network segment that's available to the attacker.

Note that if you require authentication, an attacker would need to have valid
credentials in order to exploit the vulnerability using the HTTP protocol. For
gRPC, the exploitable code is executed before authentication.

## What does it mean to me?

If you manage a Collector that has an interface to the public internet, you
should upgrade it as soon as feasible, and consider setting the parameter
`max_request_body_size` on HTTP receivers, such as the OTLP receiver, to a value
that makes sense to your workload. Up to v0.101.0, this setting applied only to
the payload size sent by the client, which could often be compressed.

Starting from v0.102.0, this setting applies to uncompressed, compressed, and
decompressed payload sizes and we are establishing a default value of 20 MiB for
this. This new default characterizes a breaking change, as clients sending
payloads bigger than 20 MiB will start seeing an error. While we believe most
authentic requests will be way within this limit, it’s still wise to monitor
your Collector for increased error rates after this update. Here’s an example of
a configuration setting a different limit to this field:

```yaml
receivers:
  otlp:
    protocols:
      http:
        endpoint: localhost:4318
        max_request_body_size: 10485760 # 10 MiB
```

For gRPC receivers, it’s sufficient to upgrade to v0.102.1, as there’s a default
value being applied to the message size already: 4 MiB.

If your Collector instances are receiving data only from trusted clients, like
your own applications, you are still encouraged to upgrade to the latest
Collector version but you can do it at your regular pace.

If you are using a custom distribution and building it with the
[OpenTelemetry Collector Builder (ocb)](/docs/collector/extend/ocb/), you can
add a
[“replaces”](https://github.com/open-telemetry/opentelemetry-collector/tree/d25efc7e2f31a3ba5347d0725a22d7bed1b4015d/cmd/builder?from_branch=main#configuration)
entry pointing to the latest version of the
[confighttp](https://pkg.go.dev/go.opentelemetry.io/collector/config/confighttp)
and
[configgrpc](https://pkg.go.dev/go.opentelemetry.io/collector/config/configgrpc)
Go module. If your base Collector version is at v0.96.0 or higher, we do not
expect any compatibility issues by just bumping to the latest version.

## Lessons learned

During this process, we found a couple of gaps in the telemetry for the
Collector, as well as to the options we provide to Collector admins. Concretely,
we noticed that we do not have a good way to verify what’s the distribution of
request sizes received by the Collector, which would have been useful to
determine whether the change would break clients for a given Collector. We also
noticed that we don’t provide a way for admins to completely disable
compression, which would be a good way to mitigate an attack without having to
upgrade the Collector. We are working to fill those gaps over the next releases.

We are also working on stabilizing the `component.UseLocalHostAsDefaultHost`
feature gate to reduce exposure of all Collector endpoints by default. This
feature gate was motivated by a previous,
[similar vulnerability on Go's standard library](https://github.com/advisories/GHSA-4374-p667-p6c8)
and has been in alpha for several months. You can follow the discussion
surrounding stabilization at
[issue 8510](https://github.com/open-telemetry/opentelemetry-collector/issues/8510).

## Acknowledgements

This issue was identified by [Miroslav Stampar](https://github.com/stamparm/),
from 7ASecurity. We’d like to thank 7ASecurity for the responsible reporting of
this vulnerability.
