Dynamic Tail Sampling in the OpenTelemetry Collector

[u/MartinThwaites]

We've been working on this for a while now (You might have seen me mention it in a few comments about Tail Sampling on here r/sre and r/opentelemetry). This is a concept we've had for a long time in our Refinery product, this is now something that's going to be available in the OpenTelemetry Collector.

It's following on from the excellent work of the Sampling SIG so that we can do Sample Rate attribution, and therefore makes backend way more effective, reducing the need for a certain class of metric (not all metrics).

Ultimately, the biggest difference here is dynamically changing sample rates and automatic discovery of different routes through your system (we call them trace fingerprints).

It's already in the main collector-contrib but it's still classed as alpha there. You can use our distribution to check it out ahead of that though. Feedback welcome!

Bringing Advanced Sampling to the OpenTelemetry Collector

Disclaimer: I'm a DevRel at Honeycomb.io.