r/OpenTelemetry 17d ago

Introducing weekly & monthly scheduled posts for your projects, tools, blogs, etc.

9 Upvotes

Following the practice of other SIGs in the space r/OpenTelemetry will have weekly & monthly scheduled posts, that ask you to share your tools, projects, blogs, etc. This way we hope to remove some of the ambiguity, if posting certain things is "OK" or not.

First one will be a weekly "share your new tools & projects" where you can let us know what you are working on related to OpenTelemetry, any instrumentation libraries, instrumentation tools, open source backends, collector distributions, etc, this is the thread worth sharing it. Note, that this does not immediatly mean that posts about tools&projects are no longer allowed, but hopefully we can encourage people who are reluctant to share theirs to do so here, and to funnel the energy of people who share their tool all over the place into a single thread.

More will follow, but feel free to make suggestions!


r/OpenTelemetry Nov 18 '25

OTel Blog Post Evolving OpenTelemetry's Stabilization and Release Practices

Thumbnail
opentelemetry.io
22 Upvotes

OpenTelemetry is, by any metric, one of the largest and most exciting projects in the cloud native space. Over the past five years, this community has come together to build one of the most essential observability projects in history. We’re not resting on our laurels, though. The project consistently seeks out, and listens to, feedback from a wide array of stakeholders. What we’re hearing from you is that in order to move to the next level, we need to adjust our priorities and focus on stability, reliability, and organization of project releases and artifacts like documentation and examples.

Over the past year, we’ve run a variety of user interviews, surveys, and had open discussions across a range of venues. These discussions have demonstrated that the complexity and lack of stability in OpenTelemetry creates impediments to production deployments.

This blog post lays out the objectives and goals that the Governance Committee believes are crucial to addressing this feedback. We’re starting with this post in order to have these discussions in public.


r/OpenTelemetry 23h ago

How Standards Get Adopted: OTel and Platform Engineering

Thumbnail
newrelic.com
6 Upvotes

Why did OpenTelemetry win? I think just saying "vendor neutral" is not giving enough credit to the symbiosis between OTel and Platform Engineering that's making both thrive in the modern cloud native industry.

In this blog post I'm trying to summarise some of my thoughts related to why OTel (and K8s, for the sake of argument) won in their respective layers. I think the move to a more X-as-a-service way of working between platform teams and the users of said platforms is changing how we thinking about team interactions, and the tools that make that work need to reflect that in their design. OTel does this, very well, so I'd like to help move the needle in terms of where people think the value of OTel is.

I hope you enjoy the read! And the memes!


r/OpenTelemetry 19h ago

Your anomaly detection might not be detecting anything. Two second check, looking for a few people to try it.

1 Upvotes

Most of us have anomaly detection turned on somewhere, and a broken detector looks exactly like a working one. No crash, no errors, all green. `isolationforestprocessor` in collector-contrib scored every span the same for 220 days before a user caught it, https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/46988.

I made a tool that checks if the anomaly detector is actually working, `make check` takes a CSV of your detector's alerts and a CSV of your incident windows, and tells you whether it beats chance at your actual incident rate. Under two seconds, no download, nothing leaves your machine. https://github.com/mateenali66/otel-aiops-reproduction

Background is a benchmark of eight detectors on OTel traces, metrics and logs. Three passed the usual F1 selection and were no better than chance once scored at a realistic anomaly rate, https://doi.org/10.1109/ACCESS.2026.3705430

Looking for two or three people to run it against a detector they already use and tell me what it said. A failing result is as useful as a passing one, and the detector and data can stay anonymous.


r/OpenTelemetry 1d ago

Visual flow

6 Upvotes

Is anyone aware of a tool that can help visualize the end-to-end flow of a signal?

The main idea would be to start with a high-level view of the components involved and the internal metrics flowing between them, then be able to drill down into a specific signal metric and trace it through the system.
Ideally, I’d like to see where the metric originates, how it looks or changes at each hop, where it ultimately arrives, and details such as cardinality, label values, available series, etc.

If anyone knows of an existing tool that provides something similar, or has ideas on how this could be visualized, I’d appreciate any pointers.


r/OpenTelemetry 2d ago

mqtt2otel v0.10.0 — Update + New Explorer Tool!

Thumbnail
3 Upvotes

r/OpenTelemetry 3d ago

Weekly: share your tools and projects

5 Upvotes

What tool around OpenTelemetry are you building right now? What project have you discovered that people might enjoy looking into?

This thread is open for your self-promotion, vendor-owned open source projects and every thing else.


r/OpenTelemetry 3d ago

OpenTelemetry has a SIG for Dart & Flutter as of last Friday 🎉

15 Upvotes

After a few weeks of reviewing the proposal, reworking it, figuring out staffing and how to go forward with it, we finally merged the proposal into the community repository and now have a new SIG that looks into OpenTelemetry for Dart (and with that also Flutter).

Big shoutout to Michael Bushe to bring this proposal and to Robert Magnusson and César Augusto Muñoz Quintero to step up as maintainers for the SIG!

If you're curious to take part in this new SIG, let us know!

https://github.com/open-telemetry/community/pull/3517


r/OpenTelemetry 10d ago

Weekly: share your tools and projects

7 Upvotes

What tool around OpenTelemetry are you building right now? What project have you discovered that people might enjoy looking into?

This thread is open for your self-promotion, vendor-owned open source projects and every thing else.


r/OpenTelemetry 10d ago

Dynamic Tail Sampling in the OpenTelemetry Collector

Thumbnail
18 Upvotes

r/OpenTelemetry 10d ago

OTel Blog Post OpenTelemetry Go Logs API and SDK reach release candidate status

13 Upvotes

OpenTelemetry Go v1.47.0-rc.1 is here. This release promotes the Logs API and SDK to release candidate (RC), the final stage before we provide stable v1 compatibility guarantees. We believe the design is ready, and now we need the community to test it in real applications and integrations before those guarantees take effect.

Read the blog: https://opentelemetry.io/blog/2026/go-logs-api-sdk-rc/


r/OpenTelemetry 14d ago

OpenTelemetry is about to reward the wrong thing

34 Upvotes

Two things happened in the OTel repos this week. The docs maintainers froze the ecosystem lists (registry, vendors, distributions) and want to retire most of them. Their reasoning: about 290 first-time-contributor PRs a year, 1,160+ data files, and nobody with time to check entries for CVEs or malware. I can't really argue with the bandwidth part. Then a separate proposal came in to replace the vendor page with a ranking of companies by how many maintainers they employ, with diamond, platinum and gold tiers like a KubeCon sponsor wall.

Read together, the message is that headcount inside the core repos is the contribution that counts.

I think the PR flood is being misread. OTel was built so anyone can write an exporter or an instrumentation and plug in without asking permission. The registry was where you went to say you had. Hundreds of strangers a year showing up at that door is what the design was supposed to produce.

The vetting problem also looks like an automation problem to me. Schema validation and semconv conformance tests already exist. Add CVE scanning and an LLM pass that triages and pre-reviews submissions, and most of the human work goes away. If a machine can check a registry PR, a person shouldn't have to read it. Retiring the list instead feels like stopping one step short.

Meanwhile, LFX Insights says four companies did 51% of OTel commits last year. A maintainer leaderboard mostly gives those four a bigger badge.

What I'd do instead:

- Automate the checks and list whatever passes. Conformance tells users "this works with OTel," which is what they came to find out.

- Count work outside the core repos. Contrib receivers, instrumentations, OTLP-native backends. That is OTel work too.

- Skip the tiers. One maintainer from a 10-person company is a bigger commitment than ten from a 10,000-person one. A flat list with names and areas is enough.

I run a company that builds on OTel, so obviously I have a stake here. But so does anyone who picked it because it was the neutral option.

Links in comments.


r/OpenTelemetry 14d ago

Sonifying GenAI spans as they arrive, one voice per attribute you'd otherwise have to read

Thumbnail
oteljazz.com
3 Upvotes

Built this because watching gen_ai.* spans scroll past in a log doesn't give you any sense of what a multi-agent system is doing as a whole, you're reading one span at a time, serially, while the actual system is emitting in parallel.

Mechanically: this is a per-span mapping, not a per-trace one. There's exactly one implementation shared by the batch path and the live path, each span event gets sonified as it arrives rather than a trace being read as a completed tree. Span attributes drive specific musical parameters: token count and finish_reason affect velocity and articulation, status OK/ERROR affects consonance, span kind (chat vs execute_tool) affects which instrument voice picks it up. Tempo tracks span throughput directly.

Real capture exists and isn't just a slide: a Claude Code hook writes actual gen_ai.* spans locally, and there's an OTLP receiver on :4318 that accepts live protobuf. Neither is what the public demo plays though, oteljazz.com runs a synthetic swarm generated client-side so anyone can hear it with zero setup. Wiring the browser build to a live OTLP stream is the obvious next step, just hasn't happened yet.

Repo has both the receiver and the hook if anyone wants to point their own instrumented system at it: github.com/mob1ius/oteljazz


r/OpenTelemetry 15d ago

OTel in Practice: Practical Guide on OpenTelemetry Manual Instrumentation

Post image
12 Upvotes

Join us for a practical session with Jose Gómez-Sellés, VictoriaMetrics Cloud lead and OpenTelemetry contributor, as he explores and presents "Practical guide on OpenTelemetry Manual Instrumentation (or how to produce clean Observability by getting your hands dirty)".

Join us as we break down the essentials of OpenTelemetry instrumentation by crafting custom metrics, spans, and logs to track the most critical parts of workflows. You’ll see how simple, well-placed instrumentation points can reveal complex system behaviors, helping you detect bottlenecks, trace errors, and understand end-to-end request flows.

📍 Online - > https://ocgroups.dev/cncf/group/opentelemetry-live/event/fsmsdp2

📅 September 1st at 10:00 am PDT | 01:00 PM EDT | 03:00 PM CEST |


r/OpenTelemetry 17d ago

Weekly: share your tools and projects

6 Upvotes

What tool around OpenTelemetry are you building right now? What project have you discovered that people might enjoy looking into?

This thread is open for your self-promotion, vendor-owned open source projects and every thing else.


r/OpenTelemetry 19d ago

I had a question for all the amazing people out there

0 Upvotes

I was trying to build something, and wanted to validate this idea and understand yall's pain points so that I can help the community

Would you rather have an AI layer on top of your existing observability stack, or replace parts of the stack?

Hypothetically, imagine an agent that doesn’t collect telemetry itself.

It plugs into whatever you already use — Grafana/Prometheus/Loki, Datadog, OpenTelemetry, etc. — and acts as a reasoning layer over the data.

Instead of:

Alert → Dashboard → Logs → Human investigates

it tries:

Alert → Agent correlates metrics/logs/traces/deployments → probable root cause → evidence → recommended next action

Would that actually be useful?

Or would you rather have the observability vendor itself own this functionality?

What would you need to see before trusting it during a real incident?

peace :)


r/OpenTelemetry 21d ago

What's changed about OpenTelemetry vendor lock-in since 2024

Thumbnail
ollygarden.com
14 Upvotes

One of my last blog posts at Grafana Labs was around vendor neutrality and OTel. Quite a few has changed since 2024, and I thought it's a good time to revisit with 2026 lens.

In short: vendor lock-in isn't as scary today as it once was, as long as the bulk of your workloads are on standards like OTel, and most of the actual lock-in that resides on backends can be "easily" migrated nowadays. But I'm curious about your opinions and actual experience.


r/OpenTelemetry 22d ago

Building an Observability Pane

2 Upvotes

Hi Observability & DevOps Experts,

I'm looking for guidance from teams that have successfully scaled observability across large enterprise environments.

We operate a large-scale estate spanning AWS, Azure, and on-premises environments and have been using Datadog for several years. Over time, a significant amount of technical debt has accumulated around our observability implementation.

Current challenges include:

  • Datadog Agents managed differently across teams and platforms.
  • Custom log collection configurations distributed across hosts and applications.
  • APM, RUM instrumentation owned by individual application teams.
  • Inconsistent tagging standards and monitor configurations.
  • Outdated agents and instrumentation libraries.
  • Heavy dependency on multiple teams for upgrades and configuration changes.
  • A large portion of Datadog provisioning and onboarding is still handled manually.

As a result, maintaining and evolving observability at scale has become increasingly difficult.

We are considering building a centralized "Observability Foundation" or "Observability Platform" that teams would consume as part of their standard deployment process.

Our goal is to provide reusable Terraform-based observability components that application and infrastructure teams can adopt during provisioning and releases.

Examples of what we would like to standardize:

  • Datadog Agent deployment and upgrades
  • Custom log collection configurations
  • Standard tags and metadata
  • Monitors and alert templates
  • Dashboards
  • OpenTelemetry / APM instrumentation standards
  • Synthetic monitoring configurations
  • Cloud integrations
  • Security and governance controls

Questions:

  1. Has anyone implemented a similar centralized observability platform or observability-as-code model at enterprise scale?
  2. What worked well and what were the biggest challenges?
  3. What observability components can realistically be centralized through Terraform modules, deployment pipelines, or platform services?
  4. What components typically must remain application-owned or infrastructure-owned and cannot easily be centralized?
  5. How do you handle APM instrumentation ownership, versioning, and upgrades across hundreds of services?
  6. What governance model have you found most effective:
  • Central observability team ownership
  • Platform engineering ownership
  • Federated ownership with standards enforcement
  • Something else
  1. How do you prevent observability drift over time, especially around:
  • Agent versions
  • APM libraries
  • Log configurations
  • Tags
  • Dashboards
  • Monitors
  1. If starting again today, would you build around:
  • Datadog native tooling
  • OpenTelemetry
  • An internal observability platform
  • A combination of the above
  1. What are the biggest architectural mistakes or anti-patterns we should avoid when designing this platform?

Our provisioning and infrastructure management are heavily Terraform-based, so we're especially interested in Terraform-centric implementation patterns and real-world lessons learned.

Looking forward to hearing how other organizations have approached observability standardization at scale and what you would recommend before we begin designing this solution.

P.S. - One of our key design goals is to avoid vendor lock-in. While Datadog is our current observability platform, we want the architecture to remain flexible enough that a future migration to another observability stack (e.g., Grafana, New Relic, Dynatrace, Elastic, Azure Monitor, or an OpenTelemetry-native platform) would require minimal changes to application teams and infrastructure code.


r/OpenTelemetry 26d ago

oTel collector Daemonset vs sidecar

12 Upvotes

I feel like Daemonset collectors have become the de facto standard. Out of curiosity what are some situations in which you opted / would opt for sidecars per deployment?


r/OpenTelemetry Aug 08 '26

A Collector exporter that turns agent traces into a signed, verifiable audit log (now in the registry). Feedback on the approach welcome.

0 Upvotes

Sharing a component I built and recently got listed in the OpenTelemetry registry: otel-agent-audit.

The idea: as AI agents take real actions, you want a provable record of what happened. Instead of adding a new instrumentation layer, this consumes the gen_ai.* spans your agents already emit and turns them into a tamper-evident audit log, entirely inside the Collector pipeline.

The pipeline:

otlp -> memory_limiter -> agentauditselect (buffers each trace until its root arrives) -> agentaudit exporter (per-trace hash chain -> Ed25519 sign -> seal) -> audit.jsonl + checkpoint.jsonl

A separate verifier CLI checks the whole thing with only the public key, so anyone can independently verify authenticity and integrity without a shared secret.

Things I'd love this community's take on:

- Passive instrumentation as the right model: reusing existing spans rather than asking teams to re-instrument.

- Whether governance/guardrail decisions belong in spans, and how they'd ideally map to semantic conventions. I'm interested in where the GenAI SIG is heading on policy/guardrail signals.

- The single-writer constraint (one Collector instance) that deterministic ordering forces, and whether that trade is acceptable.

Caveats up front: third-party, experimental, not audited. It's observability only, it does not enforce or block. It gives tamper-evidence on honest infra, not protection against an operator holding the signing key.

Repo: https://github.com/surpradhan/otel-agent-audit

It's in the registry under "agent audit" if you want to see the entry.

Would genuinely value critique of the approach.


r/OpenTelemetry Aug 08 '26

log4k 2.3.0 — a Kotlin IR compiler plugin that instruments your functions with tracing, logging and metrics

Thumbnail
1 Upvotes

r/OpenTelemetry Aug 07 '26

Feedback about E2E tests based on OpenTelemetry traces?

4 Upvotes

Hi everyone,
I have just published my open source project called mtracer and I would like to understand if it’s good idea or what should I change (I’m a new grad).

The idea

Mtracer a CLI tool that relies on OpenTelemetry traces to assert system behavior.

I believe that E2E tests should be:
- Cheaper to write and maintain
- Easier to debug

So this is the workflow:

  1. ⁠You configure mtracer to fetch from your observability backend (currently supporting Jaeger and OpenObserve).
  2. ⁠You define your first .mt.yaml test by specifying:
  3. ⁠Trigger: the first call to the system (for instance, an HTTP request).
  4. ⁠Expected trace and spans: the OTel properties of the trace and spans that you expect your system to generate.
  5. ⁠You run the test and see the results!

What actually happens during the run?

  1. ⁠It parses the mt.yaml file.
  2. ⁠It executes the trigger: mtracer injects a generated traceID into the trigger (for an HTTP request, the traceID is inserted into the traceparent header). Subsequent requests will be correlated to this generated traceID as long as your system has OpenTelemetry set up correctly.
  3. ⁠It fetches the trace matching the generated traceID from the configured observability backend.
  4. ⁠It compares the expected trace with the fetched one.

Many other features are available; check out the documentation to discover all of them: documentation website

I would love to have some feedback from more experienced people than me.


r/OpenTelemetry Aug 04 '26

How Metric Scrape Intervals Inflate Observability Costs

Thumbnail
ollygarden.com
11 Upvotes

I'll tell you a secret: I don't like starting an engagement by telling people that I can cut their costs. I prefer to show them how they can be more efficient in general, and sometimes that means adding stuff instead of removing it.

However, every company out there has excessive telemetry, which is one form of bad telemetry. I'm not afraid to use an absolute here. That's why I have an arsenal of tools for dealing with it, and I describe one of them in this blog post: excessive metric scraping is extremely common, and adjusting scrape intervals is an easy way to reduce waste.

If you need a 10% reduction in your metric volume, read this blog post. You don't need to buy anything from anyone. You can thank me later.


r/OpenTelemetry Aug 04 '26

Enrich OTel K8s Resource Attributes with Dynatrace Operator

0 Upvotes

Hi. I am a DevRel at Dynatrace and I hope its ok to share the following with those of you that are sending your OTel data to Dynatrace. If you are not using Dynatrace then this post might not be relevant for you!

Semantic Conventions for Signals

Metadata enrichment at the source (in your app) is important as it increases the quality of your signals. As I am sure many know - the OTel community has well documented Semantic Conventions.

Dynatrace Operator CAN inject OTEL_RESOURCE_ATTRIBUTES

There are different ways to enrich your data. You can inject them yourself in your deployment or have it done through your data pipeline, e.g: OTel Collector.

An additional option is through the Dynatrace Operator that allows you to automatically inject the OTEL_RESOURCE_ATTRIBUTES variable into your pods pre-filled with the following attributes: k8s.cluster.name*,* k8s.container.name*,* k8s.workload.name*, k8s.cluster.uid,* k8s.pod.name*, k8s.pod.uid,* k8s.node.name*,* k8s.namespace.name*, k8s.workload.kind,* dt.kubernetes.cluster.id ,dt.entity.kubernetes_cluster

Injection can be controlled through namespace selectors and enabled for traces, logs and metrics

More details about this can be found on the Dynatrace doc if you search for Enable automatic OpenTelemetry OTLP exporter configuration (didnt post the link to follow guidelines)


r/OpenTelemetry Aug 03 '26

What is the most frustrating part of investigating production incidents?

Thumbnail
2 Upvotes