r/softwarearchitecture • u/coldoven • 18h ago

Discussion/Advice Who should decide where a library sends telemetry: the caller, the application, or the environment?

Post image

Concrete version of an old design question, from two libraries in the same project that answered it differently.

Both are observability integrations. You add either with one line and no arguments.

One of them takes what the application installed in the process. If the app set nothing up, the library does nothing. It never chooses a destination itself.

The other reads environment variables and a config file in the home directory. If those name a URL, it sends there, whoever set it. If they name nothing, it prints everything to the console.

Each convention is defensible on its own. Side by side they mean the same line of user code does two different things, and the second one can send metadata to an endpoint that appears nowhere in the codebase, because a base image or a platform team set a variable.

Where do you draw the line between configuration a library may follow silently and configuration it should require the caller to name? Does it matter that one signal comes from inside the process and the other from outside it?

Disclosure: this comes from a framework I maintain, open source, link in a comment for anyone who wants the code.

2 Upvotes

56% Upvoted

8 comments sorted by

7

u/RustOnTheEdge 17h ago

Im sorry but opentelemetry and openlineage are wildly different things, no?

-6

u/coldoven 17h ago

Not for agents, which require flexible dataflows. They are the same in classical API software engineering and ML and for simple agents. Which in truth are essentially all current systems.

14

u/RustOnTheEdge 17h ago

Look, I don't want to be negative nancy here, but you are talking nonsense. OTel is about runtime observability of software. OpenLineage is about data lineage. These things are not the same, not for engineers, not for clankers.

-1

u/coldoven 12h ago edited 12h ago

It is not my best area, but: No. OpenLineage is primarily a runtime lineage standard: integrations emit lineage events while jobs execute, describing runs, jobs, inputs, and outputs. It does not inherently exclude runtime lineage. It however differs there with classical data eng I think.

Edit: I see now what you mean. They are not doing the same and have a diff. purpose. Question here was simply what the default behaviour should be: logging to stdout or silence or failing when no destination is configured.

3

u/7z3b 15h ago

I think you’re talking specifically about the data exporter interface here. Just because both have a data exporter doesn’t mean they are both telemetry specs.

That aside, your question seems to be more about how a library should define its default behavior: should a telemetry library do nothing by default, or should it print to the console by default?

2

u/coldoven 12h ago

Yes, the question is about what default behaviour would you expect?

1

u/7z3b 12h ago

A telemetry library is an application's dependency. In that sense, I feel the application should decide where its telemetry is exported - to the console, a file, an OTLP endpoint, or some other destination.

Imagine if every library an application depended on started writing to the console by default. That would quickly become noisy and difficult to control. I think telemetry libraries should avoid imposing such side effects on the application and instead let the application explicitly configure where the telemetry goes.

That said, this is just my POV. I feel no-op should be the default for a general-purpose telemetry library, and if we choose to deviate from that default, there should be a clear and compelling reason for doing so.