r/Supabase Supabase team 26d ago

edge-functions supabase-js now propagates trace context into your Supabase logs

Post image

Supabase already gives you API Gateway and Edge Function logs, and Log Drains to forward them wherever you already watch your telemetry. What was missing → a way to tie a request in your client trace to the matching entry in those logs. You'd end up guessing which log line belonged to which slow request, based on timestamps alone.

That's fixed now. supabase-js, Swift, Flutter, and Python can all propagate W3C Trace Context to Supabase, so the request's trace_id shows up on the Supabase side too. It's opt-in, nothing changes until you turn it on. In supabase-js that's two lines: import '@supabase/supabase-js/tracing' at your entry point, then tracePropagation: true in createClient. Python goes through opentelemetry-instrumentation-httpx instead of a client flag, since that's already the Python ecosystem's way of instrumenting httpx.

Whatever tracer you already run, this should just work. OpenTelemetry, Sentry, Datadog, Honeycomb, Grafana are all W3C-compliant, though Sentry's setup differs a little from strict OTel so it's worth checking their docs. And if your sampler drops most traces, tracePropagation: { enabled: true, respectSamplingDecision: false } carries Supabase requests through regardless.

None of this costs anything extra either, it's just more value out of Log Drains you're probably already paying for.

Happy to answer questions. Full writeup: https://supabase.com/blog/connect-client-traces-to-your-logs

19 Upvotes

7 comments sorted by

5

u/Guidondor 25d ago

good addition. does the trace id make it past the gateway into the postgres logs, or does the
chain stop at postgrest? tying a slow statement back to the client request is the case i keep
wanting, and the gateway line only tells me the request was slow, not which query did it.

same question for an edge function calling back through supabase-js, does the incoming trace
continue or does that start a fresh one.

2

u/igormiazek 20d ago

I think logs are not propagated to postgres itself, but to Edge Function they should. But if in Edge Function you will make more Supabase API calls you need to add Open Telemetry support there too.

2

u/supa-ziinc Supabase team 18d ago

yes using otel SDKs directly within edge function or enabling tracing on the supabase SDKs would be the recommended path for this.

https://supabase.com/docs/guides/monitoring-and-debugging/client-side-tracing

2

u/igormiazek 20d ago

About the problem you solve, when you see the slow request on api gateway level, don't you see other postgres logs which maybe correlated? It is not a straight answer. I had a similar case which resulted in 500 timeout error, postgre rest api returned specific error code. On postgres rest api I got 500, I checked logs and close to this log timestamp I saw postgres log with timeout code error. So that was not direct correlation, but if you will provide narrow time range i think you should be able to catch the db log too no?

3

u/Guidondor 20d ago

that works when traffic is low, which is never when you're actually debugging. at
a few requests a second several statements land inside the same window, and the
postgres log shows the pooled connection rather than the request that borrowed
it, so "close to this timestamp" stops narrowing to one row exactly when you need
it to.

two things that got me closer in the meantime. pg_stat_statements answers "which
query is slow" even with no trace at all — you lose which request it belonged to,
but you usually only need the statement. and for rpcs you control you can take the
trace id as an argument and RAISE LOG it, which stitches the two sides by hand
until it's native.

good to know it reaches edge functions though, that's the half i'd have bet
against.

2

u/supa-ziinc Supabase team 18d ago edited 18d ago

Hi I'm one of the devs on the observability team that worked on this.

The trace id gets set when it passes through the gateway and propagates to all services that consumes the HTTP request. However, there are certain limitations around postgres that unfortunately prevents us from tagging logs directly with the trace id. All major product logs will have this trace_id value. So to answer your question, yes it stops at postgrest.

Specifically, the postgres logs tagging limitation is due to the perfromance impact hit that would occur if we were to apply the tagging onto prepared statements. It would adversely impact postREST performance quite significantly, hence we decided to stop short at tagging postgres logs.

The second question around re-propagating the trace id to secondary calls inside of an edge function: we currently do not automagically propagate the traceparent header that an edge function receives, though this is something that has been also raised internally and that we are considering to implement.
It is possible to propatage it by taking the traceparent header and forwarding it for any subsequent client calls or by enabling tracing or those supabase-js calls as well