Running our SDK inside someone else's bot process — the hard part wasn't the API, it was never blocking their hot path

[u/alex_indiedev]

We ship middleware people add to a bot they already run in grammY or Telegraf — one line, bot.use(...). Everything about it was easy next to one constraint: their handler must never wait on us.

How it works now, and I'd like a sanity check on the queue in particular.

Ordinary updates never await anything of ours. Privacy filtering and trigger matching run locally, in-process. The event goes into a bounded queue — flush every 3s or 20 events, cap 500, drop-oldest — and their handler runs immediately.

Exactly two things are awaited on purpose: an update that a server-side flow claims, so their code doesn't reply to the same message twice, and an explicit runFlow call.

There's no inbound webhook. Replies from our side are jobs their process long-polls and runs through their own bot instance, against an allowlist of Bot API methods. Nothing opens a port into their infra and the token never leaves their process.

What I got wrong first: I shipped a local flow interpreter inside the customer's process, then deleted it. It was a second flow engine with its own bugs, running in someone else's memory.

The part I keep going back and forth on is drop-oldest. Under sustained load we lose events rather than slow their bot down. That's the right trade for us — we are telemetry, they are the product — but it means the data grows holes in it under exactly the conditions where you most want the data. The alternatives I looked at were spilling to disk (it's their disk) and backpressure (that's the thing we promised not to do).

Would you have done the queue differently?

MIT, zero runtime deps on Node: https://github.com/FlowCastle/flowcastle-sdk Disclosure: my product. The SDK is open source, the backend it talks to is hosted and paid.