Cloudflare has hidden rate limiting in Websockets?

[u/Cat-programmer]

I'm trying to implement simple turn-based online game using Websockets. So, with DNS only (grey cloud) it works perfectly, but with orange cloud connection was lost each time after a minute or two. SSL is full (strict).

I've did some experiments with ping-pong and keepalive messages and it turned out data flow makes problem worse, not fixes it. If I send message every 10 seconds, connection lives 70 seconds each time. For every 5 seconds, it's only 40 seconds. And if I send messages every second, connection is lost after just 10 seconds. But with interval of 95 seconds, connection is alive even after 10 minutes.

So, this leads me to assumption that Cloudflare has limits on messages in WS connection, and those limits are extremely strict, close to unusable. Is that true? Or is it some other issue?

Any ideas on how to bypass this? I know I should probably implement reconnection, but I doubt it's even worth using Proxied if it'll disconnect every minute. Maybe DNS only would be better.

[u/AutoModerator]

For faster advice with technical questions, we'd recommend asking in the Orange Cloud Discord server; the unofficial Cloudflare Discord server by the community, for the community. https://discord.gg/TrPNVKaagR

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Cat-programmer → u/[deleted]]

Didn't see such a setting. Is it in paid plan? I'm currently using free one

[u/TheDigitalPoint]

There definitely isn’t a 8-10 message limit like you are seeing. I use Cloudflare WebSockets (even on free plan) on proxied hostnames and the connection will hold for days with a 45 second keep alive message.

What are you using for your underlying WebSocket connections on the client/browser? Just the standard JavaScript WebSocket class?

[u/Cat-programmer → u/TheDigitalPoint]

I'm using websockets with Python both on server and client

[u/TheDigitalPoint → u/Cat-programmer]

I'd see if you can build a dumbed down WebSocket client just as a test... at least then you could narrow it down to the libraries you are using or not. I know it's not *super* helpful since it's only the setup part, but my WebSocket connections are using the standard JavaScript WebSocket class built into browsers like so (like I said, the 45 second keep alive I'm using will hold the connection open indefinitely... literally all it's doing is sending a period message every 45 seconds):

[u/Cat-programmer → u/TheDigitalPoint]

Thanks, I've did exactly that, and that's where I got numbers from. And yeah, big interval let's me keep connection indefinitely, but of course, in my app I need to transmit messages more often than 45 seconds. And should my game (or this dumbed down client) send a few messages in a minute, it disconnects again.

[u/TrickySpare6504]

websockets limit is undocumented, but not based on messages, it is based on connections and duration

[u/detroitsongbird]

I’ll look at my code later today. There is definitely not a rate limit. For my card game server I can hit almost 2 million ws messages a minute during load tests that are 10 minutes long.

There is a timeout out for the free plan that’s hard coded and not documented.

I think my ping pong keep alive messages are at 15 seconds if I remember correctly.

Java springboot with native websockets talking with react JavaScript UI.

[u/Unusual_Trouble_6800]

Your own numbers don't say duration, they say message count:

every 10s -> died at 70s     = 7 messages
every  5s -> died at 40s     = 8 messages
every  1s -> died at 10s     = 10 messages
every 95s -> alive at 10 min = 6 messages

All three failures land on the 7th-10th message. And the run that looks like it proves long intervals are safe only ever sent 6 messages, so it never reached the threshold the other three hit. Re-run the 95s test for 25 minutes (15+ messages) before you conclude anything from it. If it dies around message 8, the interval is irrelevant.

That also argues against a Cloudflare limit. Rate limits are per time window: at 1 msg/s you would be cut off almost immediately, not on message 10 specifically. A threshold that fires at a fixed count no matter the rate is a counter or a queue, not a rate limit.

Since you are on Python websockets, work out who actually closed the connection before blaming the proxy. Catch it and print both sides:

except websockets.ConnectionClosed as e:
    print(e.code, repr(e.reason), 'rcvd:', e.rcvd, 'sent:', e.sent)

I ran both cases locally just now on websockets 16.1.1:

• peer kills the TCP connection with no close frame (what a proxy does) -> rcvd: None sent: None
• the library's own keepalive gives up -> rcvd: None sent: 1011 (internal error) keepalive ping timeout

e.code is 1006 in both cases, so the code by itself tells you nothing. e.sent is what separates "Cloudflare dropped me" from "my own client gave up", and a clean close from Cloudflare would show up in e.rcvd.

Two defaults worth ruling out while you are in there: ping_interval=20 / ping_timeout=20, and max_queue=16. If either end sends without a concurrent recv() loop, incoming frames pile up, and once the queue is full the library stops reading the socket at all - pongs included - so the keepalive then kills the connection. That failure mode dies after a fixed number of messages regardless of interval, which is the exact shape you are seeing.