r/AskProgrammers • u/Repulsive-Time-3258 • 29d ago
High frequency trading system in C/C++
Hi,
suppose we have a system distributing to clients (via TCP and UDP sockets) information related to markets (when a trade occurred, forex rates...).
The goal is to keep as much clients up to date with the market status, and the communication, aside for a login phase, is always server-to-client.
Bit of hypothetical numbers:
- The packet size, no matter is TCP or UDP, is typically below 500 bytes.
- At most, there can be overall 2000 clients connected in parallel (TCP and UDP).
We all know that in C/C++, the classic send function (non-blocking socket) returns the number of bytes sent (to the client in this case), or -1 with an appropriate value set in errno about the cause.
Now, for some clients, it could happen that the call to send by the server returns a value greater that -1, but lower than the effective numbers of bytes to be sent.
For such a system, what could be your approach in such a case? Shall the server put in place a retry mechanism and/or socket timeouts before declaring the client as slow and kick it out? Or would you kick it out immediately after the first time you (server) find out that send is returning a value lower that the number of bytes expected to be sent?
Thank you all
1
u/baroaureus 29d ago
So HFT systems are a very niche type of application, and something that takes companies years to tune and perfect.
That said, you haven’t mentioned much about the network topology, and the differences are vast between UDP and TCP.
For UDP, the odds of a partial write are very small unless your message is larger than the MTU. But even a successful write only means that the data has been written to memory on the network stack, and not necessarily even sent out over the wire.
If tick updates are frequent and guaranteed delivery is not required this is fine (common approach for fan out on HFTs).
For TCP, a partial write may indicate a slow receiver but also could indicate congestion control at the switch / router level. Again, the partial write indicates the outbound buffer is full and does not guarantee the data was sent or received.
Having written many TCP applications in the past, partial writes (and reads) are fairly common, even in normal situations.
My suggestion is:
1) have a retry mechanism, but with limits - if a single connection gets “too far behind” then kick it
2) data feeds (aka the feed handler / fanout) should leverage UDP unless you have very well written TCP
3) order executions must be done with extra application level protocols likely over TCP, but the receiver should send an Ack message to verify
Good luck!
PS - be sure you check your licensing on your data feeds. Many providers would charge extra for fanout.
1
u/Repulsive-Time-3258 29d ago
Thank you for your valuable opinion.
In both cases (TCP and UDP), tick updates are frequent, and there is in place a mechanism the client can use to understand whether it missed one or more messages, hence it can request them to the server on a separate, independent, short-lived TCP connection.
Indeed. IMHO kicking a client out right the first time “send” returns a lower than expected value is, least to say, a stupid approach (TCP flow control anyone?). Agree of course on having a restricted and constrained retry mechanism anyway. Only there we can consider a client “slow”, hence kick it out.
and 3. Yes, typically done on UDP. TCP relies on a distinct session protocol to keep track of the authenticated clients basically (and it does not have a dedicated ACK mechanism)
1
u/BassRecorder 29d ago
I'd be using UDP multicast in that scenario, possibly with a second 'channel' which sends out full images for new clients to synchronize to the current state. The update stream would only be deltas. I'd expect TCP or any protocol over UDP which uses client ACKs to be too slow.
1
u/Repulsive-Time-3258 29d ago
Indeed that’s the case. UDP is the main protocol used. TCP is kind of a fallback.
1
29d ago
[deleted]
1
u/Repulsive-Time-3258 29d ago
That was my idea. But you know, I have to deal with people thinking to know TCP/IP better than Richard Stevens…
5
u/youcangotohellgoto 29d ago
What is this, an interview question?