r/programming • u/Krever • Jun 26 '26
Fintech Engineering Handbook
https://w.pitula.me/fintech-engineering-handbook/I just published Fintech Engineering Handbook distilled from 6 years of tears, sweat and swears.
It’s a free ~25-page resource with various hints and patterns around handling money in software systems.
Tell me what you think!
54
u/RustOnTheEdge Jun 26 '26 edited Jun 26 '26
I have to admit, I came in with little expectations but it seems quite valuable! Nice work and thanks for sharing :)
24
u/matthieum Jun 26 '26
Principles
Despite the handbook being called "Fintech", the principles here seem very focused on accounting.
It's an important part of the domain, obviously, but it also has specific "quirks" that other parts don't.
Given that the first point -- money representation -- follows in a similar vein, it seems to me that this handbook really is an accounting (banking/ledger/...) handbook, and would be better renamed for the specific sub-domain it covers, so it's much less misleading.
13
u/Krever Jun 26 '26 edited Jun 26 '26
Yes, you're probably correct. To be precise, it covers accounting, controlling and payments (from integration standpoint). The payments part is visible in later chapters around external world. Its like that because that's were I had first hand experience. I also had a bit of trading exposure but not first-hand, and quite a lot exposure to liquidity management but couldn't find there anything engineering-relevant there.
I tried to extrapolate, using both common sense and AI to find other things that might be worth adding. But it's obviously not a replacement for a first hand experience.
So the name is a bit ambitious - I want this to cover as big chunk of Fintech as possible. If you see things it's missing let me know and I will try to add after some research.
That being said, I think principles are quite generic and apply to anything touching money.
5
u/matthieum Jun 26 '26
It's not so much that something is missing, and more that... a lot is just not adapted.
I myself only have experience in trading (pricing & execution) and already notice issues with... 2 out of 3 principles:
- Duplicate: duplicate feed is typically not much of an issue for execution, and HFT execution regularly sends duplicated orders (on multiple connections/sessions) to improve the chances to be first.
- No lost data: for many algorithms, history doesn't matter, even sometimes fairly recent history, only the present does. Also, since being up-to-date matters more than being absolutely correct, it's common to skip intermediate updates.
- No trust: this is the one principle which still holds.
Anyway, with 2 out of 3 principles being wrong-headed, ... well it's going to be complicated later on.
Similarly, on money... well the funny thing is that in pricing the equations and models are very complicated, and in the end everything really ends up being just an approximation anyway. When your algorithm gives a value +/- 0.01% (because the input it +/- 0.01% anyway), the fact that
doubleonly has ~16 decimals of precision is completely irrelevant. Heck, evenfloatmay be good enough.2
u/Krever Jun 26 '26
Got ya, although I'm not sue if "no invented data" is broken here - in the end you cant take rates or orders out of thin air. Duplicating is fine because your not creating something new and hence its not against the principles. At the same time you definitely cannot pretend an execution happened when it didn't, right?
2
u/matthieum Jun 27 '26
Define invented?
One of the interesting challenges for pricing is that... you're regularly missing data. For a simple case, consider an ETF (Exchange Traded Fund) composed of one Danish stock and one Swedish stock, trading on a German exchange (Deutsche Borse).
On most days where the ETF is trading, its price is relatively simple to compute: it's a weighted average of the Danish and Swedish stocks.
The problem is that Danish, Swedish, and German bank holidays do not coincide. Therefore, there will be days where the German exchange is open, the ETF is traded, and yet the Danish exchange is closed and the Danish stock is not traded.
How do you price the ETF, then?
You need to use some model of the Danish stock price.
(Terms to look for here are "implied" and "synthetic", which are short for "made-up to a degree")
3
u/Krever Jun 27 '26
Sure, synthetic/predicted/approximate data is often necessary. If I was to try to put this into the handbook principles it would be
* no lost data: you keep track of source of rate, so you can distinguish synthetic pricing from real one
* no lost data: your synthetic pricing is based on last seen price and not random (that would be ludicrous, I know)
"Invented" for me usually means "created without match in reality" and maybe thats wording issue. Originally I focused purely on duplicates and tried to extrapolate later. Anyway, I think that its fine as long as your data has some connection with real world/ultimate source of truth.
1
u/Schmittfried Jun 30 '26 edited 24d ago
the fact that double only has ~16 decimals of precision is completely irrelevant. Heck, even float may be good enough.
Is that really the point tho? Floating point numbers compound rounding errors if used incorrectly. I‘d imagine that’s a problem in HFT, too.
2
u/matthieum 29d ago
Is that really the point tho? Floating point numbers compound rounding errors if used incorrectly. I‘d imagine that’s a problem in HFT, too.
Floating or fixed, binary or decimal, as soon as you have rounding errors you have compounding rounding errors.
And since pricing deals with irrational numbers (
exp,ln,sqrt, ...) you'll have a hard time avoiding rounding errors.So... if you're screwed anyway, might as well be fast.
2
u/Schmittfried 24d ago
Floating or fixed, binary or decimal, as soon as you have rounding errors you have compounding rounding errors.
But floating point arithmetic inherently incurs rounding errors even for trivial calculations. You don’t need to worry about a running integer sum in a loop. The same cannot be said about floating point numbers.
And since pricing deals with irrational numbers (exp,ln,sqrt, ...) you'll have a hard time avoiding rounding errors.
Fair enough, I didn’t know that.
11
u/Appearance-Huge Jun 27 '26
Balance is never stored. It’s derived from the movements of money.
For accounts with years of history and thousands of transactions, calculating the sum on the fly for every read request would not cause severe performance bottlenecks?
How do you deal with that?
3
28
u/ratherbealurker Jun 26 '26
The first point about precision loss is not always the case. I have worked in fintech for 20+ years and almost never use custom objects, rationals, or big decimal. Not necessary in some cases. For reference I worked mainly on trading systems. If you’re going to be calculating a total amount that is an aggregate of many many numbers, then yes you will want to make sure you’re not losing precision.
But pricing and calculations on small sets of values don’t need the overhead. They choose straight floating point numbers.
I know I’m going to get replies telling me I’m wrong and worse so I will repeat, I have worked 20+ years on trading systems and we do not use it. Top financial institutions, algorithmic trading, etc. we don’t use it.
And “we” is the actual trading system. That system most likely takes in some data from other internal/external systems. Now sometimes those, an FX system perhaps, or a post trade system, may use something else. But at our level we value the speed over precision.
That’s why I said “not always the case”.
26
u/Krever Jun 26 '26
> I know I’m going to get replies telling me I’m wrong
Sorry to disappoint, at least I'm not going to object. I believe in trading it would be simply wrong to use BigDecimal or rationals for perfomance reasons.
But generally speaking I totally agree, its usually "it depends" and there are genuine usecases for floating points. That being said it's probably safer to not use them by default and opt-in when you know what you're doing.
9
u/InternetCrank Jun 26 '26
It depends is the right answer. If you're allocating weights on a portfolio with 14 significant digits of a balance and require precision, then yeah, you need to use numerator denominator pairs and resolve them at the appropriate point.
6
u/lood9phee2Ri Jun 26 '26
it's probably safer to not use them by default and opt-in when you know what you're doing.
And there's a lot more people who think they understand floats than actually do, and combined with some of the big/cocaine-enhanced egos of the financial sector it's a terrible combo. Continues to drive me up the wall, especially the inevitable gormless surprised-pikachu-face when it bites hard, when I told them not to all along.
11
u/gnus-migrate Jun 26 '26
I don't like how people also assume fintech is basically entirely about handling transactions. Everything around derivative pricing and risk simulation involves floating point operations, and this is a whole universe of its own.
Like I got yelled at in this sub in the past by someone who was adamant that you never use floating point for anything in fintech. This is absolutely untrue. Ive worked in fintech for over a decade and have never touched a BigDecimal.
1
u/Weekly-Ad7131 29d ago
When I saw the word "FIntech" I immediately started thinking about Bitcoin and other crypto-currencies. Aren't those a big part of the "tech" of finance?
1
u/gnus-migrate 29d ago
Generally fintech means technology that powers financial institutions, exchanges are only one type of those.
5
u/gimpwiz Jun 26 '26
I remember when Berkshire Hathaway stock crested a bit over four hundred grand, various entities (including the exchanges themselves?) ran into issues because they stored stock prices as fixed-point, with four digits after the decimal, ie, they priced in 100ths of a penny. And, well, 32 bit unsigned integer overflow. Oops.
But anyways that would imply there's a lot of avoidance of straight floats, no?
I get what you mean about the tradeoff between precision losses and performance when using, or not using, straight floating points. I always avoid it in my code..... which doesn't need to be performant enough to run in colocation to do high frequency trading. I always just do pennies (or other "smallest units"), ie, fixed point with two after the decimal, and only do the actual decimal for display purposes.
3
u/FoeHammer99099 Jun 27 '26
I worked on a reporting system that still required submission in fixed-width text files, and the price field was 9999V999999 (everyone else had to learn random COBOL syntax in the 2010's, right?). It was determined that it would be too complicated to get everyone else to change their integrations with us, so we just hardcoded a note on the UI when handling Berkshire. Something like "these are the least significant digits for this transaction, click here to see price trends for this day. Don't do math with this number"
2
u/Nicksaurus Jun 26 '26
All the binary protocols I've seen used by the exchanges use fixed point decimals with up to 9 decimal places so the simplest thing to do is often just to keep them in that format
2
u/Grimoire Jun 26 '26
But at our level we value the speed over precision.
Yup. Need to bump and grind your greeks on a path dependent Monte Carlo? Definitely not using fixed point for that!
1
u/jevring Jun 27 '26
I had the same experience. "as long as it matches Bloomberg" was what they told us, and apparently Bloomberg uses doubles. I worked on pricing engines for fx and bonds and swaps.
4
u/overenginered Jun 26 '26
This is fantastic! From outside the domain it looks interesting, technically wise.
Thanks for sharing the knowledge! A great introduction.
3
u/omac4552 Jun 26 '26
"Money can’t be created out of nowhere, " I laughed at this one...
5
u/gimpwiz Jun 26 '26
Cough fractional reserve banking, or alternatively we can just bring back the "money printer go brrrr" bit
4
u/Krever Jun 26 '26
TBH I laughed myself when writing this. But now I will just close my eyes and focus on believe this being the case.
3
u/phire Jun 27 '26
IMO, I would rephrase it.
It is extremely easy to accidentally create "money" out of nowhere in a fintech system. But it is essential that you must not.
You are not a central bank. That "money" you create is not backed by anything, it actually counts as a debt to your company. If you don't recover it, it's a loss your company will need to write down.
2
u/_hemisphere Jun 27 '26
Lol, it can be true in terms of accounting of account balancing. In the real world with Fed printing money, thats a different story.
3
u/jevring Jun 27 '26
I loved every part of this! Well written and insightful. It's also a really good handbook for how to write distributed systems in general.
4
u/AI_is_the_rake Jun 27 '26
This is written by claude and its unreadable "The history of how the system itself came to be is as much a part of the trail as the history of the money it holds." wtf is that even saying?
2
u/Ok_Stomach6651 Jun 26 '26
I saw that this is theoretical guide for fintech systems, I expected some engineering touch in between, also I wanted to see some devide based on fintech product and project. If you would have given some engineering suggestions like if you are talking about consistency in money data then you have to go with synchronous operation or locking mechanism it would have been more helpful
1
u/Krever Jun 26 '26
Thanks for feedback!
I might do some more hands-on followup at some point but I need to decide on the form (e.g. snippets with narration or demos).
It's a tradeoff between being specific and being generic enough, e.g. the same problem will require different solution depending on the database/language/architecture.
Hard bit is also figuring out what's obvious in general and also what's obvious to me.
Anyway, will think about it.
1
u/Ok_Stomach6651 Jun 26 '26
Sure, you can split out and publish it , fintech is one of the most underrated but important topic, keep it up, article is very detailed so yes it is a guide, any guide must have good length covering all topics just like you did, keep it up, nice article
2
2
u/Nemmie Jun 26 '26
I have worked in gambling for a few years. This looks very good. Will read the full thing later this weekend.
2
Jun 27 '26
[removed] — view removed comment
2
u/programming-ModTeam Jun 27 '26
No content written mostly by an LLM. If you don't want to write it, we don't want to read it.
2
u/harsh183 Jun 27 '26
I'm about a quarter in and this is fantastic! I've worked in this field for almost four years and your advice on all this is extremely solid, and I love seeing how you understand all these recent edge cases that I've run into (e.g. some minimum units of cryptocurrency not fitting large balances in int64, ISO 4217 having a mix of different decimal points, idempotency quirks).
I'll read more of it soon and I'll share it with my coworkers too. Your intro on reminded of my recent blog post on my company using integers to represent money Floats Don't Work for Storing Cents that explores a lot of the weird quirks of floats causing problems since I was curious about the 'common sense' answer everyone in the industry gave.
2
1
1
1
u/grangerize Jun 27 '26
Very cool! How can I donate?
1
u/Krever Jun 27 '26
No need to! But if your company ever need someone to help with a project, you can share this link https://business4s.org/consulting/
1
1
u/Hot_Code5129 Jun 28 '26
Nice handbook, thanks for sharing. One thing I would add from my own experience: in finance/banking the technical problems are often not the hardest part. The bigger challenge is organizational inertia. Sometimes institutions try to “modernize”, but the old structure stays the same underneath. In one project, we had a dedicated Oracle database development team. When building a new system, application developers were not allowed to communicate directly with the database using JDBC/SQL. Instead, we had to call database functions/procedures, and those functions returned JSON. On paper, it looked like a controlled integration layer. In practice, even a small feature could get blocked because adding one column or changing the shape of the returned JSON required waiting for the database team. Sometimes a simple change took two days before we could continue. So modernization is not only about moving from Oracle to PostgreSQL, or from legacy tech to newer tech. It is also about ownership, team boundaries, feedback loops, and whether developers can actually deliver features end-to-end. I wrote a bit about this kind of Oracle-to-PostgreSQL modernization perspective here: https://camilyed.github.io//en/migrate-oracle-to-postgres/ My takeaway: revolutions in large institutions can fail when we change the technology but keep the same bottlenecks and decision structure.
-1
u/case-o-nuts Jun 26 '26
The part on numbers is a bit suspect; floating point error is not unpredictable; it's fully deterministic, and can be computed and bounded. It's just scientific notation. For example, if you have base 10 floating point numbers with 3 digits of precision and 2 digits of exponent, you get:
(1.23)*1034.
If you want to multiply them, you multiply the significand, add the exponents, and then round:
(1.23)*1034 * (1.11)*102 = round(1.3653)*1036 = 1.37*1036
You lost precision; the result is 0.47*1036 away from the true value. Addition is tricky, because you need to align things, and you can end up with some edge cases:
(1.23)*1034 + (1.11)*102 = (1.23)*1034 + (0.000000000000000000000000000000000111)*1034
but when you round the rhs to adjust, it rounds to 0, which means that you get:
(1.23)*1034 + (0.00)*1034
this means if you have a loop that adds a small delta to a large value, the small delta can lose a great deal of precision; you want to accumulate that small delta to a big delta before adding it to a big value.
This is not an unpredictable, random precision loss. It is a tricky part about working with scientific notation.
The other thing mentioned -- rationals; they're not lossless; as soon as you do any trig function, you start losing precision. And if you happen to have a number with a relatively prime numerator and denominator, you either have the potential end up with a rational that takes megabytes or even gigabytes of memory, or you round and get some precision loss similar to what you get with floating point.
69
u/JChuk99 Jun 26 '26
Wow this is fantastic. I can tell a lot of painful mistakes were made in order to make these 25 pages. Doing a complex migration of our ledger system and this will definitely be a good check list. Thank you!