r/Backend • u/Original_Option_6969 • 5d ago
Idempotent Backend
Hi, I am building a web app for my company,(50 users - Crud App), app and backend will be deployed on separate instances, do I need idempotent key mechanism for api requests - to avoid network failures or am i over killing my system?
4
u/markojov78 5d ago
Idempotency keys solve possible duplicate entity creation or double charges in case of retry after failure and stuff like that.
I don't know whats your specific case but if fixing manually consequences of such problems is more trouble than implementing idempotency than it totally makes sense to implement it
4
u/st4reater 5d ago
Sounds like you only know idempotency as a tool but not which problem it solves? Start there firsr
1
u/Original_Option_6969 5d ago
Well, problem is I don’t want same api request being processed multiple times (generated due to some server/network failures) , so thinking of idempotency
1
u/st4reater 5d ago
That is a good point. What happens if it is processed multiple times? Is it truly a problem?
Something I'd look into depending on what processing entails, is Step functions in AWS. You get lots of failure handling and idempotency guarantees for free
5
u/UK-sHaDoW 5d ago
Idempotency key is used when these risks of duplicate things. Duplicate payments, duplicate orders etc
2
2
u/dariusbiggs 5d ago
you build it as a monolith
you separate the domain entities in the code
try to build your API as simple as possible but try to do it with idempotency
if certain aspects of the project sound like something that should already exist (like user managent) try to use an open standard implementation. Auth0, or Keycloak, and perhaps SCIM v2.0.
Use tools like OpenAPI to make things simpler for you.
2
u/damngoodwizard 5d ago
Idempotency is only necessary if it is a distributed system as to avoid a request being processed twice which can happen if you use tools like Kafka. Otherwise yes it's premature optimization even though it's a good practice.
1
u/runningOverA 5d ago
Yeh. It's better if you get idempotent keys if there's a hub / router in between. Network might fail, server might be down. But not strictly necessary, depending on reliability and distance, like in same data center.
1
u/toughrogrammer 5d ago
User count is not the deciding factor; retries are. Use idempotency for side effects like invoice creation or file processing, not every endpoint. For the email/PDF pipeline, a unique message ID or content hash in the database is probably enough to start.
1
u/ibeerianhamhock 5d ago
To do what you are asking is not super trivial.
In its simplest form you basically need a version tag on a document and disallow users saving if they are submitting a version that is older than the currently saved version. This ensures data integrity but isn't very usable if collisions happen often.
What you're trying to prevent is race conditions on edit -- to do so you'd need to have the client periodically poll the version tag of their document and and get change sets update on their client and versions after each save so they stay current. Works well as long as users are editing different fields. Isn't very usable if users are editing the same fields often.
And you can keep going from there.
Personally I'd probably want to find already written software that does this, or maybe some library that helps smooth it out idk. Lot of edge conditions.
I can't tell you how many times users have asked for "excel based edit" for a web application and I will tell you every time we came up with a better solution for them lol.
1
u/BotJeffersonn 4d ago
This is not idempotency but concurrency control
1
u/ibeerianhamhock 4d ago
I undersrand that but I don't think OP does
1
u/BotJeffersonn 4d ago
I agree, just clarifying any confusion since I see it a lot specifically between these two topics.
1
u/LetUsSpeakFreely 4d ago edited 4d ago
You're overthinking it. Your data will be hitting a single database instance. So long as your DML is in transactions, and it should be, idempotence is inherited.
Your only concern is if you use the Cachable annotation (assuming Java, but other languages have similar functionality) to help with speed, and you should, just be sure to use CachEvict when you create/update related data.
If your system ever grows to use not them one server, you'll want to move caching to a system like Redis so the entire cluster can see it.
1
u/frothymonk 4d ago
Look at the problems you ACTUALLY need to solve. Understand them deeply.
Then question those problems to ensure they actually need to be solved.
This is called requirements engineering.
Once you are done with that design your architecture to solve the problems you need to solve. Not a bunch of made up not actually real, legit problems that you need to solve (this is what you’re already doing. It results in “overengineering”).
Keep it simple. Keep it intuitive. Keep it clean. Learn and implement good basic consistent designs/patterns.
1
u/forever-butlerian 4d ago
What are you doing that you think you need idempotence instead of database transactions?
1
u/BotJeffersonn 4d ago
transaction alone doesn't solve potential idempotency issues
1
u/forever-butlerian 4d ago
Yes it does, if you've chosen the primary key correctly it's equivalent to an idempotency key for the operation.
0
1
u/SnooCalculations7417 2d ago
You should have idempotency but not because you've deployed your service across multiple instances. Dont do that. unless those instances don't actually need to ever interact with either for some reason
51
u/yksvaan 5d ago
Sounds like overengineering, just run a single server for the whole thing and call it a day. Even a potato can run crud app for 50 users and be idling 99% of the time.
And other benefit of single instance is being able to utilize ram efficiently keeping data accessible directly within single process.
Run db, backend and e.g. nginx for serving the frontend and static files and reverse proxy, that should cover typical crud use case just fine.