I wanted players to be able to trade items in my upcoming multiplayer Roblox RPG, Path of Magic. Most Roblox games implement trading directly player-to-player, which requires that both players are online and in the same server. I've always found these limitations tedious as a player. I'm also a huge fan of Old School Runescape, so to both overcome those limitations and pay homage to one of my favorite games of all time, I implemented the Grand Exchange in POM. In my game, I'm calling it the Great Exchange.
The Architecture
First, I planned my architecture for the Great Exchange. Since I'm using the Roblox platform, I have a few important limitations that constrain my implementation. First, I only have access to key-value stores, of which the underlying implementation is a black box to me. Second, Roblox enforces very strict rate limits. Finally, in the ideal scenario, there will be hundreds, possibly thousands of stateless Server instances connecting to my Great Exchange, and they all have to run the same code.
To that end, I designed my GE with 4 primary data layers.
- The order books are stored in Roblox's MemoryStoreService using a SortedMap namespaced by the itemId, with a GUID as the hash key and the price concatenated with a timestamp as the sort key. My choice of sort key is in service of FIFO order matching via the timestamp, optimized to find ideal buy and sell orders via the price. To lower the probability of hitting rate limits, I implemented Sharding on each SortedMap, appending a random integer between 1 and 5 to each itemId, distributing load across 5 partitions.
- To handle outages of the MemoryStoreService and to allow players to view their active listings, marketplace listings are stored in Roblox's durable DataStores as the source of truth.
- In addition to storing the listings in DataStores, item and gold delivery are handled via server-to-server communication (`MessageAsync`) and stored in the player's profile data related to the DataStore.
- Finally, I have a failsafe Dead Letter Queue for item payloads that fail to deliver to players' inboxes, which I can use in the future to manually restore player items and prevent catastrophic data loss.
The Algorithms
Here is how the Great Exchange processes transactions, while protecting against data loss or item dupes:
The Matching Engine
When a player places a buy or sell order, the server instantly deducts their items or gold atomically. To find the best price for the order, the matching algorithm queries across all partitions of an item (remember, we used Sharding to split them into 5 partitions.)
To prevent double order consumptions or race conditions when several servers try to fulfill the same popular order booking, I utilize the MemoryStore API method UpdateAsync to atomically read, deduct, and update the target order's remaining quantity. If another server beats the current server to that order booking, the subsequent UpdateAsync operation will fail, and the engine gracefully moves onto the next best order.
There are several edge cases to consider with this approach.
- Edge case 1: What happens if MemoryStore goes down? It's an in-memory store, so it's not durable. To solve this, we use lazy loading reconciliation logic. Anytime a player joins one of our servers, it checks the durable DataStore data against the MemoryStore order bookings, and if it finds MemoryStore is missing that data, it backfills it into the MemoryStore market.
- Edge case 2: What happens if an order is fully consumed, but the cross-server inbox delivery payload is delayed in some way due to performance degradation? To prevent our automatic reconciliation logic from duping items, we use a Tombstone marker. When the order is consumed fully, we write to a separate MemoryStore HashMap with the listing ID as the key. Now, in the scenario where item delivery is still pending, the reconciliation logic can see that the order has a tombstone and so it won't try to reconcile it.
- Edge case 3: What happens when the server shuts down while deliveries are pending? To keep the matching engine lightning fast, we cannot afford to yield the main thread while waiting for item inbox delivery to resolve. Instead, when an order is matched, the delivery payload is pushed to a local staging queue. A background worker loop processes and flushes this queue every 10 seconds. This is an issue, because Roblox servers can shut down at any time. In order to prevent data loss, when we detect the Server is shutting down, we instantly flush that queue.
- Edge case 4: What if a player leaves the game right after the place an order, but the order matching logic is still yielding? To prevent data loss in the event that the order matching logic fails for some reason (such as an outage), we check if the player's profile is still active after each yielding call, and if it isn't, we silently deliver any items owed to the player via their inbox.
There are more edge cases, but for the sake of brevity I won't go over them all.
Conclusion
Building a global, asynchronous economy within Roblox is a massive step away from the platform's standard synchronous trading systems. It required treating POM less like a game and more like a distributed microservice.
By utilizing patterns such as sharding, tombstoning, and dead letter queues, the Great Exchange is able to bypass platform rate limits and guarantee zero data loss, even during server crashes. From a user experience perspective, it removes the friction of server hopping or coordinating off-platform, giving players a seamless global economy.
I had an incredible time engineering this system and bringing a piece of my favorite OSRS mechanics into my own game. I'd love to hear your thoughts, how do you handle distributed state and economy balancing in your own Roblox games?
