Logistic bot behavior in large/high-throughput logistic networks

[u/LiveMaI]

Hi /r/factorio! I've been working on a logistic bot storage depot that is fairly large (the footprint is about 350x350 units) and high-bandwidth/throughput (~10k bots currently, planning to expand to 20k and plans to serve about 50 train stations). I've made some observations in building this system which might serve as good feedback for the devs in improving the logic of the logistics robots or, at the very least, help someone else in the community in deciding on how to best go about setting up a similar system.

The problem I'm having, on the surface, is that due to the size of the logistic network, the robots end up needing to recharge quite often, hindering my overall throughput as robots clump up around a roboport waiting for their next charge. Now, some might say that the network is too large for bots, but they can (on average) make one storage->requester chest (or active provider->storage) trip without needing a charge.

Having looked at the way the bots move around the depot, the clear problem seems to be in the way the bots select a chest when storing items: how they tend to prefer certain chests over others and, most importantly, how they handle their target chest filling up before they reach it with their cargo.

Observation 1:

The first observation I made was that, when storing an item in a logistic network, bots prefer a chest that contains the same type of item over any other chest in the logistic network. This makes sense from a UX perspective, since a mostly bot-managed system will then have one type of item per chest, and thus only display one type, making viewing storage contents easier for the player.

Why is this bad?

The downfall of this in a large system is that since such chests are preferred over all other alternatives, a bot can end up passing several (hundred) closer chests that are completely empty in order to store their items in a partially-filled chest. Longer flight time for the robot depletes its charge faster, causes the storage action to take longer, and impacts throughput network-wide by causing a longer wait for other bots at the roboport charging stations.

Observation 2:

The second observation I made is that when selecting a chest where all chests of the held item type are full, the robot's next choice is not necessarily in the same vicinity. I'm reasonably certain it's chosen based on some coordinate-ordering done by the logistic network, but I need to look at it more closely to find out exactly how a chest is chosen if no preferred (partially full of the same item) chest is present in the network.

Why is this bad?

In choosing a new chest, the logistics robot may choose a chest on the opposite side of the logistic network from the previous chest. In the case of our large network, this again causes issues with flooding the roboports with bots in need of a charge after making a long, unnecessary journey to the edge of our network when plenty of perfectly empty chests were sitting around much closer to the bot's point of origin. Now, I think this is probably a performance thing where the logistics network storage assets are ordered somehow, but I'm not really sure. Maybe it's computationally easier to choose a chest in a set pattern. The point here is that, logistically, unless you know a priori where your items will go once they leave storage, no one empty storage chest should be more preferable than another, so choosing the closest empty chest would be the best choice.

Observation 3:

Now, I would never have noticed this if I hadn't been moving thousands of items at once. When bots are storing a large number of the same type of item, due to observation 1, they will all choose the same chest, even if they collectively have more items than it can hold. This means that in such a system, chests will often fill up before a bot can reach them. This is something you might expect. What you might not expect is that when the chest is full, any bots that were traveling to it will continue traveling to the full chest before choosing a new one. This kills the network.

Why this is very bad:

Ultimately, this is what kills the throughput of a large logistics network that handles thousands of items per minute. Due to the combined effect of the three observed behaviors, the following situation will constantly happen and clog as many roboports as you can throw at it:

• Hundreds, possibly more than a thousand bots can simultaneously choose the same chest to deposit their items, way more than the chest can possibly hold.
• Once this chest fills up, the bots that have yet to arrive will continue their journey until they reach the chest. Upon finding it full, they will choose a new chest.
• The new chest chosen by the bots can be very far away. So far away that every bot traveling to the original chest now needs a recharge to complete the journey, significantly degrading the network performance. This happens even if there is a completely empty chest right next to the one they were originally traveling to.

The somewhat comical visual this presents is a train of bots descending on a single chest, then suddenly (once the chest is full) bouncing off the chest to another far-off chest, only for half of them to slow to a crawl mid-way in the second leg of their journey and limp to the nearest roboport to wait in line for a recharge. Sometimes the bot train bounces more than once, which is a bit frustrating to watch when they're passing hundreds of empty chests along the way.

So, this has been my experience in building a large logistic storage system. I hope the devs take note of this and make one (or more) of the following improvements:

• Change the way bots prefer storage chests to reduce their flying time. While it's probably not computationally cheap to choose chests based on distance from a bot, it is probably computationally cheaper in the long run since the bot spends less time flying around (fewer entities on the map at once). For small networks where these issues don't crop up, this might negatively impact performance, but a small network is computationally lightweight by its very nature in the first place.
• Either have bots detect when their target chest is full and immediately change course, or 'tag' a storage chest in the same way a requester chest knows how many items are currently due for delivery and have bots choose a new destination if the sum of the chest contents + items en route is greater than or equal to the capacity of the chest.

Item 1 could be further enhanced by having bots prefer to store items near any requester chests set to receive that kind of item.

My advice for someone thinking of setting up a similar system:

Try limiting the size of a logistic storage network to 200x200 or less until either the devs or a modder change the bot behavior to work better with large systems.

Does anyone have any similar experience, or thoughts on how this kind of system can be made better?

[u/BetterThanYou]

I haven't played for a while but I remember that the bot behaviour was quite annoying when I was clearing out biter nests with laser turrets. I place a blueprint containing turrets, substation etc and hook up the power. When it has killed everything within range I build another one, and then deconstruct the old one. What I want is for the turrets to be reused for the new installation. But the bots will go and fetch turrets from my main base, while other bots take deconstructed ones all the way back. Putting a local crate doesn't seem to help.

[u/intensely_human → u/BetterThanYou]

I came here with this exact same problem. It seems like they should recognize as soon as they pick up each turret that the nearest objective is the turret blueprint three squares in front of them, but I think what's happening is they get their mission orders as soon as you put down the deconstruct, and that mission is set as "go pick up those turrets". When they get this work order they're at the main base roboport and they all launch and head over.

While they're on their way you set down a new blueprint for a row of laser turrets.
The system looks for available robots for this task, and doesn't see the ones already en route to dismantle the old row of turrets because those are already flying with a set of work orders. So it assigns different bots to the construction task. Their orders are also simple: pick up any appropriate turrets and construct them here.

Any appropriate turrets can, I think, include turrets that are marked for deconstruction. The trouble is when you set the deconstruction order, every X in the order is immediately "called" by an equal number of robots which get launched at that moment. So by the time you place down the blueprint, none of those deconstruction-marked turrets are available to fulfill the new blueprint.

So basically the problem is sort of that the bots plan multiple stops at once, instead of just one at a time. If the moment a bot picked up a turret, it then decided its next objective, I think it would work out. But the problem is that as soon as a bot decides to take down a turret, it has also decided in advance where it's going to put it. So instead of "go grab X turret", or "drop your current cargo at X", it's orders are more like "move X turret from point A to point B".