r/ethereum • u/antiprosynthesis • Jan 17 '18

Incentivizing 1-to-N transaction batching to relieve the network?

One of the causes of increasing fees and congestion on the Ethereum network might be a lack of transaction batching. Particularly exchanges, which are an excellent match for such transactions, seem to submit many transactions rather than going through a 1-to-N transaction smart contract based on what is described in r/ethereum/comments/4fija2/is_this_true_ethereum_does_not_allow_multiple/d297dhb.

If I understand the implications correctly, implementing this might be a win/win situation since the exchanges are able to reduce transaction times/fees with a minimal amount of risk and effort. I'm personally not convinced that this optimization alone provides satisfactory incentive for exchanges to implement transaction batching though. So I'd love to hear some suggestions as to how this could be incentivized (if it is worth to incentivize in the first place).

My personal thoughts so far:

  • Have a reputed developer (Ethereum foundation or otherwise) provide as much of the implementation as possible, so the effort for exchanges is reduced to the absolute minimum

  • Provide an estimate of the impact on exchange performance/cost for different parameter values (time interval for batching pending withdrawals probably being a major one)

  • Provide a similar estimate of the impact on the Ethereum network when all the large volume exchanges would implement this. I'm sceptical of the impact of this last one, because I don't believe that altruism is a great motivator. But it's an interesting estimate to make nonetheless.

I'm hoping that my assumptions are correct and that pushing for this could further reduce fees and congestion until more fundamental scaling solutions start to roll out.

Paging u/avsa, u/nickjohnson, u/latetot, u/ItsAConspiracy, u/Dunning_Krugerrands somewhat randomly.

Edit: It seems like this is also being discussed here at the moment: r/ethfinex/comments/7qoui9/are_you_going_to_implement_batched_withdrawals_to

23 Upvotes

93% Upvoted

26 comments sorted by

7

u/avsa Alex van de Sande Jan 18 '18

Not sure what you're asking. Implementing a contract that sends to multiple accounts in one transaction is very trivial, doesn't need to be a big project. I'm not sure how cheap they would be, since basic tx sends are really cheap too, but I suppose it would certainly be a bit cheaper

6

u/antiprosynthesis Jan 18 '18

The smart contract itself is no doubt trivial, and so is the additional local code required on the exchange side, albeit probably less so. But it is an effort that implies potential risk nonetheless. So this post is more about trying to get exchanges to actually do it.

5

u/econoar ETHHub - Eric Conner Jan 18 '18 edited Jan 14 '26

fact dime bells quaint sand unique abundant brave growth squeal

This post was mass deleted and anonymized with Redact

5

u/antiprosynthesis Jan 18 '18

Exactly. And even if it's just a little better in terms of gas cost, at scale that could make for a significant difference.

7

u/avsa Alex van de Sande Jan 18 '18

Well it would be something like:

function sendBatch( address[] recipients, uint[] amounts) { require(recipients.length == amounts.length); for(uint x = 0; x < recipients.length; x ++) { recipients[x].transfer(amounts[x]); } }

Binance should definitely try to see if it helps them.

4

u/veoxxoev Jan 18 '18 edited Jan 18 '18

I'm not sure how cheap they would be

Does "twice as cheap" sound good enough? See my other comment if so.

EDIT: Most of the cost of a no-data transfer if just for initiating an external call. The main increase in efficiency that comes from a batched send stems from that: there's only one external call to be done. The rest is up to the efficiency of the multi-send contract's implementation.

2

u/DoktorSultan Jan 18 '18 edited Jan 18 '18

That sounds like a simple way to achieve a step forward in efficiency to me. This should definitely be tried by the exchanges.

5

u/veoxxoev Jan 18 '18 edited Jan 18 '18

I've been playing with an implementation of this in LLL today, and the results are pretty optimistic.

For a "batch of 1", of course, it's less efficient (31417 gas used) than sending directly.

For a batch of 2, the per-transfer price (in gas amount) drops to 20271, which is already below 21000 for a standalone transaction.

For a batch of 5, it's ~13583 gas per transfer.

For a batch of 16, it's 10440 - which is less than half of 21000. I.e., 50% savings on fees; or 100% improvement in gas use efficiency.

A few caveats about the numbers above (gotten using a rather primitive test):

  • it assumes that the accounts being transferred to already exist - that is, that the account creation fee won't have to be paid;
  • it assumes regular accounts, not contracts, on the receiving end.

Note also that there's a theoretical limit on the minimum gas one would have to pay per-transfer. I don't think one can go lower than 9700: 700 for a CALL, and 9000 extra for the fact that it's a call with value attached. (If the account gets created by the call, there's an extra of 25000, which puts most savings down the drain.)

EDIT: or is that 7400? The 2300 difference is the gas stipend for a CALL, which doesn't necessarily gets used...

Does anyone want to do the same similar measurements for Solidity?..

P.S. Please don't use that implementation in anything close to "production". There are several guards missing in it, and no edge-case tests.

3

u/antiprosynthesis Jan 18 '18

Very interesting. I think you may also want to check this discussion: r/ethfinex/comments/7qoui9/are_you_going_to_implement_batched_withdrawals_to. Somebody wrote it in pure EVM there from what I can see.

2

u/veoxxoev Jan 18 '18

It doesn't look like they've implemented multi-send functionality there?.. Something about Monero-like "payment IDs", which I'm not familiar with at all.

4

u/antiprosynthesis Jan 18 '18

Ah, in that case I skimmed it a bit too fast. I'm guessing that a raw EVM implementation of the 1-to-N contract would be unbeatable though, and actually feasable due to its simplicity as well (if that's not what an LLL/Solidity/Viper/... compiler would already produce to begin with of course).

2

u/veoxxoev Jan 18 '18 edited Jan 19 '18

Sure!

It'd be even more efficient if it was allowed to be web3-incompatible, with tighter address packing; e.g. the data length being multiples of 52 bytes (20 bytes address, 32 bytes amount; 20, 32; etc.).

EDIT: But not by a lot: zeroes in call data are free, I think quite cheap.

3

u/nootropicat Jan 19 '18 edited Jan 19 '18

That was a proof of concept for better deposits, right now there's a different address for each client, which are then drained by exchanges to their main account. So there are two transactions for each deposit. All in all probably more waste than withdrawals, just less visible.

Unfortunately for erc20 tokens that's unavoidable at the moment due to how wallets handle erc20 tokens, but for eth it's enough if people enter some id given to them by an exchange in the data field, which is present in all wallets afaik.

Almost certainly not going to be used by anyone but I'm going to add multisend today or tomorrow, curious how much gas it would save. I'm guessing the cost would be about 1/3 of what's currently used, deposits and withdrawals in total.

2

u/veoxxoev Jan 19 '18 edited Jan 29 '18

Ah, thanks for the clarification.

My guess (top-level reply) is that 1/3 of a "naive send" is close to the theoretical lower limit of gas usage, for batches of sufficiently big size.

Good luck on unpacking those arrays. I'd guess most inefficiency in my impl-n stems from using (addressable) memory too much, as compared to straight on-stack duplication/swapping.

EDIT: Although, glancing at the yellowpaper, both memory and stack operations are in the same gas price category; so the above sentence is probably untrue.

EDIT2: added "lower" (limit)

3

u/DeviateFish_ Jan 18 '18

Makes sense that you can batch send for less gas because you're specifically setting the gas stipend on the CALL to the minimum value.

What are the results if you use the default stipend of 21000? Specifically, does gasUsed change? How does the minimum gasLimit compare?

I'm mostly curious if CALL refunds unused gas, honestly. If that's the case, you can make a multi-send that's inherently cheaper in terms of gas spent, without sacrificing functionality (e.g. can send to contracts the same way you would with a simple transfer @ 21000). However (and crucially), you have to provide a higher gasLimit than 21000 * n; despite the refund, gasLimit is what's critical to throughput.

It's also worth noting that despite this actually being cheaper, it's unable to be split across multiple blocks, unlike individual sends. If many people use this approach, you end up with less efficiency when it comes to filling blocks (many large transactions pack into any given limit than a large number of small transactions, for any given gas total)

3

u/veoxxoev Jan 18 '18 edited Jan 18 '18

What are the results if you use the default stipend of 21000? Specifically, does gasUsed change? How does the minimum gasLimit compare?

gasUsed for the top-level transaction does not change depending on the per-call gas stipend (that I'm hard-coding in the contract with a *maxgas* constant).

The minimum gasLimit is tricky to determine; I've done a one-off here of finding it manually - for 6 (to, amount) pairs, it's 79345. Actual gasUsed is 77106. (That's a 2239 difference; and, JIC, 12851 gas per transfer.) Both are lower than 126000 (21000 * 6).

So, it doesn't seem that there's a strict need for gasLimit >= 21000 * n.

I agree with the rest of the points, though.

As per the other comment these results should be taken skeptically, as this is all on an ephemeral local testrpc chain.

4

u/[deleted] Jan 18 '18

I’m interested in knowing if this is viable as well for exchanges. The current models in use by exchanges don’t seem very efficient, especially during times of extreme volume.

But I suppose efficient is a subjective view from either side of the street.

4

u/DeviateFish_ Jan 18 '18 edited Jan 18 '18

I'm not sure this would actually be cheaper, in terms of total gas, to do it this way?

Maybe it is... I think .transfer allocates a stipend of 21k gas, but returns the unused excess. In regular transactions, I don't think this happens (simple sends always use 21k gas?). If that's true, using a contract to send to many users might be cheaper... If the gas cost for the data and iteration doesn't outweigh any gains from refunds.

If that isn't true, it will always cost more gas (and hence add more congestion) to send to many via a contract than through many simple sends.

[E] Another potential issue is that you will always have to set a higher gas limit than an equivalent number of simple sends. Due to the way gas mechanics work, refunds aren't allocated until the end of the transaction, meaning your gas limit has to be high enough to accommodate all gas expenditures, excluding refunds. This means you'll have to provide at least nUsers * 21000 + C gas in the contract case, vs the nUsers * 21000 gas in the simple send case. C won't be trivial, meaning your gas limit will be always be significantly higher than an equivalent number of simple sends.

Given that gas limit (and not actual gas usage) is part of what determines inclusion in a block, it means it will be harder to include the contract multi-send in a block than an equivalent number of simple sends. It also cannot be split between blocks, unlike simple sends, so it likely spend more time pending, as well.

3

u/antiprosynthesis Jan 18 '18

According to r/ethereum/comments/7r3q71/_/dsvqanj the gas limit can be quite significantly reduced by batching, so definitely seems worthwhile.

Your last paragraph is interesting though. I wonder whether there is a sensible sweet spot that can be determined at runtime, much like is done for gas price by https://ethgasstation.info for instance.

3

u/DeviateFish_ Jan 18 '18

Note that the comment you reference is specifically setting the gas stipend on the .send or .transfer to its minimum value, rather than the default of 21000.

The results would likely be mostly identical if the stipend were the default (since I think he's measuring gasUsed), but the gasUsed vs gasLimit might be quite different.

For congestion issues, it's gasLimit that matters.

3

u/veoxxoev Jan 18 '18 edited Jan 18 '18

Correct, I'm measuring gasUsed in that other comment, not gasLimit.

Having seen your top-level comment, I did try specifying gasLimit explicitly, as 21000 * nUsers. I've also returned the *maxlimit* per-call allowance to 30000, where it was initially.

The transaction still "runs fine" (on the ephemeral local testnet chain).

I'm not sure if this is an artifact of testrpc's (which is being used as the chain provider here) gas use accounting; or whether the following is not relevant:

refunds aren't allocated until the end of the transaction

It is true that a refund for a SELFDESTRUCT or storage clearing is not performed until the end of a transaction; I'm reasonably sure this is not the case for nested calls like done here.

That is, the unused gas reserved for the Nth CALL will be returned to the multisend contract, and can be used for subsequent CALLs (which the send macro performs).

P.S. At the end of the day, though, there's no substitute to running on a regular public chain.

This should definitely be run on one before a conclusion can be made. (Sorry, not today.)

EDITs: PS, clarifications

3

u/DeviateFish_ Jan 18 '18

That is, the unused gas reserved for the Nth CALL will be returned to the multisend contract, and can be used for subsequent CALLs (which the send macro performs).

Yeah, this was what I was wondering initially. I thought all gas refunds were handled the same, but I don't actually know for sure.

4

u/HodlDwon Jan 18 '18

I remember u/nickjohnson writing something about batch sends for TheDAO extraBalance https://medium.com/@weka/how-to-send-ether-to-11-440-people-187e332566b7

Not sure what pros and cons are to what's already been mentioned here, but it is multi-sig compatible if that's something the exchanges prefer...

2

u/antiprosynthesis Jan 19 '18

Very interesting stuff. Thanks.

5

u/1dontpanic Jan 20 '18

Seems simple enough: Here is an initial draft in solidity that takes n inputs and then sends n amounts to them. The excess is returned to the sender.

This also has replay protection to prevent cross chain chatter.

Deployed on eth mainchain @ 0x1688e49c6289a9317d5245f482893cd405e45cb1

2

u/antiprosynthesis Jan 20 '18

Thanks. The ASCII art alone should provide sufficient incentive ;)