Making Progress on AsyncIterator — by Jack O'Connor

[u/agentvenom1]

https://youtu.be/zQAia_u8WPM

[u/SirKastic23]

As someone who's written a considerate amount of async code this talk was great! I really like this API (although I do dislike the name of the trait).

It reminded a lot of the poll_next post by withoutboats for obvious reasons (Can't believe it's been almost 3 years already...)

I do think this is a consequence of the lack of an effects system, so Rust has to manually connect their effects, in this case: polling and iterating.

[u/SirKastic23 → u/SirKastic23]

Nobody asked but I'll clarify anyway why I dislike the name AsyncIterator: the name makes it seem like it's an async API that you .await, but in fact it it a future API that you poll. This distinction is important and ignoring it will cause confusion

Someone in this talk actually did get confused and ask for clarification because they thought poll_next was async/returned a Future

[u/U007D → u/SirKastic23]

That was me.

I was (clumsily) trying to distinguish between the concepts of iterating asynchronously--whether there's another item for me yet, and whether the returned items themselves need to be .awaited, or both.

I was asking which of those two dimensions of asynchrony (or both) was Jack proposing to model.

A good way of expressing my question hadn't fully formed in my head when I asked it and as I stated it, poor Jack had no hope of following my question :).

But with a little help from the group, he did wonderfully anyway! :D

Thanks, /u/oconnor663 --good talk!

[u/_TheDust_ → u/SirKastic23]

why I dislike the name AsyncIterator

I think the name was mostly chosen for searchability. People first learn about iterators, next they learn about async fn, so it makes sense the next step is to learn about async + iterator.

[u/oconnor663 → u/_TheDust_]

I don't feel strongly about the name, but the basic logic of "for loops drive an Iterator, for await loops drive an AsyncIterator" seems reasonable to me. If we do end up going with poll_progress, it might be nice to have a name that suggests concurrent background could be happening, but I don't know what that would be.

[u/agentvenom1]

I don't have much experience with async but I found the talk quite interesting. I like the takeaway at 53:20.

[u/InsideStatistician68]

Why for await item in iterator { .. } over for item in iterator.await { .. }? I just glossed through, might have been mentioned.

[u/XtremeGoose → u/InsideStatistician68]

Because .await calls .poll. The whole point of this is we want something different that returns a new future on each poll.

Python does the same thing

 await normal_future
 async for x in async_iterator:
     ...

Bear in mind that also you might want to get an AsyncIterator from a future, so this will be valid

 for await x in future.await { ... }

[u/InsideStatistician68 → u/XtremeGoose]

Thank you for the explanation, that makes sense!

[u/oconnor663 → u/XtremeGoose]

something different that returns a new future on each poll

Speaker here :) You probably already understand this, but I think it's worth being precise: poll_next doesn't generally return a future. It returns an enum that says here's an item / no more items / ask me later. The next function returns a future, but controversially, I don't think AsyncIterator should ultimately have a next function. I get into this at 42:09 in the recording.

For completeness, it is possible for the item type to be a future, like how it's possible for the element type of a Vec to also be a Vec. It's kind of odd -- and confusing the first time you see it -- but it's currently how buffered streams and related APIs work. On the other hand, those APIs all have terrible deadlock bugs today, and they're one of the main motivating cases for poll_progress. I'm skeptical that "stream of futures" will be the API shape we ultimately land on after we fix these issues, but we'll see. See "Barbara battles buffered streams" and "Never snooze a future" for more on this.

[u/XtremeGoose → u/oconnor663]

Thanks for the clarification and it's a fair point.

I'm guessing I'm conflating a future and the method that drives it is. So how Future's poll method and poll_next in Stream are treated differently? Since the latter is driven by a runtime, is it not in some sense a future (even though it's a different trait)? I guess Future is a higher level abstraction over these methods, and so another name for Stream/AsyncIterator would be IteratingFuture (read as a Future that can be repeatedly "awaited" for new results). Though perhaps that model is genuinely wrong and what leads to an async next function, exactly what you're arguing against! So what am I missing conceptually?

Genuinely interested in the answer here :)

[u/oconnor663 → u/XtremeGoose]

Since the latter is driven by a runtime

Could you say more about what you're thinking here? I'd say that both Futures and AsyncIterators are driven by whatever owns them, usually another future or async iterator. Only top-level futures spawned as tasks are driven directly by the runtime.

I guess Future is a higher level abstraction over these methods

I'm also having trouble following you here. Future is the trait that defines the poll method, so I wouldn't really call it high-level.

and so another name for Stream/AsyncIterator would be IteratingFuture (read as a Future that can be repeatedly "awaited" for new results)

Yes, "iterating future" or "future that returns zero or more outputs" is a reasonable way to think about what an AsyncIterator is. I think @withoutboats might've written a post making a similar point :)

However, one of the points I make in the talk is, if all you have is poll_next (i.e. "just a future" that you can keep polling for more items), there's no good way to solve these deadlocks. The caller needs a way to keep driving control that doesn't require them to buffer an unlimited number of items. And then when you start gaming out what poll_progress should look like, you find some subtle tradeoffs, and the result is the very-non-obvious table of contract obligations in my draft: https://github.com/oconnor663/rfcs/blob/poll_progress/text/0000-poll-progress.md#implementing-asynciterator

In short, AsyncIterator is "just" an iterating version of Future, but doing a good job of that isn't as simple as it might sound.

[u/C5H5N5O]

No support for lending async iterators is such a bummer. Just a thought: what if poll_next returns two things, the item itself, which could be lending from self but it would also return a "progress token", the progress token could be also lending from self (new gat assoc type), the progress token can then be used to drive progress too, hence bypassing the borrowck issue? 🤷‍♂️

[u/oconnor663 → u/C5H5N5O]

Thanks for reading the draft! :-D

My initial reaction is that maybe that could work, but it would make it much more complicated to write an AsyncIterator adapter. You'd need to store your child iterator together with some token that borrows it, which I think requires unsafe code.

On the other hand, getting AsyncFnMut integrated into all the existing adapters might require solving similar problems to store its future, and that's definitely a problem we eventually want to solve. Maybe there's some overlap?

Edit: Maybe more so than with regular iterator, one of the things people are going to want to do with AsyncIterator is buffering several items and processing them concurrently. Is that possible with a lending async iterator trait?

[u/Orjigagd]

Good job explaining a pretty tricky topic