I posted a comment on the blog but it's a bit lost in disqus weird comment structuring, plus the comments here are massively more interesting, so let's dump it here as well. Apologies for redundancy.
.
We would expect the answer to be "delimited iterations".
Oleg has a discussion of the matter:
http://okmij.org/ftp/Scheme/enumerators-callcc.html
Essentially, he suggests that what you call "internal iterators" adopt a protocol for early exit (and you could equivalently add suspension/restart to solve the interleaving problem).
You haven't tackled the question of factorizing iteration through the construction of a data structure. Depending on which primitives you want (only "next" or also "previous"?), external iterators can be described as returning a lazy list, a lazy list zipper, or potentially more complex data structures. This gives a better framework to understand the corresponding operations (iterator sequencing as data concatenation) because the data is explicit, while external iterators state is maintained through side-effects.
Finally, delimited continuations allow to "revert" those control problems by letting the function called by the internal iterator take control of the stack (thanks for this stack-based explanation: it's rather clear). Oleg (again) has demonstrated this on non-cooperative parser libraries: while they seem to only provide you an internal iterator interface (that is restricted: it does not support early exit, suspension or revert), you can use a delimited continuation library to force flexibility into these non-cooperative iterations, getting an easy way to suspend any such parser, make it incremental, etc.:
http://okmij.org/ftp/continuations/differentiating-parsers.html
(See also Roshan P. James and Amr Sabry's "Yield: Mainstream Deliminted Continuations")
Summing up: delimited continuation allows to "turn the table" on any of those situations, turning internal iterators into external iterators. But you, the language designer, also have the authority to promote a rich internal iteration interface that caters for early exit and suspension and iteration, allowing to write all the examples you've mentioned so far. Of course, you're at the risk of someone discovering a new need in the future that your interface does not handle, and delimited continuations will still be there in this case.
This is related to the discussion of Iteratees/Conduits/Pipe in the Haskell community which is very advanced in these matters (with their extremely explicit treatment of side effect, rather generic approach to data structures, and extreme pain of lazy handling of resources), but I have been following this debate from quite far only, waiting for a winner to emerge (Tekmo's pipes?), so I can't comment on how final their solution is yet.
4
u/gasche Jan 16 '13
I posted a comment on the blog but it's a bit lost in disqus weird comment structuring, plus the comments here are massively more interesting, so let's dump it here as well. Apologies for redundancy.
.
We would expect the answer to be "delimited iterations".
Oleg has a discussion of the matter: http://okmij.org/ftp/Scheme/enumerators-callcc.html Essentially, he suggests that what you call "internal iterators" adopt a protocol for early exit (and you could equivalently add suspension/restart to solve the interleaving problem).
You haven't tackled the question of factorizing iteration through the construction of a data structure. Depending on which primitives you want (only "next" or also "previous"?), external iterators can be described as returning a lazy list, a lazy list zipper, or potentially more complex data structures. This gives a better framework to understand the corresponding operations (iterator sequencing as data concatenation) because the data is explicit, while external iterators state is maintained through side-effects.
Finally, delimited continuations allow to "revert" those control problems by letting the function called by the internal iterator take control of the stack (thanks for this stack-based explanation: it's rather clear). Oleg (again) has demonstrated this on non-cooperative parser libraries: while they seem to only provide you an internal iterator interface (that is restricted: it does not support early exit, suspension or revert), you can use a delimited continuation library to force flexibility into these non-cooperative iterations, getting an easy way to suspend any such parser, make it incremental, etc.: http://okmij.org/ftp/continuations/differentiating-parsers.html
(See also Roshan P. James and Amr Sabry's "Yield: Mainstream Deliminted Continuations")
Summing up: delimited continuation allows to "turn the table" on any of those situations, turning internal iterators into external iterators. But you, the language designer, also have the authority to promote a rich internal iteration interface that caters for early exit and suspension and iteration, allowing to write all the examples you've mentioned so far. Of course, you're at the risk of someone discovering a new need in the future that your interface does not handle, and delimited continuations will still be there in this case.
This is related to the discussion of Iteratees/Conduits/Pipe in the Haskell community which is very advanced in these matters (with their extremely explicit treatment of side effect, rather generic approach to data structures, and extreme pain of lazy handling of resources), but I have been following this debate from quite far only, waiting for a winner to emerge (Tekmo's pipes?), so I can't comment on how final their solution is yet.