[swift-evolution] [Proposal] Change guarantee for GeneratorType.next() to always return nil past end

[Kevin Ballard]

On Tue, Mar 8, 2016, at 04:41 PM, Dmitri Gribenko wrote:
> On Tue, Mar 8, 2016 at 12:16 PM, Kevin Ballard <kevin at sb.org> wrote:
> > On Tue, Mar 8, 2016, at 11:42 AM, Dmitri Gribenko wrote:
> >> On Sun, Mar 6, 2016 at 5:58 PM, Kevin Ballard via swift-evolution
> >> <swift-evolution at swift.org> wrote:
> >> > That's a fair point. But I think the important sentence from my comparison to Rust is "And in practice, almost nobody ever has to actually use .fuse(), …".
> >>
> >> The concern is that people who do need to invoke .fuse(), won't,
> >> because all generators they are likely to try in practice will 'just
> >> work' and return a continuous stream of nils.
> >>
> >> I think what this really comes down to is the trade off between a
> >> subtle correctness issue and a small performance win for a small
> >> number of data types (which can be even non-existent as Patrick
> >> Pijnappel shows).  Given the general direction of Swift, I'm inclined
> >> to choose correctness here.
> >
> > Anyone who uses Generators directly has to understand their behavior and has to understand their requirements. The prohibition on not copying a Generator and invoking next() on both copies I think is actually more of a potential problem than handling post-nil behavior.
> 
> And that is another problem that we need to fix.  I don't think it is
> acceptable to say that we won't fix one problem because there is
> another anyway.
> 
> > If we define post-nil behavior as needing to return nil forever, I suspect it's actually more likely for implementors of new Generators to screw up and not enforce this requirement than it is for clients to even notice it.
> 
> I disagree.  Most people just use the default IndexingGenerator.

And most people never even try to invoke next() on a Generator after it's returned nil.

I have yet to see anyone actually offer any evidence to show that this is a real problem people are hitting.

> > As a side note, I just tested (using Xcode 7.3 beta 5) and AnyGenerator doesn't currently track this state itself. You can easily pass a closure to AnyGenerator() that causes it to return non-nil values after a previous call to next() has returned nil.
> 
> AnyGenerator is used to abstract away implementation details, but it
> can't fix broken implementations.  Any time you conform to a protocol,
> you are making a statement about semantics.  If the semantics are not
> implemented correctly, then there are no guarantees about correctness
> for things that build on top.

The point was that it's pretty easy to accidentally violate the post-nil requirement using AnyGenerator. People writing new GeneratorType implementations should hopefully pay particular attention to the requirements, but people creating ad-hoc generators with AnyGenerator are more likely to not realize that what they've written can violate the post-nil requirement. And in fact since most users of Generators don't even attempt to message a Generator after it's returned nil, they probably won't notice the problem, until such time as they happen to pass it to code that does make assumptions about the post-nil behavior of Generators.

Which is to say, if I'm writing code that accepts arbitrary Generators and wants to be resilient, and if I want to be able to message a generator after it returns nil (and get nil again), I may end up using something like FuseGenerator anyway so that way my Generator works even when the input Generator was written sloppily.

Another way to put this is the prohibition against using copies of generators is something that clients are responsible for, but defining post-nil behavior as always returning nil is something that the GeneratorType implementation itself must handle. In fact, it's the only requirement not enforced by the type system that a GeneratorType implementation must conform to. And since most clients don't even invoke post-nil behavior, it's likely that people will violate this requirement without realizing it.

-Kevin Ballard