[swift-evolution] [Proposal] Change guarantee for GeneratorType.next() to always return nil past end

[Kevin Ballard]

On Tue, Mar 8, 2016, at 11:42 AM, Dmitri Gribenko wrote:
> On Sun, Mar 6, 2016 at 5:58 PM, Kevin Ballard via swift-evolution
> <swift-evolution at swift.org> wrote:
> > That's a fair point. But I think the important sentence from my comparison to Rust is "And in practice, almost nobody ever has to actually use .fuse(), …".
> 
> The concern is that people who do need to invoke .fuse(), won't,
> because all generators they are likely to try in practice will 'just
> work' and return a continuous stream of nils.
> 
> I think what this really comes down to is the trade off between a
> subtle correctness issue and a small performance win for a small
> number of data types (which can be even non-existent as Patrick
> Pijnappel shows).  Given the general direction of Swift, I'm inclined
> to choose correctness here.

Anyone who uses Generators directly has to understand their behavior and has to understand their requirements. The prohibition on not copying a Generator and invoking next() on both copies I think is actually more of a potential problem than handling post-nil behavior. In my personal experience, I've accidentally violated the copy prohibition before, but I've never even had to think about post-nil behavior except when implementing my own Generator. And allowing post-nil behavior to be implementation-defined is potentially useful behavior. In addition, even if the optimizer can get rid of the cost of tracking this state (and even with the limitation removed I don't think you can assume that will necessarily be the case, especially once you end up with an AnyGenerator wrapper in the mix), it's still extra burden on people implementing new Generators for something that very few clients are even going to care about. If we define post-nil behavior as needing to return nil forever, I suspect it's actually more likely for implementors of new Generators to screw up and not enforce this requirement than it is for clients to even notice it.

As a side note, I just tested (using Xcode 7.3 beta 5) and AnyGenerator doesn't currently track this state itself. You can easily pass a closure to AnyGenerator() that causes it to return non-nil values after a previous call to next() has returned nil. Trivial example:

var base = someSeq.generate()
return AnyGenerator {
    guard let next = base.next() else { return nil }
    return next % 3 == 0 ? nil : next * 2
}

With this example, if `someSeq` is `1..<10`, the resulting elements from the generator are [2, 4, nil, 8, 10, nil, 14 , 16, nil, nil, ...]. And of course the whole point of AnyGenerator is to hide the implementation details from the caller, so if state is added to AnyGenerator to track this, it definitely won't get optimized away. And if the AnyGenerator is used to wrap an underlying generator, this state tracking is very likely to end up being redundantly performed in both AnyGenerator and the underlying generator.

-Kevin Ballard