[swift-evolution] [Pitch] KeyPath based map, flatMap, filter

[Dave Abrahams]

on Thu Jul 06 2017, Brent Royal-Gordon <brent-AT-architechies.com> wrote:

>> On Jul 6, 2017, at 9:13 AM, Dave Abrahams <dabrahams at apple.com> wrote:
>> 
>> I'm not sure what you're objecting to about this.  Is it the very
>> appearance of curly braces?
>
> I went to bed thinking that maybe I should have explained that better,
> and I guess I was right. ;^) Here's why I think we should do something
> here.
>
> From what I can tell, mapping or filtering on a single property is
> exceptionally common. I ran a few regexes on the Swift code present on
> my machine (basically the stuff in the Swift repos, plus my
> open-source projects, plus my closed-source stuff, plus various
> playgrounds and things) to see how common different kinds of `map`,
> `filter`, and `flatMap` closures were:
>
> 	2142	OP { …$0… }
> 	1835	OP(function) or OP(some.method)
> 	589		OP { $0.property } or OP { $0.some.property }
> 	564		OP { $0.property }
> 	525		OP { function(…$0…) } or OP { some.method(…$0…) }
> 	186		OP { $0.method(…) }
> 	153		OP { function($0) } or OP { some.method($0) }
> 	100		OP { $0 as SomeType } or OP { $0 as? SomeType } or OP { $0 as! SomeType }
> 	52		OP { $0.method() }
> 	35		OP { collection[…$0…] } or OP { some.collection[…$0…] }
> 	20		OP { collection[$0] } or OP { some.collection[$0] }
> 	13		OP { $0! }
>
> (Simple regex-based match of `map`, `flatMap`, and `filter`
> calls. Permits various spacing schemes and `try`. If you want to run
> it on a more representative sample, the script is here, requires
> fish(1):
> https://gist.github.com/brentdax/2a8ee2705c39e9948aafedbd81b1366f)
>
> So, at least in my unscientific sample, more than a quarter of
> map/filter/flatMap calls which use `$0` simply look up a property or
> chain of properties on it. 

Totally believable.

> If we want to make something about `map`
> and friends more convenient, this seems like a good place to look.
>
> (Using a straight function is a few times more common than a property,
> but given that this is also the syntax used to abstract over a closure
> body, I'm not sure how much weight to put on that fact.)
>
> So what's wrong with what we have now? Syntactic weight. Consider this expression:
>
> 	person.map { $0.name }
>
> The "loudest" parts of this expression are the closure brackets and
> the `$0`, but they are actually the *least* important. 

That's funny, my eye skips right over them and focuses on “.name” Not
kidding at all.  And “name” is a single fairly short, un-chained
identifier.

> They do not express anything about what this line of code *does*; they
> exist solely to tell the compiler how to do it. They are pure glue
> code, and serve only to obscure the actual intent. Compare that to:
>
> 	person.map(\.name)
>
> Here, we still have a glue character (the `\`), but it's just one, and
> it's relatively inconspicuous compared to something like `$0`.

Meh.  This in particular doesn't look like a major improvement.

> That's not *too* bad, though. It gets a lot worse when the key path is
> actually in a variable:
>
> 	array.map { $0[keyPath: prop] }

> Again, look at how much of this line is given over to adapting a line
> of code to the compiler—and how little that actually matters when
> understanding what the line does. The most important thing in that
> expression is `prop`, but it's completely lost in this sea of
> irrelevant syntax. Compare to:
>
> 	array.map(prop)

Yes, that's a lot of extra syntax.  But again, you've used an
abbreviated, single identifier for the property and a
short, non-descriptive identifier for the array.  Let's make this a
fair/realistic comparison:

        gradeHistories.map { $0[keyPath: \.average] }

vs.
        gradeHistories.map(\.average)


Yep, I agree that passing a keypath directly is still much nicer in this
case.

> Which puts that piece of information front and center.
>
> If there was an argument that the case was too complex to handle
> nicely, I think we could justify leaving it alone. 

It's not the “too complex” criterion I'd want to talk about—it's the
“how often do you need it” criterion.  If it's rare, it doesn't matter
so much (I don't have an opinion about whether it is in fact rare).

> That's essentially what happened with the much-requested placeholder
> syntax:

Sorry, I may have missed that discussion.

> Lots of people wanted it, but critics pointed out the fundamental
> ambiguity of the syntax, and after spending gallons of electrons
> arguing about it, the proponents pretty much backed off. But key paths
> don't have that problem—they always work the same way and are
> completely unambiguous, so there's no scope for misinterpretation. And
> the cases key paths can handle appear to be about as common as the
> cases placeholder syntax was intended for.
>
>
> Nor is there a strong argument that the suggested behavior is
> fundamentally "weird". It's pretty natural to think of a
> `KeyPath<Root, Value>` as being a `(Root) -> Value` function, and the
> `foo.map(\.bar)` syntax reads pretty straightforwardly as long as you
> know what `map` does.

It's not fundamentally weird.  I'm just not sure it's important.  If it
actually is important, as I said, I feel very strongly that it shouldn't
require anyone to create an overload of map, because that quickly leads
to overloading everything that takes a closure.

> There's one more reason I think we should do this. It is not about the
> technology; it is not even really about the ergonomics. It's more
> about language "marketing", for lack of a better term.
>
> I think we were all surprised by the SE-0110 backlash. But in
> hindsight, I think it's pretty easy to explain. During the Swift 3 and
> 4 cycles, we systematically stripped conveniences and sugar from
> higher-order functions. We have good reasons to do this; we want to
> pare things down, fix the foundations, and then build up new
> conveniences. Eat your vegetables now and you can have dessert later.
>
> But we're entering our second year of eating nothing but
> vegetables. It's very difficult to maintain a strict diet forever,
> especially when—like the vast majority of Swift's users who don't
> participate in evolution or implementation—you don't really see the
> point of it. It's hard to blame them for being tired of it, or for
> complaining when yet another tasty food is pulled off the menu.
>
> Offering a treat like this on occasion will help ease the pain of
> losing the things we *need* to take away. And this is a particularly
> good candidate because, although it's a convenience for higher-order
> functions—which is where the pain is felt—it has little to do with
> parameter handling, the area where we actually need to remove things
> and refactor. It's like a dessert of ultra-dark chocolate—it's a treat
> that doesn't set the actual goal back very far.
>
> In the abstract, "fundamentals now, sugar later" is the right
> approach. But it can't be considered "right" if the users won't accept
> it. So let's look for opportunities to add conveniences where we
> can. Maybe this isn't the right feature—subtyping is always a bit
> perilous—but we should be on the lookout for features like this one,
> places where we can improve things for our functional programming fans
> without obstructing our own efforts to clean up parameter handling.

These are all good arguments.  For me it's a question of priorities and
long-term, irrevocable impacts.

By the way, if you're worried about whether subtyping will fly, I've
recently been thinking there might be a role for a “promotion” operator
that enables lossless “almost-implicit” conversions, e.g.:

    someNumber^      is equivalent to    numericCast(someNumber)
    \.someKeyPath^   is equivalent to    { $0\.someKeyPath }
    someSubstring^   is equivalent to    String(someSubstring)

    etc.

This convenience can be implemented by anyone today for keypaths, and
will provide nearly the syntax you're looking for.  This is exactly the
sort of thing I'd love to see become a widespread existing practice
before we incorporate it in the standard library, so we could properly
judge its impact on real code.

So, there are lots of options worth exploring before we jump on map and
flatmap and start adding one-off convenience overloads for keypaths.
    
-- 
-Dave