[swift-evolution] [Proposal] Explicit Synthetic Behaviour

[Xiaodi Wu]

On Mon, Sep 11, 2017 at 06:03 Haravikk via swift-evolution <
swift-evolution at swift.org> wrote:

> On 9 Sep 2017, at 18:41, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>
>
> On Sat, Sep 9, 2017 at 06:41 Haravikk via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>> On 9 Sep 2017, at 09:33, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>
>>
>> On Sat, Sep 9, 2017 at 02:47 Haravikk via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>>>
>>> On 9 Sep 2017, at 02:02, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>>
>>> On Fri, Sep 8, 2017 at 4:00 PM, Itai Ferber via swift-evolution <
>>> swift-evolution at swift.org> wrote:
>>>
>>>>
>>>>
>>>> On Sep 8, 2017, at 12:46 AM, Haravikk via swift-evolution <
>>>> swift-evolution at swift.org> wrote:
>>>>
>>>>
>>>> On 7 Sep 2017, at 22:02, Itai Ferber <iferber at apple.com> wrote:
>>>>
>>>> protocol Fooable : Equatable { // Equatable is just a simple example
>>>>     var myFoo: Int { get }}
>>>> extension Fooable {
>>>>     static func ==(_ lhs: Self, _ rhs: Self) -> Bool {
>>>>         return lhs.myFoo == rhs.myFoo
>>>>     }}
>>>> struct X : Fooable {
>>>>     let myFoo: Int
>>>>     let myName: String
>>>>     // Whoops, forgot to give an implementation of ==}
>>>> print(X(myFoo: 42, myName: "Alice") == X(myFoo: 42, myName: "Bob")) // true
>>>>
>>>> This property is *necessary*, but not *sufficient* to provide a
>>>> correct implementation. A default implementation might be able to
>>>> *assume* something about the types that it defines, but it does not
>>>> necessarily know enough.
>>>>
>>>>
>>>> Sorry but that's a bit of a contrived example; in this case the
>>>> protocol should *not* implement the equality operator if more
>>>> information may be required to define equality. It should only be
>>>> implemented if the protocol is absolutely clear that .myFoo is the only
>>>> part of a Fooable that can or should be compared as equatable, e.g- if a
>>>> Fooable is a database record and .myFoo is a primary key, the data could
>>>> differ but it would still be a reference to the same record.
>>>>
>>>> To be clear, I'm not arguing that someone can't create a regular
>>>> default implementation that also makes flawed assumptions, but that
>>>> synthesised/reflective implementations *by their very nature have to*,
>>>> as they cannot under every circumstance guarantee correctness when using
>>>> parts of a concrete type that they know nothing about.
>>>>
>>>> You can’t argue this both ways:
>>>>
>>>>    - If you’re arguing this on principle, that in order for
>>>>    synthesized implementations to be correct, they *must* be able to — *under
>>>>    every circumstance* — guarantee correctness, then you have to apply
>>>>    the same reasoning to default protocol implementations. Given a default
>>>>    protocol implementation, it is possible to come up with a (no matter how
>>>>    contrived) case where the default implementation is wrong. Since you’re
>>>>    arguing this *on principle*, you cannot reject contrived examples.
>>>>    - If you are arguing this *in practice*, then you’re going to have
>>>>    to back up your argument with evidence that synthesized examples are more
>>>>    often wrong than default implementations. You can’t declare that
>>>>    synthesized implementations are *by nature* incorrect but allow
>>>>    default implementations to slide because *in practice*, many
>>>>    implementations are allowable. There’s a reason why synthesis passed code
>>>>    review and was accepted: in the majority of cases, synthesis was deemed to
>>>>    be beneficial, and would provide correct behavior. If you are willing to
>>>>    say that yes, sometimes default implementations are wrong but overall
>>>>    they’re correct, you’re going to have to provide hard evidence to back up
>>>>    the opposite case for synthesized implementations. You stated in a previous
>>>>    email that "A synthesised/reflective implementation however may
>>>>    return a result that is simply incorrect, because it is based on
>>>>    assumptions made by the protocol developer, with no input from the
>>>>    developer of the concrete type. In this case the developer must override it
>>>>    in to provide *correct* behaviour." — if you can back this up with
>>>>    evidence (say, taking a survey of a large number of model types and see if
>>>>    in the majority of cases synthesized implementation would be incorrect) to
>>>>    provide a compelling argument, then this is something that we should in
>>>>    that case reconsider.
>>>>
>>>>
>>> Well put, and I agree with this position 100%. However, to play devil's
>>> advocate here, let me summarize what I think Haravikk is saying:
>>>
>>> I think the "synthesized" part of this is a red herring, if I understand
>>> Haravikk's argument correctly. Instead, it is this:
>>>
>>> (1) In principle, it is possible to have a default implementation for a
>>> protocol requirement that produces the correct result--though not
>>> necessarily in the most performant way--for all possible conforming types,
>>> where by conforming we mean that the type respects both the syntactic
>>> requirements (enforced by the compiler) and the semantic requirements
>>> (which may not necessarily be enforceable by the compiler) of the protocol
>>> in question.
>>>
>>> (2) However, there exist *some* requirements that, by their very nature,
>>> cannot have default implementations which are guaranteed to produce the
>>> correct result for all conforming types. In Haravikk's view, no default
>>> implementations should be provided in these cases. (I don't necessarily
>>> subscribe to this view in absolute terms, but for the sake of argument
>>> let's grant this premise.)
>>>
>>> (3) Equatable, Hashable, and Codable requirements are, by their very
>>> nature, such requirements that cannot have default implementations
>>> guaranteed to be correct for all conforming types. Therefore, they should
>>> not have a default implementation. It just so happens that a default
>>> implementation cannot currently be written in Swift itself and must be
>>> synthesized, but Haravikk's point is that even if they could be written in
>>> native Swift through a hypothetical reflection facility, they should not
>>> be, just as many other protocol requirements currently could have default
>>> implementations written in Swift but should not have them because they
>>> cannot be guaranteed to produce the correct result.
>>>
>>> My response to this line of argumentation is as follows:
>>>
>>> For any open protocol (i.e., a protocol for which the universe of
>>> possible conforming types cannot be enumerated a priori by the protocol
>>> designer) worthy of being a protocol by the Swift standard ("what useful
>>> thing can you do with such a protocol that you could not without?"), any
>>> sufficiently interesting requirement (i.e., one for which user ergonomics
>>> would measurably benefit from a default implementation) either cannot have
>>> a universally guaranteed correct implementation or has an implementation
>>> which is also going to be the most performant one (which can therefore be a
>>> non-overridable protocol extension method rather than an overridable
>>> protocol requirement with a default implementation).
>>>
>>>
>>> You're close, but still missing key points:
>>>
>>>
>>>    1. I am not arguing that features like these should *not* be
>>>    provided, but that they should *not* be provided implicitly, and
>>>    that the developer should actually be allowed to request them. That is
>>>    exactly what this proposal is about, yet no matter what I say everyone
>>>    seems to be treating me like I'm against these features entirely; *I
>>>    am not*.
>>>
>>>
>> You are entirely against Equatable having a default implementation for
>> ==. This is unequivocally stated. Others favor such a default
>> implementation and feel that in the absence of a way to spell this in Swift
>> itself, it should be magic for the time being. For the purposes of this
>> argument it really is not pertinent that you are not also against something
>> else; you're asking us to discuss why you are against a particular thing
>> that others are for.
>>
>>
>> FFS, how much clearer can I make this? *I AM NOT AGAINST THE FEATURE.*
>>
>> What I am against is the way in which it is being provided implicitly
>> rather than explicitly, in particular as a retroactive change to existing
>> protocols in a way that introduces potential for bugs that are currently
>> impossible, but also in general.
>>
>
> You are against a default implementation for ==, i.e. an implementation
> that is provided for you if you conform a type to the protocol and do
> nothing else ("implicitly rather than explicitly"), and you are against the
> default implementation being on the existing protocol Equatable
> ("retroactive change"). So, to summarize, what you are against is precisely
> a default implementation for the == requirement on Equatable.
>
> This is the topic of discussion here; I am attempting to convince you that
> you should be for rather than against these things.
>
>
> I *am* for it, the difference is that I want to *ask* for it, not have it
> thrust upon me.
>
> Reflective implementations *necessarily* go too far, because they
>> literally know *nothing* about the concrete type with any certainty,
>> except for the properties that are defined in the protocol (which do not
>> require reflection or synthesis in the first place).
>>
>
> I am confused why you are trying to argue in general terms about the
> universe of all possible default implementations that use reflection. This
> is necessarily a more difficult argument to make, and if it is to be
> convincing for all default implementations it must also be convincing for
> the two specific protocol requirements we are talking about here. Start
> small:
>
> We have agreed, as a community, that there is a reasonable default
> implementation for Equatable.== when certain conditions are met (for value
> types only at the moment, I believe). Namely, given two values of a type
> that has only Equatable stored properties, those values are equal if their
> stored properties are all equal. The author of a new value type who wishes
> to make her type Equatable but chooses not to implement a custom == then
> benefits from this default when all stored properties are Equatable.
>
>
> See, this is another flawed assumption; you are assuming that omitting a
> custom implementation of == is always intentional rather than an oversight,
> which is not guaranteed. This is one of my gripes with the retroactive
> change to Equatable, as it is currently *impossible* to omit an
> implementation.
>

Again, this applies equally to the addition of _any_ default
implementation. And again, such changes don’t even require Swift Evolution
approval.


> And precisely what kind of "evidence" am I expected to give? This is a set
>> of features that *do not exist yet*, I am trying to argue in favour of
>> an explicit end-developer centric opt-in rather than an implicit protocol
>> designer centric one. Yet no-one seems interested in the merits of allowing
>> developers to choose what they want, rather than having implicit behaviours
>> appear potentially unexpectedly.
>>
>
> Both options were examined for Codable and for Equatable/Hashable. The
> community and core team decided to prefer the current design. At this
> point, new insights that arise which could not be anticipated at the time
> of review could prompt revision. However, so far, you have presented
> arguments already considered during review.
>
>
> And so far all I have heard about this is how it was "decided"; no-one
> seems interested in showing how any of these concerns were addressed (if at
> all), so as far as I can tell they were not, or they were wilfully ignored.
>

They were addressed by being considered.

Therefore, your argument reduces to one about which default implementations
>> generally ought or ought not to be provided--that is, that they ought to be
>> provided only when their correctness can be guaranteed for all (rather than
>> almost all) possible conforming types. To which point I sketched a rebuttal
>> above.
>>
>>
>> If a protocol defines something, and creates a default implementation
>> based only upon those definitions then it must by its very nature be
>> correct. A concrete type may later decided to go further, but that is a
>> feature of the concrete type, not a failure of the protocol itself which
>> can function correctly within the context it created. You want to talk
>> evidence, yet there has been no example given that proves otherwise; thus
>> far only Itai has attempted to do so, but I have already pointed out the
>> flaws with that example.
>>
>> The simple fact is that a default implementation may either be flawed or
>> not within the context of the protocol itself; but a reflective or
>> synthetic implementation by its very nature goes beyond what the protocol
>> defines and so is automatically flawed because as it does not rely on the
>> end-developer to confirm correctness, not when provided implicitly at least.
>>
>
> Again, if it applies generally, it must apply specifically. What is
> "automatically flawed" about the very reasonable synthesized default
> implementation of ==?
>
>
> It makes the assumption that every equatable property of a type is
> necessarily relevant to its equality.
>

No necessarily, only provisionally and rebuttably. If it’s not the case,
override the default.

Consider for example if a type stores a collection index for performance
> reasons; this isn't an intrinsic part of the type, nor relevant to testing
> equality, yet this default implementation will treat it as such because it *knows
> nothing about the concrete type's properties*. If a protocol does not
> define a property then any action taken upon such a property is necessarily
> based upon an assumption; just because it might be fine some of the time,
> does not make it any less flawed.
>
> The big difference here between explicit and implicit synthetic
> implementations is where this assumption originates; if a method is
> synthesised implicitly then the assumption is made by the protocol designer
> alone, with no real involvement by the end developer. If I explicitly
> opt-in to that default however I am signalling to the protocol that it is
> okay to proceed. In the former case the assumption is unreasonable, in the
> latter it is explicitly authorised. It is a difference between "I want to
> make the decision on what's correct" and "I am happy for you (the protocol
> designer) to decide".
>
> Right now, when I conform to Equatable, it is a declaration of "I will
> implement this", but with this retroactive implicit change it is now a
> declaration of "implement this for me", these are two entirely different
> things. Consider; what if I'm working on a piece of code that requires
> types to be Equatable, but one of the types I'm using currently isn't, so I
> quickly throw Equatable conformance onto it and go back to what I was
> doing, with the intention of completing conformance later. With this change
> that type may now receive a default implementation that is wrong, and I've
> lost the safety net that currently exists.
>

Right now, it still wouldn’t compile, so I don’t see why you would do that.
In the future, if you want to make it not compile, there is nothing
stopping you from conforming to a non-existent “NotYetEquatable”. This was
something that you asked about earlier and it was answered.


> And all of this continues to be a side-issue to the fact that in the
>>> specific case of Equatable/Hashable, which thus far has gone ignored, is
>>> that bolting this on retroactively to an existing protocol *hides bugs*.
>>> The issue of reflective default implementations is less of a concern on
>>> very clearly and well defined *new* protocols, though I still prefer
>>> more, rather than less, control, but in the specific case of *existing* protocols
>>> this fucking about with behaviours is reckless and foolish in the extreme,
>>> yet no-one on the core teams seems willing or able to justify it, which
>>> only opens much wider concerns (how am I to have any faith in Swift's
>>> development if the core team can't or won't justify the creation of new
>>> bugs?).
>>>
>>
>> This has emphatically not gone ignored, as I have myself responded to
>> this point in an earlier thread in which you commented, as well as many
>> others. Crucially, no existing conforming type changes its behavior, as
>> they have all had to implement these requirements themselves. And as I said
>> to you already, the addition of a synthesized default implementation no
>> more "hides bugs" going forward than the addition of a non-synthesized
>> default implementation to an existing protocol, and we do that with some
>> frequency without even Swift Evolution review.
>>
>>
>> Feel free to a supply a non-synthesised default implementation for
>> Equatable without the use of reflection. Go-on, I'll wait.
>> You insist on suggesting these are the same thing, yet if you can't
>> provide one then clearly they are not.
>>
>
> That is not the argument. The argument is that they are indistinguishable
> in the sense that the author of a type who intends to supply a custom
> implementation but neglects to do so will have a default implementation
> supplied for them. It is plainly true that this is no more or less likely
> to happen simply because the default implementation is synthesised.
>
>
> A non-synthesised/reflective implementation cannot strictly be incorrect,
> because as long as it is implemented properly it will always be correct
> within the context of the protocol itself. It may not go quite as far as an
> end developer might want, but that is because they want to add something
> onto the protocol, not because the protocol is wrong.
>
> A synthesised/reflective implementation differs because if it goes too far
> it is wrong not only within the context of the concrete type, but also the
> protocol itself, it is simply incorrect.
>

Again, this is an assertion that misses the mark. If the default
implementation is unsuitable for a type, it’s unsuitable whether it
“doesn’t go quite as far” or “goes too far.”

You state but do not give any rationale for the claim that the former is
not wrong in some context while the latter is always wrong.

By this line of argumentation, you’d be perfectly content if instead we
simply had the default implementation of == as “return true” because it
would be somehow not wrong.

Put another way, what the proposal about synthesizing implementations for
>> Equatable and Hashable was about can be thought of in two parts: (a) should
>> there be default implementations; and (b) given that it is impossible to
>> write these in Swift, should we use magic? Now, as I said above, adding
>> default implementations isn't (afaik) even considered an API change that
>> requires review on this list. Really, what people were debating was (b),
>> whether it is worth it to implement compiler-supported magic to make these
>> possible. Your disagreement has to do with (a) and not (b).
>>
>>
>> Wrong. The use of magic in this case produces something else entirely;
>> that's the whole point. It is *not the same*, otherwise it wouldn't be
>> needed at all. It doesn't matter if it's compiler magic, some external
>> script or a native macro, ultimately they are all doing something with a
>> concrete type that is currently not possible.
>>
>> And once again; *I am not arguing against a default implementation that
>> cuts boilerplate*, I am arguing against it being implicit. What I want
>> is to be the one asking for it, because it is not reasonable to assume that
>> just throwing it in there is always going to be fine, because it quite
>> simply is not.
>>
>
> If you have to ask for it, then it's not a default. You *are* against a
> default implementation.
>
>
> A default implementation is an implementation that I, as the concrete type
> developer, do not have to provide myself. If you want default to mean only
> "automatic" then your attempt to pigeon-hole what I am arguing is
> incorrect, because what I am arguing is then neither about default
> implementations nor the means of actually implementing it, but something
> else entirely.
>
> But as far as I'm concerned it still absolutely still a default
> implementation whether it is requested or not; the difference is I, as the
> end developer, am able to refine what type of defaults that I want.
>

The word “default” indicates something that arises in the absence of a user
indication otherwise.


> On 9 Sep 2017, at 23:17, Gwendal Roué <gwendal.roue at gmail.com> wrote:
>
> All right, I'll be more positive: our science, IT, is a *constructive*
> science, by *essence*. If there is a problem, there must be a way to show
> it.
> It you can't, then there is no problem.
>
>
> You mean just as I have asked for examples that prove
> non-synthetic/reflective default implementations are as dangerous as
> synthetic/reflective ones? Plenty have suggested this is the case yet no
> reasonable examples of that have been given either.
>
> However, examples highlighting problems with the synthesised behaviour are
> simple:
>
> struct Foo : Equatable { var data:String } // Currently an error, won't
> be in future
>
>
> Or something a bit more substantial:
>
> struct KeyPair : Equatable {
> static var count:Int = 0
>
> var count:Int
> let key:String // This is the only property that should be equatable
> var value:String
>
> init(key:String, value:String) {
> let count = KeyPair.count &+ 1
> KeyPair.count = count; self.count = count
> self.key = key; self.value = value
> }
> }
>
> Here the only important property in the key pair is the key, the value
> isn't important (only the keys are to be considered unique) and the count
> is just a throwaway value. The synthesised default implementation for this
> concrete type will therefore be completely wrong, likewise for Hashable,
> which will likely produce radically different results for instances that
> should be the same.
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170912/72b4f9ad/attachment.html>