[swift-evolution] [Oversight] Reference types allow mutating methods through generics

Matthew Johnson matthew at anandabits.com
Thu May 5 11:38:02 CDT 2016 

> On May 5, 2016, at 11:06 AM, Joe Groff via swift-evolution <swift-evolution at swift.org> wrote:
> 
>> 
>> On May 5, 2016, at 8:56 AM, Dave Abrahams <dabrahams at apple.com> wrote:
>> 
>> 
>> on Wed May 04 2016, Joe Groff <jgroff-AT-apple.com> wrote:
>> 
>>>> On May 4, 2016, at 5:28 AM, T.J. Usiyan via swift-evolution <swift-evolution at swift.org> wrote:
>>>> 
>>>> Something about your first paragraph reminded me of a question I've
>>>> had for a while. Is there a reasoning behind not being able to
>>>> restrict a protocol to value types? One way that this might be
>>>> workable is if we could overload protocols for Value vs for
>>>> reference.
>>> 
>>> I'm not totally comfortable with this, because references are a kind
>>> of value. 
>> 
>> We're using the word “value” in 3 different ways now.  If we don't sort
>> them out, this is going to become impossible to resolve.  So let me
>> propose some terms:
>> 
>> 1. Joe's sense, i.e. the compiler-implementation-level sense, in which a
>>  value of type T is essentially what an UnsafePointer<T> points at.
>>  In this sense, a variable of any type T “has a value.”  Even though
>>  it's not strictly correct (since the term really applies to
>>  expressions), I propose “rvalue” for this one.  
>> 
>> 2. The “value types vs reference types” sense, where every type falls
>>  into one of two crisp buckets based on how it's declared.  I propose
>>  we always say “reference type” or “value type” for this one.
>> 
>> 3. The “value semantics vs reference semantics” sense, where a type's
>>  category depends on how it is implemented,  and it's possible (though
>>  inadvisable) to fall outside either bucket.  This is the only
>>  interesting category when you're discussing protocols and
>>  constraints, and doesn't have any intrinsic relationship to sense
>>  #2.  I propose we always say “value semantics” or “reference
>>  semantics” for this one.
> 
> I claim that my sense is the same as #2. A reference has all the properties of a value-type value; it has the additional ability to be used as a handle to access independent program state. Int isn't any less of a value because it can be used to reference values in an array. Rather than try to break the world down according to categories of types, I think it's more effective to look at the semantics of *operations*. So called "reference types" have value semantics operations too; reassigning a class reference, '+='-ing a pointer, and adding or removing elements from an Array of references are all operations that mutate only the value being operated on, even though none of these types are strictly "value types”.

The distinction between “reference as a value” and “value the reference points to” is a pretty important one.  All of the operations you discuss are really operations on the “reference as a value”. 

I think the reason we talk about *types* rather than *operations* having value semantics is because that tells us that *all* operations on any instance are going to have value semantics.  This means that *nobody else* is going to mutate the variable in question behind our back.  If the type has “reference semantic operations” we don’t have that guarantee today.  The fact that the type may also have some "value semantic operations" seems relatively unimportant.

One possible avenue for improvement in the language where this distinction could be useful is by allowing us to specify which operations have "value semantics" on reference types and allow us to designate specific *references* as being value semantic.  `let` wouldn’t work as that only means that the reference itself is immutable, not the value it refers to.  We would need some additional syntax to specify that the value the reference points to cannot be mutated.  Such a reference could only be assigned to when the compiler can prove it will refer to a value that has *no* existing mutable references.  This would allow us to capture the benefits of value semantics for specific *instances* of a reference type.

-Matthew


> 
> -Joe
> 
>>> I can see value in there being some kind of PureValue protocol, for
>>> types that represent fully self-contained values, but conforming to
>>> that protocol requires a bit more thought than just being a struct or
>>> enum, since there are structs that have reference semantics (such as
>>> UnsafePointer), and there are hybrid value types that contain
>>> references to data that isn't part of the value (an Array<Class>, for
>>> instance).
>>> 
>>> -Joe
>>> 
>>>> TJ
>>>> 
>>>> On Tue, May 3, 2016 at 11:02 PM, Jordan Rose <jordan_rose at apple.com> wrote:
>>>> Dave and I have pondered this before, and considered that one possible (drastic) solution is to ban classes from implementing protocols with mutating members, on the grounds that it’s very hard to write an algorithm that’s correct for both.
>>>> 
>>>> func removing(_ element: Element) -> Self {
>>>> var result = self // not necessarily a copy…
>>>> result.remove(element)
>>>> return result // not necessarily an independent value
>>>> }
>>>> 
>>>> func zapBadElements<C: RangeReplaceableCollection where C.Generator.Element == Int>(_ nums: inout C) {
>>>> // requires inout on ‘nums’  even when it’s a class
>>>> for i in nums.indices {
>>>>   if nums[i] < 0 {
>>>>     nums.removeAtIndex(i)
>>>>   }
>>>> }
>>>> // …because of this.
>>>> if nums.lazy.filter { $0 == 0 }.count > 5 {
>>>>   nums = C()
>>>> }
>>>> }
>>>> 
>>>> var refCollection: SharedArrayOfSomeKind<Int> = …
>>>> // either the variable ‘refCollection’ or the instance of ‘SharedArrayOfSomeKind’ might be mutated…or both!
>>>> zapBadElements(&refCollection)
>>>> 
>>>> There are of course ways to safely use a protocol with mutating requirements with classes, namely if you only use them for mutation (i.e. they’re only called from ‘mutating’ members or on ‘inout’ parameters) and never rely on value copying (no assignment, no returning). Most simple wrappers around mutating members would fall into this category.
>>>> 
>>>> We didn’t really develop the idea very far yet because there’s been more pressing things to worry about. I’m bringing it up here because it’s an important idea that shouldn’t get lost.
>>>> 
>>>> ---
>>>> 
>>>> In lieu of this, I and a few others brought up the “incorrect” behavior of reassigning ‘self’ in a protocol extension when the model type is a class, and got shot down. I don’t have those discussions on hand at the moment, but I remember we deliberately decided to leave protocol extensions the way they were, allowing them to reassign class references. I think it’s because it means things like zapBadElements are more likely to work correctly^W as expected―if you don’t have any other references at the time you do the mutation, it can work. But yeah, I’m uncomfortable with the situation we’re in right now.
>>>> 
>>>> Jordan
>>>> 
>>>> 
>>>>> On May 3, 2016, at 13:09, James Froggatt via swift-evolution <swift-evolution at swift.org> wrote:
>>>>> 
>>>>> Thanks for the response, I agree this is currently the best
>>>>> solution. Unfortunately, it's not just as simple as just
>>>>> implementing each method, since without being able to call super, I
>>>>> have to fully reimplement the original behaviour, which at best
>>>>> seems like bad practice, and would break in future versions of
>>>>> Swift, and at worst could lead to hard-to-detect bugs right now.
>>>>> 
>>>>> To recap for anyone reading, protocol extensions currently apply
>>>>> mutating methods unmodified to reference types, as I found trying
>>>>> to make a reference-type collection. This results in the compiler
>>>>> disallowing ‘let’ when calling these functions, and allows methods
>>>>> to reassign the reference ‘self’ to a new object. The best solution
>>>>> is to manually implement each method, removing the mutating
>>>>> modifier, yet this workaround doesn't extend to generic code.
>>>>> 
>>>>> To fix this behaviour, we would need to distinguish between ‘true’
>>>>> mutating functions, which reassign self, and ‘partially’ mutating
>>>>> functions, for use in generics and protocol extensions, which can
>>>>> reassign properties only.
>>>>> Is there any support for making this change? Or are there any simpler solutions?
>>>>> 
>>>>> I did submit a bug report, but I'm pretty sure a decent fix is not
>>>>> possible without some evolution of the language regarding the
>>>>> mutating keyword, so I'm trying to bring this up here in hope of us
>>>>> getting an actual solution. I've changed the title to what I hope
>>>>> is something that better reflects the problem; this thread was
>>>>> originally titled ‘[swift-evolution] [Bug?] Reference types and
>>>>> mutating methods’.
>>>>> 
>>>>> 
>>>>> PS: I have noticed another side-effect of calling mutating
>>>>> functions on my reference-type collection: it seems to trigger
>>>>> didChange on properties, even when, upon comparing the new and old
>>>>> objects, the reference isn't being changed. I haven't done much
>>>>> experimentation with this behaviour; this may be an unexpected
>>>>> side-effect of an extension method assigning to self, but it feels
>>>>> like it could be undefined behaviour.
>>>>> 
>>>>> From James F
>>>>> 
>>>>> On 30 Apr 2016, at 16:38, T.J. Usiyan <griotspeak at gmail.com> wrote:
>>>>> 
>>>>>> The problem here seems to be with using the default implementation provided. If you override `append` in ObservedArray, the compiler allows it. That seems 'safe' but odd at first. I wouldn't *want* to implement every mutating method, but that is the current solution. I haven't puzzled out the reasoning behind this myself.
>>>>>> 
>>>>>> 
>>>>>> ``` swift
>>>>>> class ObservedArray<T> : ArrayLiteralConvertible {
>>>>>>   var value: [T]
>>>>>>   init(value: [T]) {
>>>>>>       self.value = value
>>>>>>   }
>>>>>>   required init() {
>>>>>>       self.value = []
>>>>>>   }
>>>>>> 
>>>>>>   required convenience init(arrayLiteral elements: T...) {
>>>>>>       self.init(elements)
>>>>>>   }
>>>>>> 
>>>>>> }
>>>>>> 
>>>>>> extension ObservedArray {
>>>>>>   typealias Index = Int
>>>>>> 
>>>>>>   var startIndex: Index {
>>>>>>       return value.startIndex
>>>>>>   }
>>>>>> 
>>>>>>   var endIndex: Index {
>>>>>>       return value.endIndex
>>>>>>   }
>>>>>> 
>>>>>>   subscript(position: Index) -> T {
>>>>>>       return value[position]
>>>>>>   }
>>>>>> 
>>>>>> }
>>>>>> 
>>>>>> extension ObservedArray : RangeReplaceableCollectionType {
>>>>>>   typealias Generator = IndexingGenerator<[T]>
>>>>>> 
>>>>>>   func generate() -> Generator {
>>>>>>       return value.generate()
>>>>>>   }
>>>>>> }
>>>>>> 
>>>>>> extension ObservedArray {
>>>>>>   func replaceRange<C : CollectionType where C.Generator.Element == Generator.Element>(subRange: Range<Index>, with newElements: C) {
>>>>>>       value.replaceRange(subRange, with: newElements)
>>>>>>   }
>>>>>> 
>>>>>>   func append(newElement: T) { // <- adding this makes it work
>>>>>>       value.append(newElement)
>>>>>>   }
>>>>>> }
>>>>>> 
>>>>>> let array: ObservedArray<String> = []
>>>>>> array.append("1")
>>>>>> 
>>>>>> 
>>>>>> ```
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Sat, Apr 30, 2016 at 7:52 AM, James Froggatt via swift-evolution <swift-evolution at swift.org> wrote:
>>>>>> I don't believe this has been addressed, please correct me if I'm wrong.
>>>>>> 
>>>>>> --My Situation--
>>>>>> I've recently been working on an observable collection type. Because each stores ‘subscriptions’ to changes that occur, it made sense to me that this should be a reference type, so subscriptions can't be copied with the values themselves.
>>>>>> 
>>>>>> I have made this class conform to RangeReplaceableCollectionType, providing it with all the standard collection functions. I do the following:
>>>>>> 
>>>>>> let array: ObservedArray<String> = []
>>>>>> array.append("1") //Error: Cannot use mutating member on immutable value: ‘array’ is a ‘let’ constant
>>>>>> 
>>>>>> I have to make the reference immutable just to use my new collection type? This is a bit of a deal-breaker.
>>>>>> 
>>>>>> --The Problem--
>>>>>> Mutating methods allow ‘self’ to be reassigned, which is just another way to mutate a value type. However, reassigning ‘self’ has a special meaning to reference types, which is presumably the reason they are disallowed in classes.
>>>>>> 
>>>>>> However, classes can conform to protocols with mutating methods, leading to the compiler disallowing calls to mutating methods for ‘let’ values of type ‘protocol<MutatingProtocol, AnyObject>’, which can be an annoyance in generic code. In addition, classes can inherit mutating methods from protocol extensions, leading to the behaviour I describe above.
>>>>>> 
>>>>>> Is this intentional behaviour? Am I going about this in the wrong way? Or is this really an omission in the language?
>>>>>> _______________________________________________
>>>>>> swift-evolution mailing list
>>>>>> swift-evolution at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>> 
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>> 
>>>> 
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
>> -- 
>> Dave
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160505/8e2b923b/attachment.html>

More information about the swift-evolution mailing list