[swift-evolution] Why you can't make someone else's class Decodable: a long-winded explanation of 'required' initializers

[Charles Srstka]

> On Aug 3, 2017, at 12:05 PM, Itai Ferber via swift-evolution <swift-evolution at swift.org> wrote:
> 
> Thanks for putting these thoughts together, Jordan! Some additional comments inline.
> 
>> On Aug 2, 2017, at 5:08 PM, Jordan Rose <jordan_rose at apple.com <mailto:jordan_rose at apple.com>> wrote:
>> 
>> David Hart recently asked on Twitter <https://twitter.com/dhartbit/status/891766239340748800> if there was a good way to add Decodable support to somebody else's class. The short answer is "no, because you don't control all the subclasses", but David already understood that and wanted to know if there was anything working to mitigate the problem. So I decided to write up a long email about it instead. (Well, actually I decided to write a short email and then failed at doing so.)
>> 
>> The Problem
>> 
>> You can add Decodable to someone else's struct today with no problems:
>> 
>> extension Point: Decodable {
>>   enum CodingKeys: String, CodingKey {
>>     case x
>>     case y
>>   }
>>   public init(from decoder: Decoder) throws {
>>     let container = try decoder.container(keyedBy: CodingKeys.self)
>>     let x = try container.decode(Double.self, forKey: .x)
>>     let y = try container.decode(Double.self, forKey: .y)
>>     self.init(x: x, y: y)
>>   }
>> }
>> 
>> But if Point is a (non-final) class, then this gives you a pile of errors:
>> 
>> - init(from:) needs to be 'required' to satisfy a protocol requirement. 'required' means the initializer can be invoked dynamically on subclasses. Why is this important? Because someone might write code like this:
>> 
>> func decodeMe<Result: Decodable>() -> Result {
>>   let decoder = getDecoderFromSomewhere()
>>   return Result(from: decoder)
>> }
>> let specialPoint: VerySpecialSubclassOfPoint = decodeMe()
>> 
>> …and the compiler can't stop them, because VerySpecialSubclassOfPoint is a Point, and Point is Decodable, and therefore VerySpecialSubclassOfPoint is Decodable. A bit more on this later, but for now let's say that's a sensible requirement.
>> 
>> - init(from:) also has to be a 'convenience' initializer. That one makes sense too—if you're outside the module, you can't necessarily see private properties, and so of course you'll have to call another initializer that can.
>> 
>> But once it's marked 'convenience' and 'required' we get "'required' initializer must be declared directly in class 'Point' (not in an extension)", and that defeats the whole purpose. Why this restriction?
>> 
>> 
>> The Semantic Reason
>> 
>> The initializer is 'required', right? So all subclasses need to have access to it. But the implementation we provided here might not make sense for all subclasses—what if VerySpecialSubclassOfPoint doesn't have an 'init(x:y:)' initializer? Normally, the compiler checks for this situation and makes the subclass reimplement the 'required' initializer…but that only works if the 'required' initializers are all known up front. So it can't allow this new 'required' initializer to go by, because someone might try to call it dynamically on a subclass. Here's a dynamic version of the code from above:
>> 
>> func decodeDynamic(_ pointType: Point.Type) -> Point {
>>   let decoder = getDecoderFromSomewhere()
>>   return pointType.init(from: decoder)
>> }
>> let specialPoint = decodeDynamic(VerySpecialSubclassOfPoint.self)
>> 
>> 
>> The Implementation Reason
>> 
>> 'required' initializers are like methods: they may require dynamic dispatch. That means that they get an entry in the class's dynamic dispatch table, commonly known as its vtable. Unlike Objective-C method tables, vtables aren't set up to have entries arbitrarily added at run time.
>> 
>> (Aside: This is one of the reasons why non- at objc methods in Swift extensions can't be overridden; if we ever lift that restriction, it'll be by using a separate table and a form of dispatch similar to objc_msgSend. I sent a proposal to swift-evolution about this last year but there wasn't much interest.)
>> 
>> 
>> The Workaround
>> 
>> Today's answer isn't wonderful, but it does work: write a wrapper struct that conforms to Decodable instead:
>> 
>> struct DecodedPoint: Decodable {
>>   var value: Point
>>   enum CodingKeys: String, CodingKey {
>>     case x
>>     case y
>>   }
>>   public init(from decoder: Decoder) throws {
>>     let container = try decoder.container(keyedBy: CodingKeys.self)
>>     let x = try container.decode(Double.self, forKey: .x)
>>     let y = try container.decode(Double.self, forKey: .y)
>>     self.value = Point(x: x, y: y)
>>   }
>> }
>> 
>> This doesn't have any of the problems with inheritance, because it only handles the base class, Point. But it makes everywhere else a little less convenient—instead of directly encoding or decoding Point, you have to use the wrapper, and that means no implicitly-generated Codable implementations either.
>> 
>> I'm not going to spend more time talking about this, but it is the officially recommended answer at the moment. You can also just have all your own types that contain points manually decode the 'x' and 'y' values and then construct a Point from that.
> I would actually take this a step further and recommend that any time you intend to extend someone else’s type with Encodable or Decodable, you should almost certainly write a wrapper struct for it instead, unless you have reasonable guarantees that the type will never attempt to conform to these protocols on its own.
> 
> This might sound extreme (and inconvenient), but Jordan mentions the issue here below in The Dangers of Retroactive Modeling. Any time you conform a type which does not belong to you to a protocol, you make a decision about its behavior where you might not necessarily have the "right" to — if the type later adds conformance to the protocol itself (e.g. in a library update), your code will no longer compile, and you’ll have to remove your own conformance. In most cases, that’s fine, e.g., there’s not much harm done in dropping your custom Equatable conformance on some type if it starts adopting it on its own. The real risk with Encodable and Decodable is that unless you don’t care about backwards/forwards compatibility, the implementations of these conformances are forever.
> 
> Using Point here as an example, it’s not unreasonable for Point to eventually get updated to conform to Codable. It’s also not unreasonable for the implementation of Point to adopt the default conformance, i.e., get encoded as {"x": …, "y": …}. This form might not be the most compact, but it leaves room for expansion (e.g. if Point adds a z field, which might also be reasonable, considering the type doesn’t belong to you). If you update your library dependency with the new Point class and have to drop the conformance you added to it directly, you’ve introduced a backwards and forwards compatibility concern: all new versions of your app now encode and decode a new archive format, which now requires migration. Unless you don’t care about other versions of your app, you’ll have to deal with this:
> Old versions of your app which users may have on their devices cannot read archives with this new format
> New versions of your app cannot read archives with the old format
> 
> Unless you don’t care for some reason, you will now have to write the wrapper struct, to either
> Have new versions of your app attempt to read old archive versions and migrate them forward (leaving old app versions in the dust), or
> Write all new archives with the old format so old app versions can still read archives written with newer app versions, and vice versa
> 
> Either way, you’ll need to write some wrapper to handle this; it’s significantly safer to do that work up front on a type which you do control (and safely allow Point to change out underneath you transparently), rather than potentially end up between a rock and a hard place later on because a type you don’t own changes out from under you.

I should point out that there’s a third case: the case where you want to add conformance to a type from another framework, but you own both frameworks.

Plenty of examples of this can be found in the Cocoa frameworks, actually. For example, as NSString is, of course, declared in Foundation, its original declaration cannot conform to the NSPasteboardReading protocol, which is declared in AppKit. As a result, Apple declares NSString’s NSPasteboardReading support in a category in AppKit. There are reasons one might want to do the same thing in their own code—make one library and/or framework for use with Foundation-only programs, and extend a type from that library/framework with NSPasteboardReading support in a separate framework. It can’t currently be done with Swift, though.

Charles

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170806/4b92e503/attachment.html>