<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">David Hart <a href="https://twitter.com/dhartbit/status/891766239340748800" class="">recently asked on Twitter</a> if there was a good way to add Decodable support to somebody else's class. The short answer is "no, because you don't control all the subclasses", but David already understood that and wanted to know if there was anything working to mitigate the problem. So I decided to write up a long email about it instead. (Well, actually I decided to write a short email and then failed at doing so.)<div class=""><br class=""></div><div class=""><b class="">The Problem</b></div><div class=""><br class=""></div><div class="">You can add Decodable to someone else's struct today with no problems:<div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class="">extension<span style="color: rgb(0, 0, 0);" class=""> </span><span style="color: rgb(79, 129, 135);" class="">Point</span><span style="color: rgb(0, 0, 0);" class="">: </span><span style="color: rgb(112, 61, 170);" class="">Decodable</span><span style="color: rgb(0, 0, 0);" class=""> {</span></font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">enum</span> CodingKeys: <span style="color: #703daa" class="">String</span>, <span style="color: #703daa" class="">CodingKey</span> {</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">case</span> x</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">case</span> y</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> }</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">public</span> <span style="color: #ba2da2" class="">init</span>(from decoder: <span style="color: #703daa" class="">Decoder</span>) <span style="color: #ba2da2" class="">throws</span> {</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> container = <span style="color: #ba2da2" class="">try</span> decoder.<span style="color: #3e1e81" class="">container</span>(keyedBy: <span style="color: #4f8187" class="">CodingKeys</span>.<span style="color: #ba2da2" class="">self</span>)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> x = <span style="color: #ba2da2" class="">try</span> container.<span style="color: #3e1e81" class="">decode</span>(<span style="color: #703daa" class="">Double</span>.<span style="color: #ba2da2" class="">self</span>, forKey: .<span style="color: #31595d" class="">x</span>)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> y = <span style="color: #ba2da2" class="">try</span> container.<span style="color: #3e1e81" class="">decode</span>(<span style="color: #703daa" class="">Double</span>.<span style="color: #ba2da2" class="">self</span>, forKey: .<span style="color: #31595d" class="">y</span>)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">self</span>.<span style="color: #ba2da2" class="">init</span>(x: x, y: y)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> }</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class="">}</font></div></blockquote><br class=""><div class="">But if Point is a (non-final) class, then this gives you a pile of errors:</div><div class=""><br class=""></div><div class="">- init(from:) needs to be 'required' to satisfy a protocol requirement. 'required' means the initializer can be invoked dynamically on subclasses. Why is this important? Because someone might write code like this:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""><span style="color: #ba2da2" class="">func</span> decodeMe<Result: <span style="color: #703daa" class="">Decodable</span>>() -> <span style="color: #703daa" class="">Result</span> {</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> decoder = getDecoderFromSomewhere()</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">return</span> <span style="color: #703daa" class="">Result</span>(from: decoder)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class="">}</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""><span style="color: #ba2da2" class="">let</span> specialPoint: <span style="color: #703daa" class="">VerySpecialSubclassOfPoint</span> = decodeMe()</font></div></blockquote><br class=""><div class="">…and the compiler can't stop them, because VerySpecialSubclassOfPoint is a Point, and Point is Decodable, and therefore VerySpecialSubclassOfPoint is Decodable. A bit more on this later, but for now let's say that's a sensible requirement.</div><div class=""><br class=""></div><div class="">- init(from:) also has to be a 'convenience' initializer. That one makes sense too—if you're outside the module, you can't necessarily see private properties, and so of course you'll have to call another initializer that can.</div><div class=""><br class=""></div><div class="">But once it's marked 'convenience' and 'required' we get "'required' initializer must be declared directly in class 'Point' (not in an extension)", and that defeats the whole purpose. Why this restriction?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">The Semantic Reason</b></div><div class=""><b class=""><br class=""></b></div><div class="">The initializer is 'required', right? So all subclasses need to have access to it. But the implementation we provided here might not make sense for all subclasses—what if VerySpecialSubclassOfPoint doesn't have an 'init(x:y:)' initializer? Normally, the compiler checks for this situation and makes the subclass reimplement the 'required' initializer…but that only works if the 'required' initializers are all known up front. So it can't allow this new 'required' initializer to go by, because someone might try to call it dynamically on a subclass. Here's a dynamic version of the code from above:</div><div class=""><br class=""></div><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""><span style="color: rgb(186, 45, 162);" class="">func</span> decodeDynamic(<span style="color: rgb(186, 45, 162);" class="">_</span> pointType: <span style="color: rgb(112, 61, 170);" class="">Point</span>.Type) -> <span style="color: rgb(112, 61, 170);" class="">Point</span> {</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: rgb(186, 45, 162);" class="">let</span> decoder = getDecoderFromSomewhere()</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: rgb(186, 45, 162);" class="">return</span> pointType.<span style="color: rgb(186, 45, 162);" class="">init</span>(from: decoder)</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class="">}</font></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""><span style="color: rgb(186, 45, 162);" class="">let</span> specialPoint = decodeDynamic(VerySpecialSubclassOfPoint.<span style="color: rgb(186, 45, 162);" class="">self</span>)</font></div></blockquote><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">The Implementation Reason</b></div><div class=""><b class=""><br class=""></b></div><div class="">'required' initializers are like methods: they may require dynamic dispatch. That means that they get an entry in the class's dynamic dispatch table, commonly known as its <i class="">vtable</i>. Unlike Objective-C method tables, vtables aren't set up to have entries arbitrarily added at run time.</div><div class=""><br class=""></div><div class="">(Aside: This is one of the reasons why non-@objc methods in Swift extensions can't be overridden; if we ever lift that restriction, it'll be by using a separate table and a form of dispatch similar to objc_msgSend. I sent a proposal to swift-evolution about this last year but there wasn't much interest.)</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">The Workaround</b></div><div class=""><b class=""><br class=""></b></div><div class="">Today's answer isn't wonderful, but it does work: write a wrapper struct that conforms to Decodable instead:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""><span style="color: rgb(186, 45, 162);" class="">struct</span> DecodedPoint: <span style="color: rgb(112, 61, 170);" class="">Decodable</span> {</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">var</span> value: <span style="color: #4f8187" class="">Point</span></font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">enum</span> CodingKeys: <span style="color: #703daa" class="">String</span>, <span style="color: #703daa" class="">CodingKey</span> {</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">case</span> x</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">case</span> y</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> }</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">public</span> <span style="color: #ba2da2" class="">init</span>(from decoder: <span style="color: #703daa" class="">Decoder</span>) <span style="color: #ba2da2" class="">throws</span> {</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> container = <span style="color: #ba2da2" class="">try</span> decoder.<span style="color: #3e1e81" class="">container</span>(keyedBy: <span style="color: #4f8187" class="">CodingKeys</span>.<span style="color: #ba2da2" class="">self</span>)</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> x = <span style="color: #ba2da2" class="">try</span> container.<span style="color: #3e1e81" class="">decode</span>(<span style="color: #703daa" class="">Double</span>.<span style="color: #ba2da2" class="">self</span>, forKey: .<span style="color: #31595d" class="">x</span>)</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">let</span> y = <span style="color: #ba2da2" class="">try</span> container.<span style="color: #3e1e81" class="">decode</span>(<span style="color: #703daa" class="">Double</span>.<span style="color: #ba2da2" class="">self</span>, forKey: .<span style="color: #31595d" class="">y</span>)</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> <span style="color: #ba2da2" class="">self</span>.<span style="color: #4f8187" class="">value</span> = <span style="color: #4f8187" class="">Point</span>(x: x, y: y)</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class=""> }</font></div></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><font face="Menlo" class="">}</font></div></div></blockquote><div class=""><br class=""></div><div class="">This doesn't have any of the problems with inheritance, because it only handles the base class, Point. But it makes everywhere else a little less convenient—instead of directly encoding or decoding Point, you have to use the wrapper, and that means no implicitly-generated Codable implementations either.</div><div class=""><br class=""></div><div class="">I'm not going to spend more time talking about this, but it is the officially recommended answer at the moment. You can also just have all your own types that contain points manually decode the 'x' and 'y' values and then construct a Point from that.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">Future Direction: 'required' + 'final'</b></div><div class=""><b class=""><br class=""></b></div><div class="">One language feature we could add to make this work is a 'required' initializer that is also 'final'. Because it's 'final', it wouldn't have to go into the dynamic dispatch table. But because it's 'final', we have to make sure its implementation works on all subclasses. For that to work, it would only be allowed to call other 'required' initializers…which means you're still stuck if the original author didn't mark anything 'required'. Still, it's a safe, reasonable, and contained extension to our initializer model.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">Future Direction: runtime-checked convenience initializers</b></div><div class=""><b class=""><br class=""></b></div><div class="">In most cases you don't care about hypothetical subclasses or invoking init(from:) on some dynamic Point type. If there was a way to mark init(from:) as something that was always available on subclasses, but dynamically checked to see if it was okay, we'd be good. That could take one of two forms:</div><div class=""><br class=""></div><div class="">- If 'self' is not Point itself, trap.</div><div class="">- If 'self' did not inherit or override all of Point's designated initializers, trap.</div><div class=""><br class=""></div><div class="">The former is pretty easy to implement but not very extensible. The latter seems more expensive: it's information we already check in the compiler, but we don't put it into the runtime metadata for a class, and checking it at run time requires walking up the class hierarchy until we get to the class we want. This is all predicated on the idea that this is rare, though.</div><div class=""><br class=""></div><div class="">This is a much more intrusive change to the initializer model, and it's turning a compile-time check into a run-time check, so I think we're less likely to want to take this any time soon.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">Future Direction: Non-inherited conformances</b></div><div class=""><b class=""><br class=""></b></div><div class="">All of this is only a problem because people might try to call init(from:) on a subclass of Point. If we said that subclasses of Point weren't automatically Decodable themselves, we'd avoid this problem. This sounds like a terrible idea but it actually doesn't change very much in practice. Unfortunately, it's also a very complicated and intrusive change to the Swift protocol system, and so I don't want to spend more time on it here.</div><div class=""><b class=""><br class=""></b></div><div class=""><b class=""><br class=""></b></div><div class=""><b class="">The Dangers of Retroactive Modeling</b></div><div class=""><b class=""><br class=""></b></div><div class="">Even if we magically make this all work, however, there's still one last problem: <i class="">what if two frameworks do this?</i> Point can't conform to Decodable in two different ways, but neither can it just pick one. (Maybe one of the encoded formats uses "dx" and "dy" for the key names, or maybe it's encoded with polar coordinates.) There aren't great answers to this, and it calls into question whether the struct "solution" at the start of this message is even sensible.</div><div class=""><br class=""></div><div class="">I'm going to bring this up on swift-evolution soon as part of the Library Evolution discussions (there's a very similar problem if the library that owns Point decides to make it Decodable too), but it's worth noting that the wrapper struct solution doesn't have this problem.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Whew! So, that's why you can't do it. It's not a very satisfying answer, but it's one that falls out of our compile-time safety rules for initializers. For more information on this I suggest checking out my write-up of some of our <a href="https://github.com/apple/swift/blob/master/docs/InitializerProblems.rst" class="">initialization model problems</a>. And I plan to write another email like this to discuss some solutions that are actually doable.</div><div class=""><br class=""></div><div class="">Jordan</div><div class=""><br class=""></div><div class="">P.S. There's a reason why Decodable uses an initializer instead of a factory-like method on the type but I can't remember what it is right now. I think it's something to do with having the right result type, which would have to be either 'Any' or an associated type if it wasn't just 'Self'. (And if it is 'Self' then it has all the same problems as an initializer and would require extra syntax.) Itai would know for sure.</div></body></html>