<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 20, 2017, at 5:34 AM, Brent Royal-Gordon <<a href="mailto:brent@architechies.com" class="">brent@architechies.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="">On Mar 19, 2017, at 8:19 PM, Matthew Johnson <<a href="mailto:matthew@anandabits.com" class="">matthew@anandabits.com</a>> wrote:</div><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">First, your solution does not allow a user to see a context if they can't name the type (you can't get it as Any and use reflection, etc).</div></div></blockquote><div class=""><br class=""></div><div class="">What I meant is that, if you retrieve the context, you know it is of the type you expect. You don't need to *also* cast it.</div></div></div></div></blockquote><div><br class=""></div><div>Right. What I’m saying is that if all we’re doing is moving the cast in to the encoder / decoder I don’t see value in doing that over the obvious thing of exposing the context as Any? and letting the caller cast it. If the encoder / decoder uses the requested type in an algorithm to find the matching context then we obviously do need to pass the type as a parameter. :)</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I don't see this restriction as being beneficial. Second, your solution introduces several subtle problems mentioned in my last email which you didn't respond to (overlapping context types, etc). </div></div></blockquote><div class=""><br class=""></div><div class="">I mentioned that, if we give up storing values in a dictionary, we can come up with some sort of sensible-ish behavior for overlapping context types.</div></div></div></div></blockquote><div><br class=""></div>Oh, sorry. I missed that the breadth-first algorithm for finding a matching context was the answer to this.<br class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>protocol Encoder {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Retrieve the context instance of the indicated type.</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func context<Context>(ofType type: Context.Type) -> Context?</div></div></div></blockquote><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// This context is visible for `encode(_:)` calls from this encoder's containers all the way down, recursively.</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func addContext<Context>(_ context: Context, ofType type: Context.Type)</div></div></div></blockquote><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""></div><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">What happens if you call `addContext` more than once with values of the same type?</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">It overrides the previous context, but only for the containers created by this `encode(to:)` method and any containers nested within them.</div><div class=""><br class=""></div><div class="">(Although that could cause trouble for an encoder which only encodes objects with multiple instances once. Hmm.)</div><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">And why do you require the type to be passed explicitly when it is already implied by the type of the value?</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">As you surmised later, I was thinking in terms of `type` being used as a dictionary key; in that case, if you stored a `Foo` into the context, you would not later be able to look it up using one of `Foo`'s supertypes. But if we really do expect multiple contexts to be rare, perhaps we don't need a dictionary at all—we can just keep an array, loop over it with `as?`, and return the first (or last?) match. If that's what we do, then we probably don't need to pass the type explicitly.</div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">The array approach is better because at least there is an order to the contexts and we can assign precise semantics in the presence of overlapping context types by saying type get the first (most recent) context that can be cast to the type you ask for. </div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">That said, I think what you're really trying to model here is a context stack, isn't it? Why don't we just do that?</div></div></blockquote><div class=""><br class=""></div><div class="">You mention this a couple times, but I don't think it's really possible. Here's why.</div><div class=""><br class=""></div><div class="">Suppose you write these types:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct SomeObjectContext {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var oldFormat: Bool</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct Root: Codeable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var str: SomeStruct</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var obj: SomeObject</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(to encoder: Encoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>encoder.push(SomeObjectContext(oldFormat: true))</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let container = encoder.container(keyedBy: CodingKeys.self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>try container.encode(str, forKey: .str)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>try container.encode(obj, forKey: .obj)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>...</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct SomeStruct: Codeable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var obj: SomeObject</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(to encoder: Encoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>encoder.push(SomeObjectContext(oldFormat: false))</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let container = encoder.container(keyedBy: CodingKeys.self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>try container.encode(obj, forKey: .obj)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>class SomeObject: Codeable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(to encoder: Encoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let context = encoder.context(ofType: SomeObjectContext.self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>print(context.oldFormat)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">And you construct an object graph like this:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let object = SomeObject()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let root = Root(</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>str: SomeStruct(obj: object),</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>obj: object</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>)</div><div class=""><br class=""></div><div class="">And finally, you encode it with a coder which respects object identity, so that even if a given object appears in several different places in the object graph, it only encodes that object once.</div></div></div></div></blockquote><div><br class=""></div><div>Ahh, yes. Objects are pesky things! I wasn’t thinking about that because I only ever encode / decode values in a tree structure.</div><div><br class=""></div><div>This brings to mind a related question about how objects will be handled. What if I have an immutable object with value semantics? I might actually *want* all instances of that object to be encoded independently. The fact that the same value-semantic object is referenced in multiple places is incidental, it is not part of the data model. Do encoders / decoders need to know if they are dealing with a value-semantic Codable object so they can do the right thing here?</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><div class="">Which context does `SomeObject.encode(to:)` see? The one with `oldFormat` or the one without? Is there a "first to call wins" rule? Do we think that's sustainable?</div></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><div class="">There are probably a couple of complicated rules we could use to try to cancel this out. But by far the easiest is this: The set of contexts is cast in stone at the beginning of the encoding process and cannot be altered after the fact. </div></div></div></div></blockquote><div><br class=""></div><div>Yes, I agree there are many advantages to doing that.</div><div><br class=""></div><div>I think I was thrown off by this statement in your prior post: "The problem with providing all the contexts at the top level is that then the top level has to *know* what all the contexts needed are.”. This had me thinking you were proposing that all calls to encode / decode would be grabbing context from the Codable. I didn’t look at the code closely enough and missed that this wasn’t what was happening.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class="">And the simplest way I can come up with to do that, while still allowing libraries and subsystems to encapsulate their dependencies, is to let a context specify other contexts it needs.</div></div></div></div></blockquote><div><br class=""></div><div>This requires the top level to “know” about all of the needed contexts are in some sense. But I can see how the indirection of accessing that through the single context it provides could be useful.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><div class=""><div class=""><div class="">The problem with providing all the contexts at the top level is that then the top level has to *know* what all the contexts needed are. Again, if you're encoding a type from FooKit, and it uses a type from GeoKit, then you—the user of FooKit—need to know that FooKit uses GeoKit and how to make contexts for both of them. There's no way to encapsulate GeoKit's role in encoding.</div></div></div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">The use cases I know of for contexts are really around helping a type choose an encoding strategy. I can't imagine a real world use case where a Codable type would have a<span class="Apple-converted-space"> </span><i class="">required</i> context - it's easy enough to choose one strategy as the default.</div></blockquote><div class=""><br class=""></div><div class="">Hmm...</div><div class=""><br class=""></div><div class="">Here's a toy version of a real problem: I have an ebook editing app which uses a package format. The text of each chapter is stored in a separate HTML file, while a "toc.plist" file specifies the order and (recursive) structure of the chapters.</div><div class=""><br class=""></div><div class="">This is currently all handled with Objective-C and some custom serialization code I've written, but it could be done in Swift with Codable and a context containing the chapters:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct Chapter {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var id: ChapterID</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var html: Data</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct TOCEntry {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var chapter: Chapter</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var children: [TOCEntry]</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>extension TOCEntry: Codable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>class Context {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var chapters: [ChapterID: Chapter]</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func encode(to encoder: Encoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let context = encoder.context(ofType: Context.self)!</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let container = encoder.container(keyedBy: CodingKeys.self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>guard context.chapters[chapter.id] == nil else {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>throw BookError.crosslinkedChapter(chapterID: chapter.id)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>chapters[chapter.id] = chapter</div></div></div></div></blockquote><div><br class=""></div><div>Is this supposed to be `context.chapters[chapter.id] = chapter`?</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>try container.encode(chapter.id, forKey: .chapter)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>try container.encode(children, forKey: .children)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>init(from decoder: Decoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let context = encoder.context(ofType: Context.self)!</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>let container = encoder.container(keyedBy: CodingKeys.self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let chapterID = try container.decode(ChapterID.self, forKey: .chapter)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>guard let chapter = context.chapters[chapterID] else {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>throw BookError.missingChapter(chapterID: chapterID)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>self.chapter = chapter</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>self.children = try container.decode([TOCEntry].self)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">This is a type which simply would not encode or decode properly if it didn't have the right context. Of course, that breaks the "don't mutate the context" rule I've been suggesting so far, but what's a little hypocrisy between friends?</div></div></div></div></blockquote><div><br class=""></div><div>Lol. Mutating the context is kind of ugly! On the other hand this allows for the behavior of dynamic contexts to be recreated by users if that is the best solution to a specific problem.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><div class="">* * *</div><div class=""><br class=""></div><div class="">But.</div><div class=""><br class=""></div><div class="">The question is not whether the type can encode itself without being provided with hints about the encoding strategy. The question is whether it can encode itself *correctly for the use case in question* without being provided with hints about the encoding strategy. Even if a type can somehow stuff itself into the coder without a context, that doesn't mean it will stuff itself into the coder in the format you need.</div></div></div></div></blockquote><div><br class=""></div><div>I think types that <i class="">require</i> a context to be encoded correctly are probably very rare. There is usually a way to do something sensible as a default. But if you’re enforcing dynamic constraints like the one you show above, sure there is a need for a context.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">That said, I<span class="Apple-converted-space"> </span><i class="">can</i> imagine really evil and degenerate API designs that would require the same type to be encoded differently in different parts of the tree. I could imagine dynamic contexts being helpful in solving<span class="Apple-converted-space"> </span><i class="">some</i> of these cases, but often you would need to look at the codingKeyContext to get it right.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div class=""><br class=""></div><div class="">I could too—it makes me think of the infamous "PDF is not my favorite format" rant <<a href="https://github.com/zepouet/Xee-xCode-4.5/blob/master/XeePhotoshopLoader.m#L108" class="">https://github.com/zepouet/Xee-xCode-4.5/blob/master/XeePhotoshopLoader.m#L108</a>>. To be honest, I won't be terribly unhappy if our design discourages that sort of thing. :^)</div></div></div></div></blockquote><div><br class=""></div><div>I don’t think anyone would argue that this kind of thing isn’t horrible. It is. I don’t mind discouraging it at all. I just don’t want to make it impossible to deal with. Sometimes the world is messy and we have to deal with the horrible.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">If you have a concrete real world use case involving module boundaries please elaborate. I'm having trouble imagining the details about a precise problem you would solve using dynamic contexts. I get the impression you have something more concrete in mind than I can think of.</div></blockquote><div class=""><br class=""></div><div class="">I don't really, but I can elaborate on the hypothetical example I've alluded to previously.</div><div class=""><br class=""></div><div class="">Suppose you're writing a framework to interact with a web service which is definitely not Yelp. It represents coordinates using a type from Core Location.</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct KelpBusiness {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var coordinates: CLLocation</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">The Kelp backend server is currently some Node.js monstrosity, but your developers are working on a 2.0 in Swift. As long as they're there, they're cleaning up a few things. One of them is that the old backend expected locations in this format:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>"-30.000000,50.000000"</div><div class=""><br class=""></div><div class="">But the new one will use this instead:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>{ "latitude": -30.0, "longitude": 50.0 }</div><div class=""><br class=""></div><div class="">Fortunately, CLLocation supports both of these properties—you just need to configure the CLCodingContext appropriately:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public struct CLCodingContext: CodingContext {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public enum JSONLocationFormat {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>case commaSeparatedString</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>case subobject</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public var jsonLocationFormat: JSONLocationFormat</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">But the change in this context has to happen in sync with the change in servers. So you write this:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public struct KelpCodingContext: CodingContext {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public enum Version {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>case v1, v2</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public var version: Version = .v1</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public var underlyingContexts: [CodingContext] {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>switch version {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>case .v1:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>return CLCodingContext(jsonLocationFormat: .commaSeparatedString)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>case .v2:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>return CLCodingContext(jsonLocationFormat: .subobject)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">And, et voilà, you're always using the right CLCodingContext for the server version.</div></div></div></div></blockquote><div><br class=""></div><div>That makes sense. Good example.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><div class=""><div class=""><div class="">On the other hand, there *could* be a way to encapsulate it. Suppose we had a context protocol:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>protocol CodingContext {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>var underlyingContexts: [CodingContext] { get }</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>extension CodingContext {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>var underlyingContexts: [CodingContext] { return [] }</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><br class=""></div><div class="">Then you could have this as your API surface:</div><div class=""><br class=""></div><div class=""><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>protocol Encoder {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Retrieve the context instance of the indicated type.</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func context<Context: CodingContext>(ofType type: Context.Type) -> Context?</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Likewise on Decoder</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Encoder and decoder classes should accept contexts in their top-level API:</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>open class JSONEncoder {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>open func encode<Value : Codable>(_ value: Value, with context: CodingContext? = nil) throws -> Data</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><br class=""></div><div class="">And libraries would be able to add additional contexts for dependencies as needed.</div><div class=""><br class=""></div><div class="">(Hmm. Could we maybe do this?</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>protocol Codable {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>associatedtype CodingContextType: CodingContext = Never</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func encode(to encoder: Encoder) throws</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>init(from decoder: Decoder) throws</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><br class=""></div><div class=""><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>protocol Encoder {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Retrieve the context instance of the indicated type.</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func context<CodableType: Codable>(for instance: Codable) -> CodableType.CodingContextType?</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Likewise on Decoder</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span></div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// Encoder and decoder classes should accept contexts in their top-level API:</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>open class JSONEncoder {</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>open func encode<Value : Codable>(_ value: Value, with context: Value.CodingContextType? = nil) throws -> Data</div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}</div></div><div class=""><br class=""></div><div class="">That would make sure that, if you did use a context, it would be the right one for the root type. And I don't believe it would have any impact on types which didn't use contexts.)</div></div></div></div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I think this is far more than we need. I think we could just say encoders and decoders keep a stack of contexts. Calls to encode or decode (including top level) can provide a context (or an array of contexts which are interpreted as a stack bottom on left, top on right). When the call returns the stack is popped to the point it was at before the call. We could also include an explicit `func push(contexts: Context...)` method on encoder and decoder to allow a Codable to set context used by all of its members. All calls to `push` would be popped when the current call to encode / decode returns.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Users ask for a context from an encoder / decoder using `func context<Context>(of: Context.Type) -> Context?`. The stack is searched from the top to the bottom for a value that can be successfully cast to Context.</div></blockquote><div class=""><br class=""></div>Again, I don't think a stack will work.</div><div class=""><br class=""></div><div class="">The CodingContextType thing was a bit of a flight of fancy; I sometimes like to push a design beyond what's actually practical. What do you think of the `underlyingContexts` part?<br class=""></div></div></div></blockquote><div><br class=""></div><div>Now that I had a second look and understand it better I like it very much.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><div class=""><div class=""><div class="">I also see it as an incentive for users to build a single context type rather than sprinkling in a whole bunch of separate keys. I really would prefer not to see people filling a `userInfo` dictionary with random primitive-typed values like `["json": true, "apiVersion": "1.4"]`; it seems too easy for names to clash or people to forget the type they're actually using. `context(…)` being a function instead of a subscript is similarly about ergonomics: it discourages you from trying to mutate your context during the encoding process (although it doesn't prevent it for reference types.)</div></div></div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I agree with this sentiment and indicated to Tony the desire to steer people away from treating this as a dictionary to put a lot of stuff in and towards defining an explicit context type. This and the fact that keys will feel pretty arbitrary are behind my desire to avoid the keys and dictionary approach.</div></blockquote><div class=""><br class=""></div><div class="">Yes, I'm not a fan of the arbitrary keys, either.</div><br class=""><blockquote type="cite" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><div class=""><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Unfortunately, I don't think there is a good answer to the question about multiple context values with the same type though. I can’t think of a good way to prevent this statically. Worse, the values might not have the same type, but be equally good matches for a type a user requests (i.e. both conform to the same protocol). I’m not sure how a user-defined encoder / decoder could be expected to find the “best” match using semantics that would make sense to Swift users (i.e. following the rules that are kind of the inverse to overload resolution). </div><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""></div><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Even if this were possible there are ambiguous cases where there would be equally good matches. Which value would a user get when requesting a context in that case? We definitely don’t want accessing the context to be a trapping or throwing operation. That leaves returning nil or picking a value at random. Both are bad choices IMO.</div><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"></div></div></div></blockquote><div class=""><br class=""></div><div class="">If we use the `underlyingContexts` idea, we could say that the context list is populated breadth-first and the first context of a particular type encountered wins. That would tend to prefer the context "closest" to the top-level one provided by the caller, which will probably have the best fidelity to the caller's preferences.</div></div></div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I'm not totally sure I follow you here, but I think you're describing stack-like semantics that are at least similar to what I have described. I think the stack approach is a pretty cool one that targets the kinds of problems multiple contexts are trying to solve more directly than the dictionary approach would.</div></blockquote></div><div class=""><br class=""></div><div class="">What I'm saying is that an Encoder or Decoder would do something like this:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>class MyEncoder: Encoder {</div><span class="Apple-tab-span" style="white-space: pre;"> </span>private var allContexts: [CodingContext]<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>init(context: CodingContext? = nil) {<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>allContexts = context.map { [$0] } ?? []<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// We walk through the allContexts array, appending underlyingContexts to the <br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// end as we go. This acts as a breadth-first search; the shallowest underlyingContexts<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// are towards the beginning of the list.<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// <br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>// We intentionally do not use high-level looping constructs so we can mutate the array.<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>var i = 0<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>while i < allContexts.count {<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>allContexts += allContexts[i].underlyingContexts<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>i++<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>func context<Context: CodingContext>(of type: Context.Type) -> Context? {<div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>// We search the list from top to bottom. Since the list is ordered shallowest to deepest, </div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>// this favors shallower contexts over deeper ones.<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>for context in allContexts {<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>if let matchingContext = context as? Context {<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>return matchingContext<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>return nil<br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class=""><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div></div></div></div></blockquote><div><br class=""></div><div>This makes a lot of sense (I think). It’s hard to say for sure if it would have counter-intuitive behavior in some cases but I think it would do the expected thing at leas the <i class="">majority</i> of the time. I’m interested in hearing what the Foundation team and others think about this direction. It seems promising to me.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""><div class="">
<span class="Apple-style-span" style="border-collapse: separate; font-variant-ligatures: normal; font-variant-position: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; line-height: normal; border-spacing: 0px;"><div class=""><div style="font-size: 12px; " class="">-- </div><div style="font-size: 12px; " class="">Brent Royal-Gordon</div><div style="font-size: 12px; " class="">Architechies</div></div></span>
</div>
<br class=""></div></div></div></blockquote></div><br class=""></body></html>