<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 17, 2017, at 3:23 PM, Brent Royal-Gordon via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="">On Mar 16, 2017, at 12:33 PM, Itai Ferber <<a href="mailto:iferber@apple.com" class="">iferber@apple.com</a>> wrote:</div><div class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><p dir="auto" class=""><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Optional</code> values are accepted and vended directly through the API. The <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_:forKey:)</code> methods take optional values directly, and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">decodeIfPresent(_:forKey:)</code> vend optional values.</p><p dir="auto" class=""><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Optional</code> is special in this way — it’s a primitive part of the system. It’s actually not possible to write an <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(to:)</code> method for <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Optional</code>, since the representation of null values is up to the encoder and the format it’s working in; <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">JSONEncoder</code>, for instance, decides on the representation of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">nil</code> (JSON <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">null</code>).</p></div></div></div></div></blockquote>Yes—I noticed that later but then forgot to revise the beginning. Sorry about that.<br class=""><blockquote type="cite" class=""><div class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class=""> It wouldn’t be possible to ask <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">nil</code> to encode itself in a reasonable way.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></div></blockquote><div class="">I really think it could be done, at least for most coders. I talked about this in another email, but in summary:</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class="">NSNull would become a primitive type; depending on the format, it would be encoded either as a null value or the absence of a value.</li><li class="">Optional.some(<i class="">x</i>) would be encoded the same as <i class="">x</i>.</li><li class="">Optional.none would be encoded in the following fashion:</li><ul class=""><li class="">If the <i class="">Wrapped</i> associated type was itself an optional type, it would be encoded as a keyed container containing a single entry. That entry's key would be some likely-unique value like "_swiftOptionalDepth"; its value would be the number of levels of optionality before reaching a non-optional type.</li><li class="">If the <i class="">Wrapped</i> associated type was non-optional, it would be encoded as an NSNull.</li></ul></ul><div class=""><br class=""></div><div class="">That sounds complicated, but the runtime already has machinery to coerce Optionals to Objective-C id: Optional.some gets bridged as the<i class=""> </i>Wrapped value, while Optional.none gets bridged as either NSNull or _SwiftNull, which contains a depth. We would simply need to make _SwiftNull conform to Codable, and give it a decoding implementation which was clever enough to realize when it was being asked to decode a different type.</div></div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">What about a more complex enum, like the standard library's `UnicodeDecodingResult`:<br class="">
<br class="">
enum UnicodeDecodingResult {<br class="">
case emptyInput<br class="">
case error<br class="">
case scalarValue(UnicodeScalar)<br class="">
}<br class="">
<br class="">
Or, say, an `Error`-conforming type from one of my projects:<br class="">
<br class="">
public enum SQLError: Error {<br class="">
case connectionFailed(underlying: Error)<br class="">
case executionFailed(underlying: Error, statement: SQLStatement)<br class="">
case noRecordsFound(statement: SQLStatement)<br class="">
case extraRecordsFound(statement: SQLStatement)<br class="">
case columnInvalid(underlying: Error, key: ColumnSpecifier, statement: SQLStatement)<br class="">
case valueInvalid(underlying: Error, key: AnySQLColumnKey, statement: SQLStatement)<br class="">
}<br class="">
<br class="">
(You can assume that all the types in the associated values are `Codable`.)</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">Sure — these cases specifically do not derive <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Codable</code> conformance because the specific representation to choose is up to you. Two possible ways to write this, though there are many others (I’m simplifying these cases here a bit, but you can extrapolate this):</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote><div class="">Okay, so tl;dr is "There's nothing special to help with this; just encode some indication of the case in one key, and the associated values in separate keys". I suppose that works.</div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">Have you given any consideration to supporting types which only need to decode? That seems likely to be common when interacting with web services.</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">We have. Ultimately, we decided that the introduction of several protocols to cover encodability, decodability, and both was too much of a cognitive overhead, considering the number of other types we’re also introducing. You can always implement <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(to:)</code> as <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">fatalError()</code>.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote><div class="">I understand that impulse.</div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""><p dir="auto" class="">Structured types (i.e. types which encode as a collection of properties) encode and decode their properties in a keyed manner. Keys may be String-convertible or Int-convertible (or both),</p>
</blockquote><p dir="auto" class="">What does "may" mean here? That, at runtime, the encoder will test for the preferred key type and fall back to the other one? That seems a little bit problematic.</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">Yes, this is the case. A lot is left up to the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> because it can choose to do something for its format that your implementation of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(to:)</code> may not have considered.<br class="">
If you try to encode something with an <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int</code> key in a string-keyed dictionary, the encoder may choose to stringify the integer if appropriate for the format. If not, it can reject your key, ignore the call altogether, <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">preconditionFailure()</code>, etc. It is also perfectly legitimate to write an <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> which supports a flat encoding format — in that case, keys are likely ignored altogether, in which case there is no error to be had. We’d like to not arbitrarily constrain an implementation unless necessary.</p></div></div></div></blockquote>Wait, what? If it's ignoring the keys altogether, how does it know what to decode with each call? Do you have to decode in the same order you encoded?</div><div class=""><br class=""></div><div class="">(Or are you saying that the encoder might use the keys to match fields to, say, predefined fields in a schema provided to the encoder, but not actually write anything about the keys to disk? That would make sense. But ignoring the keys altogether doesn't.)</div><div class=""><br class=""></div><div class="">In general, my biggest concern with this design is that, in a hundred different places, it is very loosely specified. We have keyed containers, but the keys can convert to either, or both, or neither of two different types. We have encode and decode halves, but you only <i class="">have</i> to support one or the other. Nils are supported, but they're interpreted as equivalent to the absence of a value. If something encounters a problem or incompatibility, it should throw an error or trip a precondition.</div><div class=""><br class=""></div><div class="">I worry that this is so loosely specified that you can't really trust an arbitrary type to work with an arbitrary encoder; you'll just have to hope that your testing touches every variation on every part of the object graph.</div><div class=""><br class=""></div><div class="">This kind of design is commonplace in Objective-C, but Swift developers often go to great lengths to expose these kinds of requirements to the type system so the compiler can verify them. For instance, I would expect a similar Swift framework to explicitly model the raw values of keys as part of the type system; if you tried to use a type providing string keys with an encoder that required integer keys, the compiler would reject your code. Even when something can't be explicitly modeled by the type system, Swift developers usually try to document guarantees about how to use APIs safely; for instance, Swift.RangeReplaceableCollection explicitly states that its calls may make indices retrieved before the call invalid, and individual conforming types document specific rules about which indices will keep working and which won't.</div><div class=""><br class=""></div><div class="">But your Encoder and Decoder designs seem to document semantics very loosely; they don't formally model very important properties, like "Does this coder preserve object identities*?" and "What sorts of keys does this coder use?", even when it's easy to do so, and now it seems like they also don't specify important semantics, like whether or not the encoder is required to inspect the key to determine the value you're looking for, either. I'm very concerned by that.</div><div class=""><br class=""></div><div class="">The design you propose takes advantage of several Swift niceties—Optional value types, enums for keys, etc.—and I really appreciate those things. But in its relatively casual attitude towards typing, it still feels like an Objective-C design being ported to Swift. I want to encourage you to go beyond that.</div></div></div></blockquote><div><br class=""></div><div>+1. I share all of Brent’s concerns. I think we can tighten up the design and make it harder to use incorrectly without making it more difficult to use correctly.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><i class="">* That is, if you encode a reference to the same object twice and then decode the result, do you get one instance with two references, or two instances with one reference each? JSONEncoder can't provide that behavior, but NSKeyedArchiver can. There's no way for a type which won't encode properly without this property to reject encoders which cannot guarantee it.</i></div><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><p dir="auto" class="">For these exact reasons, integer keys are not produced by code synthesis, only string keys. If you want integer keys, you’ll have to write them yourself. :)</p></div></div></div></blockquote><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><div class="">That's another thing I realized on a later reading and forgot to correct. Sorry about that.</div><div class=""><br class=""></div><div class="">(On the other hand, that reminds me of another minor concern: Your statement that superContainer() instances use a key with the integer value 0. I'd suggest you document that fact in boldface in the documentation for integer keys, because I expect that every developer who uses integer keys will want to start at key 0.)</div></div></div></div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">So I would suggest the following changes:<br class="">
<br class="">
* The coding key always converts to a string. That means we can eliminate the `CodingKey` protocol and instead use `RawRepresentable where RawValue == String`, leveraging existing infrastructure. That also means we can call the `CodingKeys` associated type `CodingKey` instead, which is the correct name for it—we're not talking about an `OptionSet` here.<br class="">
<br class="">
* If, to save space on disk, you want to also people to use integers as the serialized representation of a key, we might introduce a parallel `IntegerCodingKey` protocol for that, but every `CodingKey` type should map to `String` first and foremost. Using a protocol here ensures that it can be statically determined at compile time whether a type can be encoded with integer keys, so the compiler can select an overload of `container(keyedBy:)`.<br class="">
<br class="">
* Intrinsically ordered data is encoded as a single value containers of type `Array<Codable>`. (I considered having an `orderedContainer()` method and type, but as I thought about it, I couldn't think of an advantage it would have over `Array`.)</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">This is possible, but I don’t see this as necessarily advantageous over what we currently have. In 99.9% of cases, <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">CodingKey</code> types will have string values anyway — in many cases you won’t have to write the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">enum</code> yourself to begin with, but even when you do, derived <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">CodingKey</code> conformance will generate string values on your behalf.<br class="">
The only time a key will not have a string value is if the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">CodingKey</code> protocol is implemented manually and a value is either deliberately left out, or there was a mistake in the implementation; in either case, there wouldn’t have been a valid string value anyway.</p></div></div></div></blockquote>Again, I think this might come down to an Objective-C vs. Swift mindset difference. The Objective-C mindset is often "very few people will do X, so we might as well allow it". The Swift mindset is more "very few people will do X, so we might as well forbid it". :^)</div><div class=""><br class=""></div><div class="">In this case: Very few people will be inconvenienced by a requirement that they provide strings in their CodingKeys, so why not require it? Doing so ensures that encoders can always rely on a string key being available, and with all the magic we're providing to ensure the compiler fills in the actual strings for you, users will not find the requirement burdensome.<br class=""></div></div></div></blockquote><div><br class=""></div><div>+1</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""><p dir="auto" class=""> /// Returns an encoding container appropriate for holding a single primitive value.<br class="">
///<br class="">
/// - returns: A new empty single value container.<br class="">
/// - precondition: May not be called after a prior `self.container(keyedBy:)` call.<br class="">
/// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.<br class="">
func singleValueContainer() -> SingleValueEncodingContainer</p>
</blockquote><p dir="auto" class="">Speaking of which, I'm not sure about single value containers. My first instinct is to say that methods should be moved from them to the `Encoder` directly, but that would probably cause code duplication. But...isn't there already duplication between the `SingleValue*Container` and the `Keyed*Container`? Why, yes, yes there is. So let's talk about that.</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">In the Alternatives Considered section of the proposal, we detail having done just this. Originally, the requirements now on <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">SingleValueContainer</code> sat on <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Decoder</code>.<br class="">
Unfortunately, this made it too easy to do the wrong thing, and required extra work (in comparison) to do the right thing.</p><p dir="auto" class="">When <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> has <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_ value: Bool?)</code>, <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_ value: Int?)</code>, etc. on it, it’s very intuitive to try to encode values that way:</p>
<pre style="background-color: rgb(247, 247, 247); border-top-left-radius: 5px; border-top-right-radius: 5px; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; margin-left: 15px; margin-right: 15px; max-width: 90vw; overflow-x: auto; padding: 5px;" bgcolor="#F7F7F7" class=""><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0" bgcolor="#F7F7F7" class=""><span style="color: #008800; font-weight: bold" class="">func</span> <span style="color: #0066BB; font-weight: bold" class="">encode</span>(to encoder: Encoder) throws {
<span style="color: #888888" class="">// The very first thing I try to type is encoder.enc… and guess what pops up in autocomplete:</span>
try encoder.encode(myName)
try encoder.encode(myEmail)
try encoder.encode(myAddress)
}
</code></pre><p dir="auto" class="">This might look right to someone expecting to be able to encode in an ordered fashion, which is <em class="">not</em> what these methods do.<br class="">
In addition, for someone expecting keyed encoding methods, this is very confusing. Where are those methods? Where don’t these "default" methods have keys?</p><p dir="auto" class="">The very first time that code block ran, it would <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">preconditionFailure()</code> or throw an error, since those methods intend to encode only one single value.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote>That's true. But this is mitigated by the fact that the mistake is self-correcting—it will definitely cause a precondition to fail the first time you make it.</div><div class=""><br class=""></div><div class="">However, I do agree that it's not really a good idea. I'm more interested in the second suggestion I had, having the Keyed*Container return a SingleValue*Container.</div><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><p dir="auto" class="">The return type of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">decode(Int.self, forKey: .id)</code> is <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int</code>. I’m not convinced that it’s possible to misconstrue that as the correct thing to do here. How would that return a <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">nil</code> value if the value was <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">nil</code> to begin with?<br class=""></p></div></div></div></blockquote>I think people will generally assume that they're going to get out the value they put in, and will be surprised that something encode(_:) accepts will cause decode(_:) to error out. I do agree that the type passed to `decode(_:forKey_:)` will make it <i class="">relatively</i> obvious what happened, but I think it'd be even better to just preserve the user's types.<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">I think we'd be better off having `encode(_:forKey:)` not take an optional; instead, we should have `Optional` conform to `Codable` and behave in some appropriate way. Exactly how to implement it might be a little tricky because of nested optionals; I suppose a `none` would have to measure how many levels of optionality there are between it and a concrete value, and then encode that information into the data. I think our `NSNull` bridging is doing something broadly similar right now.</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class=""><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Optional</code> cannot encode to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Codable</code> for the reasons given above. It is a primitive type much like <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">String</code>, and it’s up to the encoder and the format to represent it.<br class="">
How would <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Optional</code> encode <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">nil</code>?</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote>I discussed this above: Treat null-ness as a primitive value with its own encode() call and do something clever for nested Optionals.<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">It's so simple, it doesn't even need to be specialized. You might even be able to get away with combining the encoding and decoding variants if the subscript comes from a conditional extension. `Value*Container` *does* need to be specialized; it looks like this (modulo the `Optional` issue I mentioned above):</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">Sure, let’s go with this for a moment. Presumably, then, <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> would be able to vend out both <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">KeyedEncodingContainer</code>s and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">ValueEncodingContainer</code>s, correct?</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote>Yes.<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">public protocol ValueEncodingContainer {<br class="">
func encode<Value : Codable>(_ value: Value?, forKey key: Key) throws</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">I’m assuming that the key here is a typo, correct?<br class=""></p></div></div></div></blockquote>Yes, sorry. I removed the forKey: label from the other calls, but not this one. (I almost left it on all of the calls, which would have been <i class="">really</i> confusing!)<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class="">
Keep in mind that combining these concepts changes the semantics of how single-value encoding works. Right now <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">SingleValueEncodingContainer</code> only allows values of primitive types; this would allow you to encode a value in terms of a different arbitrarily-codable value.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote>Yes. I don't really see that as a problem; if you ask `Foo` to encode itself, and it only wants to encode a `Bar`, is anything really gained by insisting that it add a level of nesting first? More concretely: If you're encoding an enum with a `rawValue`, why not just encode the `rawValue`?<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class=""> var codingKeyContext: [CodingKey]<br class="">
}<br class="">
<br class="">
And use sites would look like:<br class="">
<br class="">
func encode(to encoder: Encoder) throws {<br class="">
let container = encoder.container(keyedBy: CodingKey.self)<br class="">
try container[.id].encode(id)<br class="">
try container[.name].encode(name)<br class="">
try container[.birthDate].encode(birthDate)<br class="">
}</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">For consumers, this doesn’t seem to make much of a difference. We’ve turned <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">try container.encode(id, forKey:. id)</code> into <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">try container[.id].encode(id)</code>.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote><div class="">It isn't terribly different for consumers, although the subscript <i class="">is</i> slightly less wordy. But it means that encoders/decoders only provide one—not two—sets of encoding/decoding calls, and it allows some small bits of cleverness, like passing a SingleValue*Container off to a piece of code that's supposed to handle it.</div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><p dir="auto" class="">These types were chosen because we want the API to make static guarantees about concrete types which all <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code>s and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Decoder</code>s should support. This is somewhat less relevant for JSON, but more relevant for binary formats where the difference between <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int16</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int64</code> is critical.</p></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class="">This turns the concrete type check into a runtime check that <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Encoder</code> authors need to keep in mind. More so, however, any type can conform to <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">SignedInteger</code> or <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">UnsignedInteger</code> as long as it fulfills the protocol requirements. I can write an <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Int37</code> type, but no encoder could make sense of that type, and that failure is a runtime failure. If you want a concrete example, <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Float80</code> conforms to <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">FloatingPoint</code>; no popular binary format I’ve seen supports 80-bit floats, though — we cannot prevent that call statically…</p></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class="">Instead, we want to offer a static, concrete list of types that <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Encoder</code>s and <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Decoder</code>s must be aware of, and that consumers have guarantees about support for.</p></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote>But this way instead forces encoders to treat a whole bunch of types as "primitive" which, to those encoders, aren't primitive at all.</div><div class=""><br class=""></div><div class="">Maybe it's just that we have different priorities here, but in general, I want an archiving system that (within reason) handles whatever types I throw at it, if necessary by augmenting the underlying encoder format with default Foundation-provided behavior. If a format only supports 64-bit ints and I throw a 128-bit int at it, I don't want it to truncate it or throw up its hands; I want it to read the two's-compliment contents of the `BinaryInteger.words` property, convert it to a `Data` in some standard endianness, and write that out. Or convert to a human-readable `String` and use that. It doesn't matter a whole lot, as long as it does something it can undo later.</div><div class=""><br class=""></div><div class="">I also like that a system with very few primitives essentially makes no assumptions about what a format will need to customize. A low-level binary format cares a lot about different integer sizes, but a higher-level one probably cares more about dates, URLs, and dictionaries. For instance, I suspect (hope?) that the JSONEncoder is going to hook Array and Dictionary to make them form JSON arrays and objects, not the sort of key-based representation NSKeyedArchiver uses (if I recall correctly). If we just provide, for instance, these:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(_ value: String) throws</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(_ value: NSNull) throws</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(_ value: Codable) throws</div><div class=""><br class=""></div><div class="">Then there's exactly one path to customization—test for types in `encode(_: Codable)`—and everyone will use it. If you have some gigantic set of primitives, many coders will end up being filled with boilerplate to funnel ten integer types into one or two implementations, and nobody will be really happy with the available set.</div><div class=""><br class=""></div><div class="">In reality, you'll probably need a few more than just these three, particularly since BinaryInteger and FloatingPoint both have associated values, so several <i class="">very</i> important features (like their `bitWidth` and `isSigned` properties) can only be accessed through a separate primitive. But the need for a few doesn't imply that we need a big mess of them, particularly when the difference is only relevant to one particular class of encoders.</div><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">To accommodate my previous suggestion of using arrays to represent ordered encoded data, I would add one more primitive:<br class="">
<br class="">
func encode(_ values: [Codable]) throws</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">Collection types are purposefully not primitives here:</p>
<ul class="">
<li class="">If <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Array</code> is a primitive, but does not conform to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Codable</code>, then you cannot encode <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Array<Array<Codable>></code>.</li>
<li class="">If <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Array</code> is a primitive, and conforms to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Codable</code>, then there may be ambiguity between <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_ values: [Codable])</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_ value: Codable)</code>.
<ul class="">
<li class="">Even in cases where there are not, inside of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode(_ values: [Codable])</code>, if I call <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode([[1,2],[3,4]])</code>, you’ve lost type information about what’s contained in the array — all you see is <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Codable</code></li>
<li class="">If you change it to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode<Value : Codable>(_ values: [Value])</code> to compensate for that, you still cannot infinitely recurse on what type <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Value</code> is. Try it with <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">encode([[[[1]]]])</code> and you’ll see what I mean; at some point the inner types are no longer preserved.</li>
</ul></li>
</ul><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote><div class="">Hmm, I suppose you're right.</div><div class=""><br class=""></div><div class="">Alternative design: In addition to KeyedContainers, you also have OrderedContainers. Like my proposed behavior for KeyedContainers, these merely vend SingleValue*Containers—in this case as an Array-like Collection.</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>extension MyList: Codable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode(to encoder: Encoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let container = encoder.orderedContainer(self.count)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>for (valueContainer, elem) in zip(container, self) {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>try valueContainer.encode(elem)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>init(from decoder: Decoder) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let container = decoder.orderedContainer()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>self.init(try container.map { try $0.decode(Element.self) })</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">This helps us draw an important distinction between keyed and ordered containers. KeyedContainers locate a value based on the key. Perhaps the way in which it's based on the key is that it extracts an integer from the key and then finds the matching location in a list of values, but then that's just how keys are matched to values in that format. OrderedContainers, on the other hand, are contiguous, variable-length, and have an intrinsic order to them. If you're handed an OrderedContainer, you are meant to be able to enumerate its contents; a KeyedContainer is more opaque than that.</div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">(Also, is there any sense in adding `Date` to this set, since it needs special treatment in many of our formats?)</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">We’ve considered adding <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Date</code> to this list. However, this means that any format that is a part of this system needs to be able to make a decision about how to format dates. Many binary formats have no native representations of dates, so this is not necessarily a guarantee that all formats can make.</p><p dir="auto" class="">Looking for additional opinions on this one.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote><div class="">I think that, if you're taking the view that you want to provide a set of pre-specified primitive methods as a list of things you want encoders to make a policy decision about, Date is a good candidate. But as I said earlier, I'd prefer to radically reduce the set of primitives, not add to it.</div><div class=""><br class=""></div><div class="">IIUC, two of your three proposed, Foundation-provided coders need to do something special with dates; perhaps one of the three needs to do something special with different integer sizes and types. Think of that as a message about your problem domain.</div><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><p dir="auto" class="">I see what you're getting at here, but I don't think this is fit for purpose, because arrays are not simply dictionaries with integer keys—their elements are adjacent and ordered. See my discussion earlier about treating inherently ordered containers as simply single-value `Array`s.</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">You’re right in that arrays are not simply dictionaries with integer keys, but I don’t see where we make that assertion here.<br class=""></p></div></div></div></blockquote>Well, because you're doing all this with a <i class="">keyed</i> container. That sort of implies that the elements are stored and looked up by key.</div><div class=""><br class=""></div><div class="">Suppose you want to write <i class="">n</i> elements into a KeyedEncodingContainer. You need a different key for each element, but you don't know ahead of time how many elements there are. So I guess you'll need to introduce a custom key type for no particular reason:</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>struct /* wat */ IndexCodingKeys: CodingKey {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var index: Int</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>init(stringValue: String?, intValue: Int) throws {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>guard let i = intValue ?? Int(stringValue) else {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>throw …</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>index = i</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var stringValue: String? {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>return String(index)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var intValue: Int? {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>return index</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class="">And then you write them all into keyed slots? And on the way back in, you inspect `allKeys` (assuming it's filled in at all, since you keep saying that coders don't necessarily have to use the keys), and use that to figure out the available elements, and decode them?</div><div class=""><br class=""></div><div class="">I'm just not sure I understand how this is supposed to work reliably when you combine arbitrary coders and arbitrary types.<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class="">The way these containers are handled is completely up to the <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Encoder</code>. An <code bgcolor="#F7F7F7" class="" style="background-color: rgb(247, 247, 247); border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; margin: 0px; padding: 0px 0.4em;">Encoder</code> producing an array may choose to ignore keys altogether and simply produce an array from the values given to it sequentially. (This is not recommended, but possible.)</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote><div class="">Again, as I said earlier, this idea that a keyed encoder could just ignore the keys entirely is very strange and worrying to me. It sounds like a keyed container has no dependable semantics at all.</div><div class=""><br class=""></div>There's preserving implementation flexibility, and then there's being so vague about behavior that nothing has any meaning and you can't reliably use anything. I'm very worried that, in some places, this design leans towards the latter. A keyed container might not <i class="">write</i> the keys anywhere in the file, but it certainly ought to <i class="">use</i> them to determine which field you're looking for. If it doesn't—if the key is just a suggestion—then all this API provides is a naming convention for methods that do vaguely similar things, potentially in totally incompatible ways.<br class=""></div></div></div></blockquote><div><br class=""></div><div>+1</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class="">
</blockquote><p dir="auto" class="">This comes very close to—but doesn't quite—address something else I'm concerned about. What's the preferred way to handle differences in serialization to different formats?<br class="">
<br class="">
Here's what I mean: Suppose I have a BlogPost model, and I can both fetch and post BlogPosts to a cross-platform web service, and store them locally. But when I fetch and post remotely, I ned to conform to the web service's formats; when I store an instance locally, I have a freer hand in designing my storage, and perhaps need to store some extra metadata. How do you imagine handling that sort of situation? Is the answer simply that I should use two different types?</p>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">This is a valid concern, and one that should likely be addressed.</p><p dir="auto" class="">Perhaps the solution is to offer a <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">userInfo : [UserInfoKey : Any]</code> (<code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">UserInfoKey</code> being a <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">String</code>-<code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">RawRepresentable</code> struct or similar) on <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Decoder</code> set at the top-level to allow passing this type of contextual information from the top level down.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote>At a broad level, that's a good idea. But why not provide something more precise than a bag of `Any`s here? You're in pure Swift; you have that flexibility.</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>protocol Codable {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>associatedtype CodingContext = ()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>init<Coder: Decoder>(from decoder: Coder, with context: CodingContext) throws</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encoder<Coder: Encoder>(from encoder: Coder, with context: CodingContext) throws</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>protocol Encoder {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>associatedtype CodingContext = ()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key, CodingContext></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>class KeyedEncodingContainer<Key: CodingKey, CodingContext> {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode<Value: Codable>(_ value: Value,? forKey key: Key, with context: Value.CodingContext) throws { … }</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>// Shorthand when contexts are the same:</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>func encode<Value: Codable>(_ value: Value,? forKey key: Key) throws</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>where Value.CodingContext == CodingContext</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>{ … }</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>…</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div></div></div></blockquote><div><br class=""></div><div>This is sort of similar to the design I suggested for contexts. The difference is that you’re requiring all Codable to be context aware and by introducing an associated type you break the ability to use Codable as an existential. Many Codable conforming types won’t need to know anything about a context. I would still want to be able to encode them along with my custom context-aware types. A good example is types from Foundation that will conform to Codable. They will definitely not know anything about my context but I still want to be able to encode a URL alongside my custom context-aware types.</div><div><br class=""></div><div>Did you take a look at the design I suggested? What do you think of it?</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class="">
<div style="white-space:normal" class=""><p dir="auto" class="">We don’t support this type of polymorphic decoding. Because no type information is written into the payload (there’s no safe way to do this that is not currently brittle), there’s no way to tell what’s in there prior to decoding it (and there wouldn’t be a reasonable way to trust what’s in the payload to begin with).<br class="">
We’ve thought through this a lot, but in the end we’re willing to make this tradeoff for security primarily, and simplicity secondarily.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""></blockquote></blockquote></div></div></div></blockquote>Well, `String(reflecting: typeInstance)` will give you the fully-qualified type name, so you can certainly <i class="">write</i> it. (If you're worried about `debugDescription` on types changing, I'm sure we can provide something, either public or as SPI, that won't.) You can't <i class="">read</i> it and convert it back to a type instance, but you can read it and match it against the type provided, including by walking into superContainer()s and finding the one corresponding to the type instance the user passed. Or you could call a type method on the provided type and ask it for a subtype instance to use for initialization, forming a sort of class cluster. Or, as a safety measure, you can throw if there's a class name mismatch.</div><div class=""><br class=""></div><div class="">(Maybe it'd be better to write out and check the key type, rather than the instance type. Hmm.)</div><div class=""><br class=""></div><div class="">Obviously not every encoder will want to write out types—I wouldn't expect JSONEncoder to do it, except perhaps with some sort of off-by-default option—but I think it could be very useful if added.</div><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999" class=""><p dir="auto" class=""><span style="color: rgb(119, 119, 119);" class="">How important is this performance? If the answer is "eh, not really that much", I could imagine a setup where every "primitive" type eventually represents itself as `String` or `Data`, and each `Encoder`/`Decoder` can use dynamic type checks in `encode(_:)`/`decode(_:)` to define whatever "primitives" it wants for its own format.</span></p></blockquote>
</blockquote></div>
<div style="white-space:normal" class=""><p dir="auto" class="">Does this imply that <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Int32</code> should decide how it’s represented as <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Data</code>? What if an encoder forgets to implement that?<br class=""></p></div></div></div></blockquote>Yes, Int32 decides how—if the encoder doesn't do anything special to represent integers—it should be represented in terms of a more immediately serializable type like Data. If an encoder forgets to provide a special representation for Int32, then it falls back to a sensible, Foundation-provided default. If the encoder author later realizes their mistake and wants to correct the encoder, they'd probably better build backwards compatibility into the decoder.<br class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><p dir="auto" class="">
Again, we want to provide a static list of types that <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7" class="">Encoder</code>s know they <em class="">must</em> handle, and thus, consumers have <em class="">guarantees</em> that those types are supported.</p><div class=""></div></div><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class=""></blockquote></div></div></div></blockquote>I <i class="">do</i> think that consumers are guaranteed these types are supported: Even if the encoder doesn't do anything special, Foundation will write them out as simpler and simpler types until, sooner or later, you get to something that <i class="">is</i> supported, like Data or String. This is arguably a stronger level of guarantee than we have when there are a bunch of primitive types, because if an encoder author feels like nobody's going to actually use UInt8 when it's a primitive, the natural thing to do is to throw or trap. If the author feels the same way about UInt8 when it's <i class="">not</i> a primitive, then the natural thing to do is to let Foundation do what it does, which is write out UInt8 in terms of some other type.<br class=""></div></div></div></blockquote><div><br class=""></div><div>+1.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><div style="font-family:sans-serif" class=""><div style="white-space:normal" class=""><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px" class="">
</blockquote></div>
<div style="white-space:normal" class="">
</div>
</div>
</div>
</blockquote></div><br class=""><div class="">
<span class="Apple-style-span" style="border-collapse: separate; font-variant-ligatures: normal; font-variant-position: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; line-height: normal; border-spacing: 0px;"><div class=""><div style="font-size: 12px; " class="">-- </div><div style="font-size: 12px; " class="">Brent Royal-Gordon</div><div style="font-size: 12px; " class="">Architechies</div></div></span>
</div>
<br class=""></div>_______________________________________________<br class="">swift-evolution mailing list<br class=""><a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-evolution<br class=""></div></blockquote></div><br class=""></body></html>