<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8">
</head>
<body>
<div style="font-family:sans-serif"><div style="white-space:normal">
<p dir="auto">On 16 Mar 2017, at 1:00, Brent Royal-Gordon wrote:</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">On Mar 15, 2017, at 3:43 PM, Itai Ferber via swift-evolution <swift-evolution@swift.org> wrote:<br>
<br>
Hi everyone,<br>
This is a companion proposal to the Foundation Swift Archival & Serialization API. This introduces new encoders and decoders to be used as part of this system.<br>
The proposal is available online and inlined below.</p>
</blockquote><p dir="auto">Executive summary: I like where you're going with this, but I'm worried about flexibility.<br>
<br>
I'm not going to quote every bit of the JSON section because Apple Mail seems to destroy the formatting when I reply, but: I think you've identified several of the most important customization points (Date, Data, and illegal Floats). However, I think:<br>
<br>
* People may want to map illegal Floats to legal floating-point values (say, `greatestFiniteMagnitude`, `-greatestFiniteMagnitude`, and `0`) or map them to `null`s. They may also want different behavior for different things: imagine `(positiveInfinity: Double.greatestFiniteMagnitude, negativeInfinity: -Double.greatestFiniteMagnitude, nan: .throw)`.</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">I agree, this may be something that users could want.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">* Large integers are another big concern that you don't address. Because JSON only supports doubles, APIs that use 64-bit IDs often need them to be passed as strings, frequently with a different key ("id_str" instead of "id").</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">This is not true — JSON has no limitations on what numbers it can represent. 340282366920938463463374607431768211455 (2^128-1) is a perfectly legitimate number in JSON, though you may have a hard reading it in on some platforms. <em>Javascript</em> numbers are IEEE 754 doubles, but that’s a Javascript problem, not a JSON problem.</p>
<p dir="auto">If what you mean here is that some large numbers should be encoded as strings instead of integers for the benefit of the other side reading it in a valid way, then perhaps.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">* For that matter, style and capitalization are a problem. JSON style varies, but it *tends* to be snake_case, where Cocoa favors camelCase. You can address this at the CodingKey level by manually specifying string equivalents of all the coding keys, but that's kind of a pain, and it affects all of your code and all of your serializations.<br>
<br>
I'm sorely tempted to suggest that we give the JSON encoder and decoder a delegate:<br>
<br>
public protocol JSONCodingDelegate {<br>
/// Returns the string name to be used when encoding or decoding the given CodingKey as JSON.<br>
///<br>
/// - Returns: The string to use, or `nil` for the default.<br>
func jsonName(for key: CodingKey, at keyPath: [CodingKey], in encoderOrDecoder: AnyObject) throws -> String?<br>
<br>
// These are used when encoding/decoding any of the integer types.<br>
func jsonValue(from integer: Int64, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?<br>
func integer(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Int64?<br>
<br>
// These are used when encoding/decoding any of the floating-point types.<br>
func jsonValue(from number: Double, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?<br>
func number(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Double?<br>
<br>
// These are used when encoding/decoding Date.<br>
func jsonValue(from date: Date, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?<br>
func date(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Date?<br>
<br>
// These are used when encoding/decoding Data.<br>
func jsonValue(from data: Data, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?<br>
func data(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Data?<br>
<br>
func jsonValue(from double: Double, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?<br>
func integer(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Double?<br>
}<br>
public enum JSONValue {<br>
case string(String)<br>
case number(Double)<br>
case bool(Bool)<br>
case object([String: JSONValue])<br>
case array([JSONValue])<br>
case null<br>
}</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">I disagree with generalizing this to the point of being on a delegate. This is all work that you could be doing in <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">encode(to:)</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">decode(from:)</code>. In <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">encode(to:)</code>, it’s always possible to clamp an invalid floating-point number to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Double.greatestFiniteMagnitude</code>, and always possible to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">encode("\(id)", forKey: .id)</code> instead of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">encode(id, forKey: .id)</code>.</p>
<p dir="auto">The options that we have on <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">JSONEncoder</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">JSONDecoder</code> straddle a fine line between being pedantically correct (and refusing to break encapsulation for encoded types), and being pragmatically useful. In theory, it certainly feels "wrong" that we would allow someone to change the way in which a <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Date</code> is encoded, or how <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Double</code>s are represented; in a pragmatic sense, though, JSON has no native representation of such <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Double</code> values, or a standardized representation of dates, and it’s useful to provide options to for controlling that.</p>
<p dir="auto">However, allowing a delegate to intercept all such calls feels like it leans too much in the wrong direction. We’d like to offer as limited a set of knobs as possible while still being useful.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">Or, perhaps, that a more general form of this delegate be available on all encoders and decoders. But that may be overkill, and even if it *is* a good idea, it's one we can add later.<br>
</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">Property List<br>
<br>
We also intend to support the property list format, with PropertyListEncoder and PropertyListDecoder:</p>
</blockquote><p dir="auto">No complaints here.<br>
</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">Foundation-Provided Errors<br>
<br>
Along with providing the above encoders and decoders, we would like to promote the use of a common set of error codes and messages across all new encoders and decoders. A common vocabulary of expected errors allows end-users to write code agnostic about the specific encoder/decoder implementation they are working with, whether first-party or third-party:<br>
<br>
extension CocoaError.Code {<br>
/// Thrown when a value incompatible with the output format is encoded.<br>
public static var coderInvalidValue: CocoaError.Code<br>
<br>
/// Thrown when a value of a given type is requested but the encountered value is of an incompatible type.<br>
public static var coderTypeMismatch: CocoaError.Code<br>
<br>
/// Thrown when read data is corrupted or otherwise invalid for the format. This value already exists today.<br>
public static var coderReadCorrupt: CocoaError.Code<br>
<br>
/// Thrown when a requested key or value is unexpectedly null or missing. This value already exists today.<br>
public static var coderValueNotFound: CocoaError.Code<br>
}</p>
</blockquote><p dir="auto">[snip]<br>
</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">All of these errors will include the coding key path that led to the failure in the error's userInfo dictionary under NSCodingKeyContextErrorKey, along with a non-localized, developer-facing failure reason under NSDebugDescriptionErrorKey.</p>
</blockquote><p dir="auto">Now comes the part where I whine like a four-year-old:<br>
<br>
"Do we haaaaaaaave to use the `userInfo` dictionary, papa?"<br>
<br>
An enum with an associated value would be a much more natural way to express these errors and the data that comes with them. Failing that, at least give us some convenience properties. The untyped bag of stuff in the `userInfo` dictionary fills developers who spend all their time in Swift with fear and loathing.<br>
<br>
Actually, if you wanted to help us out with the "untyped bag of stuff" problem in general, I for one wouldn't say "no":<br>
<br>
public struct TypedKey<Key: Hashable, Value> {<br>
public var key: Key<br>
public init(key: Key, valueType: Value.Type) {<br>
self.key = key<br>
}<br>
}<br>
extension Dictionary where Value == Any {<br>
public subscript<CastedValue>(typedKey: TypedKey<Key, CastedValue>) -> CastedValue? {<br>
get {<br>
return self[typedKey.key] as? CastedValue<br>
}<br>
set {<br>
self[typedKey.key] = newValue<br>
}<br>
}<br>
}</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">As explained in a different email, <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSError</code> is Foundation’s common currency for errors, and we are not looking to change that as part of this proposal. If we were to add a <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">userInfo</code> dictionary to <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Encoder</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Decoder</code> mentioned in my other email to you, it would likely take on more of the form that you suggest here.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">NSKeyedArchiver & NSKeyedUnarchiver Changes<br>
<br>
Although our primary objectives for this new API revolve around Swift, we would like to make it easy for current consumers to make the transition to Codable where appropriate. As part of this, we would like to bridge compatibility between new Codabletypes (or newly-Codable-adopting types) and existing NSCoding types.<br>
<br>
To do this, we want to introduce changes to NSKeyedArchiver and NSKeyedUnarchiver in Swift that allow archival of Codable types intermixed with NSCoding types:<br>
<br>
// These are provided in the Swift overlay, and included in swift-corelibs-foundation.<br>
extension NSKeyedArchiver {<br>
public func encodeCodable(_ codable: Codable?, forKey key: String) { ... }<br>
}<br>
<br>
extension NSKeyedUnarchiver {<br>
public func decodeCodable<T : Codable>(_ type: T.Type, forKey key: String) -> T? { ... }<br>
}<br>
<br>
NOTE: Since these changes are being made in extensions in the Swift overlay, it is not yet possible for these methods to be overridden. These can therefore not be added to NSCoder, since NSKeyedArchiver and NSKeyedUnarchiver would not be able to provide concrete implementations. In order to call these methods, it is necessary to downcast from an NSCoder to NSKeyedArchiver/NSKeyedUnarchiver directly. Since subclasses of NSKeyedArchiver and NSKeyedUnarchiver in Swift will inherit these implementations without being able to override them (which is wrong), we will NSRequiresConcreteImplementation() dynamically in subclasses.<br>
<br>
The addition of these methods allows the introduction of Codable types into existing NSCoding structures, allowing for a transition to Codable types where appropriate.</p>
</blockquote><p dir="auto">I wonder about this.<br>
<br>
Could `NSCoding` be imported in Swift as refining `Codable`? Then we could all just forget `NSCoding` exists, other than that certain types are less likely to properly handle being put into a JSONEncoder/Decoder. (Which, to tell the truth, is probably inevitable here; the Encoder and Decoder types look like they're probably too loosely defined to truly guarantee that you can mix-and-match coders and types without occasional problems.)</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">Likely not — <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSCoding</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code> don’t support the same features:</p>
<ul>
<li><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSCoding</code> implementations currently write type information into produced archives. Off the top of my head, I don’t think this is a strict necessity of the API, but a <em>lot</em> of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSCoding</code> implementations rely on this</li>
<li><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code> requires type information on decode; <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSCoding</code> does not (because of the aforementioned type information in archives). We cannot translate decodes properly</li>
<li><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">NSCoding</code> has different primitive types than <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code></li>
</ul>
<p dir="auto">They don’t translate 1-to-1, and would have to stay completely distinct.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">Semantics of Codable Types in Archives<br>
<br>
There are a few things to note about including Codable values in NSKeyedArchiverarchives:<br>
<br>
• Bridgeable Foundation types will always bridge before encoding. This is to facilitate writing Foundation types in a compatible format from both Objective-C and Swift<br>
• On decode, these types will decode either as their Objective-C or Swift version, depending on user need (decodeObject(forKey:) will decode as an Objective-C object; decodeCodable(_:forKey:) as a Swift value)</p>
</blockquote><p dir="auto">This sounds sensible.<br>
</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto"> • User types, which are not bridgeable, do not write out a $class and can only be decoded in Swift. In the future, we may add API to allow Swift types to provide an Objective-C class to decode as, effectively allowing for user bridging across archival</p>
</blockquote><p dir="auto">Even pure Swift class types? I guess that's probably necessary since even our private ability to look up classes at runtime doesn't cover things like generics, but...ugh.</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">I’m not sure what you mean by this comment. If you have an old codebase which you’re converting from Objective-C to Swift but want to support writing archives from newer versions of the codebase still readable by old versions, it isn’t unreasonable to provide an Objective-C class to encode as or from…</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">Along with these, the Array, Dictionary, and Set types will gain Codableconformance (as part of the Conditional Conformance feature), and encode through NSKeyedArchiver as NSArray, NSDictionary, and NSSet respectively.</p>
</blockquote><p dir="auto">You might need to be careful here—we'll need to make sure that data structures of Swift types bridge properly. I suppose that means `_SwiftValue` will need to support `NSCoding` after all...</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">It shouldn’t have to. NSKeyedArchiver can make callbacks into Swift for the encoding of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">_SwiftValue</code>s it finds which did not bridge.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">-- <br>
Brent Royal-Gordon<br>
Architechies</p>
</blockquote></div>
<div style="white-space:normal">
</div>
</div>
</body>
</html>