[swift-evolution] [Proposal] Foundation Swift Encoders

[Brent Royal-Gordon]

> On Mar 15, 2017, at 3:43 PM, Itai Ferber via swift-evolution <swift-evolution at swift.org> wrote:
> 
> Hi everyone,
> This is a companion proposal to the Foundation Swift Archival & Serialization API. This introduces new encoders and decoders to be used as part of this system.
> The proposal is available online and inlined below.

Executive summary: I like where you're going with this, but I'm worried about flexibility.

I'm not going to quote every bit of the JSON section because Apple Mail seems to destroy the formatting when I reply, but: I think you've identified several of the most important customization points (Date, Data, and illegal Floats). However, I think:

* People may want to map illegal Floats to legal floating-point values (say, `greatestFiniteMagnitude`, `-greatestFiniteMagnitude`, and `0`) or map them to `null`s. They may also want different behavior for different things: imagine `(positiveInfinity: Double.greatestFiniteMagnitude, negativeInfinity: -Double.greatestFiniteMagnitude, nan: .throw)`.

* Large integers are another big concern that you don't address. Because JSON only supports doubles, APIs that use 64-bit IDs often need them to be passed as strings, frequently with a different key ("id_str" instead of "id").

* For that matter, style and capitalization are a problem. JSON style varies, but it *tends* to be snake_case, where Cocoa favors camelCase. You can address this at the CodingKey level by manually specifying string equivalents of all the coding keys, but that's kind of a pain, and it affects all of your code and all of your serializations.

I'm sorely tempted to suggest that we give the JSON encoder and decoder a delegate:

	public protocol JSONCodingDelegate {
		/// Returns the string name to be used when encoding or decoding the given CodingKey as JSON.
		/// 
		/// - Returns: The string to use, or `nil` for the default.
		func jsonName(for key: CodingKey, at keyPath: [CodingKey], in encoderOrDecoder: AnyObject) throws -> String?

		// These are used when encoding/decoding any of the integer types.
		func jsonValue(from integer: Int64, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?
		func integer(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Int64?
		
		// These are used when encoding/decoding any of the floating-point types.
		func jsonValue(from number: Double, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?
		func number(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Double?
		
		// These are used when encoding/decoding Date.
		func jsonValue(from date: Date, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?
		func date(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Date?
		
		// These are used when encoding/decoding Data.
		func jsonValue(from data: Data, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?
		func data(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Data?
		
		func jsonValue(from double: Double, at keyPath: [CodingKey], in encoder: JSONEncoder) throws -> JSONValue?
		func integer(from jsonValue: JSONValue, at keyPath: [CodingKey], in decoder: JSONDecoder) throws -> Double?
	}
	public enum JSONValue {
		case string(String)
		case number(Double)
		case bool(Bool)
		case object([String: JSONValue])
		case array([JSONValue])
		case null
	}

Or, perhaps, that a more general form of this delegate be available on all encoders and decoders. But that may be overkill, and even if it *is* a good idea, it's one we can add later.

> Property List
> 
> We also intend to support the property list format, with PropertyListEncoder and PropertyListDecoder:

No complaints here.

> Foundation-Provided Errors
> 
> Along with providing the above encoders and decoders, we would like to promote the use of a common set of error codes and messages across all new encoders and decoders. A common vocabulary of expected errors allows end-users to write code agnostic about the specific encoder/decoder implementation they are working with, whether first-party or third-party:
> 
> extension CocoaError.Code {
>     /// Thrown when a value incompatible with the output format is encoded.
>     public static var coderInvalidValue: CocoaError.Code
> 
>     /// Thrown when a value of a given type is requested but the encountered value is of an incompatible type.
>     public static var coderTypeMismatch: CocoaError.Code
> 
>     /// Thrown when read data is corrupted or otherwise invalid for the format. This value already exists today.
>     public static var coderReadCorrupt: CocoaError.Code
> 
>     /// Thrown when a requested key or value is unexpectedly null or missing. This value already exists today.
>     public static var coderValueNotFound: CocoaError.Code
> }

[snip]

> All of these errors will include the coding key path that led to the failure in the error's userInfo dictionary under NSCodingKeyContextErrorKey, along with a non-localized, developer-facing failure reason under NSDebugDescriptionErrorKey.

Now comes the part where I whine like a four-year-old:

"Do we haaaaaaaave to use the `userInfo` dictionary, papa?"

An enum with an associated value would be a much more natural way to express these errors and the data that comes with them. Failing that, at least give us some convenience properties. The untyped bag of stuff in the `userInfo` dictionary fills developers who spend all their time in Swift with fear and loathing.

Actually, if you wanted to help us out with the "untyped bag of stuff" problem in general, I for one wouldn't say "no":

	public struct TypedKey<Key: Hashable, Value> {
		public var key: Key
		public init(key: Key, valueType: Value.Type) {
			self.key = key
		}
	}
	extension Dictionary where Value == Any {
		public subscript<CastedValue>(typedKey: TypedKey<Key, CastedValue>) -> CastedValue? {
			get {
				return self[typedKey.key] as? CastedValue
			}
			set {
				self[typedKey.key] = newValue
			}
		}
	}

> NSKeyedArchiver & NSKeyedUnarchiver Changes
> 
> Although our primary objectives for this new API revolve around Swift, we would like to make it easy for current consumers to make the transition to Codable where appropriate. As part of this, we would like to bridge compatibility between new Codabletypes (or newly-Codable-adopting types) and existing NSCoding types.
> 
> To do this, we want to introduce changes to NSKeyedArchiver and NSKeyedUnarchiver in Swift that allow archival of Codable types intermixed with NSCoding types:
> 
> // These are provided in the Swift overlay, and included in swift-corelibs-foundation.
> extension NSKeyedArchiver {
>     public func encodeCodable(_ codable: Codable?, forKey key: String) { ... }
> }
> 
> extension NSKeyedUnarchiver {
>     public func decodeCodable<T : Codable>(_ type: T.Type, forKey key: String) -> T? { ... }
> }
> 
> NOTE: Since these changes are being made in extensions in the Swift overlay, it is not yet possible for these methods to be overridden. These can therefore not be added to NSCoder, since NSKeyedArchiver and NSKeyedUnarchiver would not be able to provide concrete implementations. In order to call these methods, it is necessary to downcast from an NSCoder to NSKeyedArchiver/NSKeyedUnarchiver directly. Since subclasses of NSKeyedArchiver and NSKeyedUnarchiver in Swift will inherit these implementations without being able to override them (which is wrong), we will NSRequiresConcreteImplementation() dynamically in subclasses.
> 
> The addition of these methods allows the introduction of Codable types into existing NSCoding structures, allowing for a transition to Codable types where appropriate.

I wonder about this.

Could `NSCoding` be imported in Swift as refining `Codable`? Then we could all just forget `NSCoding` exists, other than that certain types are less likely to properly handle being put into a JSONEncoder/Decoder. (Which, to tell the truth, is probably inevitable here; the Encoder and Decoder types look like they're probably too loosely defined to truly guarantee that you can mix-and-match coders and types without occasional problems.)

> Semantics of Codable Types in Archives
> 
> There are a few things to note about including Codable values in NSKeyedArchiverarchives:
> 
> 	• Bridgeable Foundation types will always bridge before encoding. This is to facilitate writing Foundation types in a compatible format from both Objective-C and Swift
> 		• On decode, these types will decode either as their Objective-C or Swift version, depending on user need (decodeObject(forKey:) will decode as an Objective-C object; decodeCodable(_:forKey:) as a Swift value)

This sounds sensible.

> 	• User types, which are not bridgeable, do not write out a $class and can only be decoded in Swift. In the future, we may add API to allow Swift types to provide an Objective-C class to decode as, effectively allowing for user bridging across archival

Even pure Swift class types? I guess that's probably necessary since even our private ability to look up classes at runtime doesn't cover things like generics, but...ugh.

> Along with these, the Array, Dictionary, and Set types will gain Codableconformance (as part of the Conditional Conformance feature), and encode through NSKeyedArchiver as NSArray, NSDictionary, and NSSet respectively.

You might need to be careful here—we'll need to make sure that data structures of Swift types bridge properly. I suppose that means `_SwiftValue` will need to support `NSCoding` after all...

-- 
Brent Royal-Gordon
Architechies