[swift-evolution] [Proposal] Foundation Swift Archival & Serialization

[Matthew Johnson]

This is a fantastic proposal!  I am very much looking forward to robust Swift-native encoding and decoding in Foundation.  The compiler synthesized conformances is especially great!    I want to thank everyone who worked on it.  It is clear that a lot of work went into the proposal.

The proposal covers a lot of ground so I’m breaking my comments up by topic in the order the occur in the proposal.

Encode / Decode only types:

Brent raised the question of decode only types.  Encode only types are also not uncommon when an API accepts an argument payload that gets serialized into the body of a request. The compiler synthesis feature in the proposal makes providing both encoding and decoding easy in common cases but this won’t always work as needed.

The obvious alternative is to have Decodable and Encodable protocols which Codable refines.  This would allow us to omit a conformance we don’t need when it can’t be synthesized.  

Your reply to Brent mentions using `fatalError` to avoid implementing the direction that isn't needed.  I think it would be better if the conformance can reflect what is actually supported by the type.  Requiring us to write `fatalError` as a stub for functionality we don’t need is a design problem IMO.  I don’t think the extra protocols are really that big a burden.  They don’t add any new functionality and are very easy to understand, especially considering the symmetry they would have with the other types you are introducing.

Coding Keys:

As others have mentioned, the design of this protocol does not require a value of a conforming type to actually be a valid key (it can return nil for both `intValue` and `stringValue`).  This seems problematic to me.

In the reply to Brent again you mention throwing and `preconditionFailure` as a way to handle incompatible keys.  This also seems problematic to me and feels like a design problem. If we really need to support more than one underlying key type and some encoders will reject some key types this information should be captured in the type system.  An encoder would only vend a keyed container for keys it actually supports.  Ideally the conformance of a type’s CodingKeys could be leveraged to produce a compiler error if an attempt was made to encode this type into an encoder that can’t support its keys.  In general, the idea is to produce static errors as close to the origin of the programming mistake as possible.  

I would very much prefer that we don’t defer to runtime assertions or thrown errors, etc for conditions that could be caught statically at compile time given the right design.  Other comments have indicated that static guarantees are important to the design (encoders *must* guarantee support of primitives specified by the protocols, etc).  Why is a static guarantee of compatible coding keys considered less important?

Keyed Containers:

Joe posted raised the topic of the alternative of using manual type erasure for the keyed containers rather than abstract classes.  Did you explore this direction at all?  It feels like a more natural approach for Swift and as Joe noted, it can be designed in such a way that eases migration to existentials in the future if that is the “ideal” design (which you mentioned in your response).

Decoding Containers:

returns: A value of the requested type, if present for the given key and convertible to the requested type.

Can you elaborate on the details of “convertible to the requested type” means?  It think this is an important detail for the proposal.  

For example, I would expect numeric values to attempt conversion using the SE-0080 failable numeric conversion initializers (decoding JSON was a primary motivating use case for that proposal).  If the requested type conforms to RawRepresentable and the encoded value can be converted to RawValue (perhaps using a failable numeric initializer) I would expect the raw value initializer to be used to attempt conversion.  If Swift ever gained a standard mechanism for generalized value conversion I would also expect that to be used if a conversion is available from the encoded type to the requested type.

If either of those conversions fail I would expect something like an “invalid value” error or a “conversion failure” error rather than a “type mismatch” error.  The types don’t exactly mismatch, we just have a failable conversion process that did not succeed.

Context:

I’m glad Brent raised the topic of supporting multiple formats.  In his example the difference was remote and local formats.  I’ve also seen cases where the same API requires the same model to be encoded differently in different endpoints.  This is awful, but it also happens sometimes in the wild.  Supporting an application specified encoding context would be very useful for handling these situations (the `codingKeyContex` would not always be sufficient and would usually not be a good way to determine the required encoding or decoding format).  

A `[UserInfoKey: Any]` context was mentioned as a possibility.  This would be better than nothing, but why not use the type system to preserve information about the type of the context?  I have a slightly different approach in mind.  Why not just have a protocol that refines Codable with context awareness?

public protocol ContextAwareCodable: Codable {
    associatedtype Context
    init(from decoder: Decoder, with context: Context?) throws
    func encode(to encoder: Encoder, with context: Context?) throws
}
extension ContextAwareCodable {
    init(from decoder: Decoder) throws {
        try self.init(from: decoder, with: nil
    }
    func encode(to encoder: Encoder) throws {
        try self.encode(to: encoder, with: nil)
    }
}

Encoders and Decoders would be encouraged to support a top level encode / decode method which is generic and takes an application supplied context.  When the context is provided it would be given to all `ContextAwareCodable` types that are aware of this context type during encoding or decoding.  The coding container protocols would include an overload for `ContextAwareCodable` allowing the container to know whether the Value understands the context given to the top level encoder / decoder:

open func encode<Value : ContextAwareCodable>(_ value: Value?, forKey key: Key) throws

A common top level signature for coders and decoders would look something like this:

open func encode<Value : Codable, Context>(_ value: Value, with context: Context) throws -> Data

This approach would preserve type information about the encoding / decoding context.  It falls back to the basic Codable implementation when a Value doesn’t know about the current context.  The default implementation simply translates this to a nil context allowing ContextAwareCodable types to have a single implementation of the initializer and the encoding method that is used whether they are able to understand the current context or not.

- Matthew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170316/87bdb810/attachment.html>