[swift-users] Decode a JSON object of unknown format into a Dictionary with Decodable in Swift 4
Tony Parker
anthony.parker at apple.com
Thu Jun 22 20:13:25 CDT 2017
Hi Jon,
> On Jun 22, 2017, at 6:00 PM, Jon Shier <jon at jonshier.com> wrote:
>
> Tony:
> My main concern here is that, as Swift’s official JSON parsing method, Codable should be able to handle any JSON representation and use and it doesn’t. In fact, it can’t. If that’s considered okay by the designers of the library and Apple itself, then fine. I think it’s silly (just like I think it’s silly that, after years of waiting, NSISO8601DateFormatter doesn’t actually support the full ISO 8601 standard) and does a disservice to developers, but fine. For example, many JSON APIs (from the very worst to some of the best) use a common reply structure:
>
> {
> “value” : { } or null
> “error” : { } or null
> }
>
> Having implemented a few of these using Argo, I usually follow a common pattern:
>
> struct APIResponse: Argo.Decodable {
> let value: JSON?
> let error: APIError?
> }
>
> I then decode the response, check to see which of the parts is non-null, and continue decoding based on that result. I decode the value based on the generic type passed in, usually in an Alamofire response serializer. Codable can sort of represent this, with this:
>
> struct APIResponse<T: Decodable>: Decodable {
> let value: T?
> let error: String? // String as I haven’t defined an error type.
> }
>
> This works, AFAICT, but loses the rather important ability to inspect the decoding result before attempting to decode the generic value. Codable would be perfectly happy to return me an APIResponse with two nil values, which is an invalid state for this API and should be an error. (As an aside, defaulting all errors for optional values to nil is poor practice and we lose some error fidelity there.) This is really just a symptom of Foundation not having a real JSON limitation. But it’s doubly concerning that the JSON representation it does have, Any, can’t be parsed by it’s own JSON decoder.
Have you considered using an enum?
let jsonA = """
{
"key1" : 1
}
""".data(using: .utf8)!
let jsonB = """
{
"key2" : "foo"
}
""".data(using: .utf8)!
enum EitherOr : Decodable {
case A(Int)
case B(String)
private enum CodingKeys : CodingKey { case key1; case key2 }
init(from decoder: Decoder) throws {
let c = try decoder.container(keyedBy: CodingKeys.self)
if c.contains(.key1) {
self = .A(try c.decode(Int.self, forKey: .key1))
} else if c.contains(.key2) {
self = .B(try c.decode(String.self, forKey: .key2))
} else {
throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: [], debugDescription: "”))
}
}
}
let decoder = JSONDecoder()
let result1 = try! decoder.decode(EitherOr.self, from: jsonA)
print(result1)
let result2 = try! decoder.decode(EitherOr.self, from: jsonB)
print(result2)
A(1)
B("foo")
> Dealing with data also limits Codable’s use for JSON to only things that have data representation. For example, push notifications can contain custom JSON payloads. On Apple platforms the push handler automatically decodes this JSON before passing the result to the delegates in the app. In the past I’ve been able to use the same JSON representations to parse the Any returned by the push delegates as I do in my networking API, which gives me exactly the same parsing for both methods. Codable can’t be used here.
> In addition, Codable’s cumbersome API for custom APIs is rather painful to deal with, and the official recommended solution to use *Raw types that come from your API and are used to initialize your “real” types is untenable (on one project, every type I got in a response wouldn’t have required this, forcing me to maintain 60 types, manually). Add to that the manual nature of any sort of transform or validation, and Codable for real JSON decoding becomes rather painful.
> For me, this means that I’ll likely stick to libraries like Argo for actual JSON decoding (greater fidelity and flexibility in far less code) and use Codable for disk and other transfer representations (I look forward to writing my watch communication using it) only, in all but the most ideal circumstances (if I design the JSON API I can optimize it around Codable).
>
>
> Jon Shier
Sure, there is only so much we can do completely automatically. One of our goals was to make the simplest stuff possible with as little boilerplate as possible, but provide the ability to customize when you want to do something more advanced (as above).
- Tony
>
>
>> On Jun 22, 2017, at 7:10 PM, Tony Parker <anthony.parker at apple.com <mailto:anthony.parker at apple.com>> wrote:
>>
>> Hi Jon,
>>
>> Usually this boils down to a question of: what are you going to do with the Any?
>>
>> If you intended to cast it to a dictionary and get at its values using string keys, then writing a struct with the properties you care about is the way we recommend doing this. You don’t have to include every possible key.
>>
>> There are some more dynamic behaviors possible with the Any, but the point of this API was to try to move a lot of the parsing into a place where the compiler could help you not only to generate the boilerplate but catch errors at compile time.
>>
>> The omission of an API to just get the rest of the data as some kind of Any is both for reasons of abstraction (not every data format is JSON, and the Any truly could be anything in other formats), and a statement of intent of how we believe this API is best used.
>>
>> - Tony
>>
>>> On Jun 18, 2017, at 7:23 PM, Jon Shier via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
>>>
>>> Given that, per his description, “metadata” can be anything, creating a struct doesn’t really help.
>>>
>>>
>>>
>>> Jon Shier
>>>
>>>> On Jun 18, 2017, at 9:00 PM, somu subscribe <somu.subscribe at gmail.com <mailto:somu.subscribe at gmail.com>> wrote:
>>>>
>>>> Create a struct for Metadata and conform to Coding
>>>>
>>>> Code:
>>>>
>>>> let string = """
>>>> {
>>>> "id": "4yq6txdpfadhbaqnwp3",
>>>> "email": "john.doe at example.com <mailto:john.doe at example.com>",
>>>> "name":"John Doe",
>>>> "metadata": {
>>>> "link_id": "linked-id",
>>>> "buy_count": 4
>>>> }
>>>> }
>>>> """
>>>>
>>>> let data = string.data(using: .utf8)! //Force unwrapping just for ease of explanation, use guard instead
>>>>
>>>> struct User: Codable {
>>>>
>>>> //Struct to handle the nested JSON
>>>> struct Metadata : Codable {
>>>> var linkID : String
>>>> var buyCount : Int
>>>>
>>>> //Customisation, if you want if you want your properties to be different from the JSON key
>>>> private enum CodingKeys : String, CodingKey {
>>>> case linkID = "link_id"
>>>> case buyCount = "buy_count"
>>>> }
>>>> }
>>>>
>>>> var id: String
>>>> var email : String
>>>> var name : String
>>>> var metadata : Metadata
>>>> }
>>>>
>>>> let decoder = JSONDecoder()
>>>>
>>>> do {
>>>> let user = try decoder.decode(User.self, from: data)
>>>> print(user)
>>>> }
>>>> catch {
>>>> print("error:\(error)")
>>>> }
>>>>
>>>> Reference:
>>>>
>>>> https://developer.apple.com/videos/play/wwdc2017/212/ <https://developer.apple.com/videos/play/wwdc2017/212/>
>>>>
>>>>
>>>> Regards,
>>>> Muthu
>>>>
>>>>
>>>>
>>>>> On 19 Jun 2017, at 3:29 AM, Jon Shier via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
>>>>>
>>>>> The more I use Codable, the less suited to networking it becomes. In reading a variety of blog posts about implementing custom Decodable support from JSON, I kept running across the same pattern. Basically, users had started implementing their own decoding protocols which wrap Decodable types, and have a type that represents the JSON representation and then their real type, with an initializer connecting the two. But apparently this is Apple’s official solution, which is rather terrible since it would be completely unnecessary if the Decodable APIs were more flexible or we could access keys by key path rather than nesting full containers. I can’t image how much code I would have to add to decode the nasty JSON APIs I’ve used Argo to parse before. Every type would need an underlying raw representation that, at the very least, would need custom keys, lest I pollute even those models with the terrible keys the JSON actually has. Not to mention the various transforms I needed. Once I hit any reasonably complex API, Argo is far far simpler to implement in fewer lines of code.
>>>>> In trying to make Argo’s JSON enum Decodable itself, I can’t seem to find a way to access the Any representation of the raw JSON. In fact, it appears there’s no way to represent an Any value in Codable at all, which makes Codable rather useless for the scenarios like the one that prompted this thread. Without such an ability it’s impossible to actually use Codable with all of the JSON out there, where other solutions work just fine. Argo’s JSON type is decodable by Argo, so you can use it to represent a blob of JSON just fine. Other existing JSON frameworks have similar solutions.
>>>>>
>>>>>
>>>>>
>>>>> Jon
>>>>>
>>>>>> On Jun 18, 2017, at 3:21 AM, Rien via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
>>>>>>
>>>>>> Dang, hit send too soon. Sorry.
>>>>>>
>>>>>> This does not address your question, so please ignore… (foot in mouth)!
>>>>>>
>>>>>> Regards,
>>>>>> Rien
>>>>>>
>>>>>> Site: http://balancingrock.nl <http://balancingrock.nl/>
>>>>>> Blog: http://swiftrien.blogspot.com <http://swiftrien.blogspot.com/>
>>>>>> Github: http://github.com/Balancingrock <http://github.com/Balancingrock>
>>>>>> Project: http://swiftfire.nl <http://swiftfire.nl/> - An HTTP(S) web server framework in Swift
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 18 Jun 2017, at 09:19, Rien <Rien at Balancingrock.nl <mailto:Rien at Balancingrock.nl>> wrote:
>>>>>>>
>>>>>>> Are you looking for a general purpose JSON interpreter / API ?
>>>>>>>
>>>>>>> There are many of them around, and in fact I do have my own: https://github.com/Balancingrock/VJson <https://github.com/Balancingrock/VJson>
>>>>>>>
>>>>>>> With VJson I would write:
>>>>>>>
>>>>>>> let json = VJson.parse(… your json object…)
>>>>>>>
>>>>>>> and then access the metadata as:
>>>>>>>
>>>>>>> let buyCount = (json | ”metadata” | ”buy_count”)?.intValue
>>>>>>>
>>>>>>> or:
>>>>>>>
>>>>>>> var buyCount: Int &= json | “metadata” | “buy_count”
>>>>>>>
>>>>>>> To loop over the content of metadata:
>>>>>>>
>>>>>>> for item in (json | “metadata”) ?? [ ] {
>>>>>>> print (item.nameValue)
>>>>>>> switch item.type {
>>>>>>> case .object: …
>>>>>>> case .number: …
>>>>>>> case .string: …
>>>>>>> etc...
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Obviously I am plugging my own code here, but there are many others around, I understand that SwiftyJSON is quite popular but I have not used that myself.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Rien
>>>>>>>
>>>>>>> Site: http://balancingrock.nl <http://balancingrock.nl/>
>>>>>>> Blog: http://swiftrien.blogspot.com <http://swiftrien.blogspot.com/>
>>>>>>> Github: http://github.com/Balancingrock <http://github.com/Balancingrock>
>>>>>>> Project: http://swiftfire.nl <http://swiftfire.nl/> - An HTTP(S) web server framework in Swift
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 18 Jun 2017, at 04:07, Chris Anderson via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
>>>>>>>>
>>>>>>>> Say I have a JSON object such as:
>>>>>>>>
>>>>>>>> {
>>>>>>>> "id": "4yq6txdpfadhbaqnwp3",
>>>>>>>> "email": "john.doe at example.com <mailto:john.doe at example.com>",
>>>>>>>> "name":"John Doe",
>>>>>>>> "metadata": {
>>>>>>>> "link_id": "linked-id",
>>>>>>>> "buy_count": 4
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> And with a struct of:
>>>>>>>>
>>>>>>>> struct User: Codable {
>>>>>>>> var id: String
>>>>>>>> var email: String
>>>>>>>> var name: String
>>>>>>>> }
>>>>>>>>
>>>>>>>> How can I decode the `metadata` field into a Dictionary?
>>>>>>>>
>>>>>>>> I’ve tried doing things such as, in my struct,
>>>>>>>>
>>>>>>>> var metadata: Dictionary
>>>>>>>>
>>>>>>>> or
>>>>>>>>
>>>>>>>> var metadata: [String: Any]
>>>>>>>>
>>>>>>>> But I get the error
>>>>>>>>
>>>>>>>> MyPlayground.playground:3:7: note: cannot automatically synthesize 'Encodable' because '<<error type>>' does not conform to 'Encodable'
>>>>>>>> var metadata: Dictionary
>>>>>>>>
>>>>>>>> A meta or metadata field on many APIs (such as www.stripe.com <http://www.stripe.com/>) can contain whatever you want, and I still want to be able to process it on the Swift end. How can I store that meta data field into a Dictionary that I can parse apart manually after?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Chris Anderson
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> swift-users mailing list
>>>>>>>> swift-users at swift.org <mailto:swift-users at swift.org>
>>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> swift-users mailing list
>>>>>> swift-users at swift.org <mailto:swift-users at swift.org>
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
>>>>>
>>>>> _______________________________________________
>>>>> swift-users mailing list
>>>>> swift-users at swift.org <mailto:swift-users at swift.org>
>>>>> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
>>>>
>>>
>>> _______________________________________________
>>> swift-users mailing list
>>> swift-users at swift.org <mailto:swift-users at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20170622/90da9859/attachment.html>
More information about the swift-users
mailing list