[swift-corelibs-dev] Adding type conversion capabilities to JSON encode/decode
Youming Lin
ylin at us.ibm.com
Wed Aug 30 17:19:04 CDT 2017
Brandon
Sorry, I didn't realize you want it on the String type since you used other
string ivars in your example as well.
I'm not sure that forcing all numbers to be valid decoded strings is a good
idea, because this behavior gets applied to other string ivars that you may
not want to be compatible with numbers, which is the inverse of the problem
you stated.
A custom Codable type to represent number-equivalent strings is a better
solution I think.
Thanks,
Youming Lin
IBM Cloud, Swift at IBM, Kitura developer
Austin, TX
GitHub: @youming-lin
From: "Sneed, Brandon" <brsneed at ebay.com>
To: Youming Lin <ylin at us.ibm.com>
Cc: Tony Parker <anthony.parker at apple.com>,
"swift-corelibs-dev at swift.org" <swift-corelibs-dev at swift.org>,
"swift-corelibs-dev-bounces at swift.org"
<swift-corelibs-dev-bounces at swift.org>
Date: 08/30/2017 04:42 PM
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
Thanks Youming,
That’s not quite what I meant. I may have misinterpreted what Tony was
saying though.
I wanted to do conversion on String, not the containing type. The problem
of doing it on the containing type is that as soon as you need one field to
be custom, you’re roped into handling all the others. For a small struct,
not a big deal, for larger ones, it is. Something like this is what I
tried:
import Cocoa
let jsonString = "{" +
"\"name\": \"Endeavor\"," +
"\"abv\": 8.9," +
"\"brewery\": \"Saint Arnold\"," +
"\"style\": \"ipa\"}"
struct Beer: Codable {
let name: String
let abv: String
let brewery: String
let style: String
}
extension String {
init(from decoder: Decoder) throws {
print("i got hit.")
let value = try decoder.singleValueContainer().decode(String.self)
self.init(value)
}
}
let jsonData = jsonString.data(using: .utf8)
let decoder = JSONDecoder()
let beer = try! decoder.decode(Beer.self, from: jsonData!)
fatal error: 'try!' expression unexpectedly raised an error:
Swift.DecodingError.typeMismatch(Swift.String, Swift.DecodingError.Context
(codingPath: [__lldb_expr_17.Beer.(CodingKeys in
_C8902E33F84CE6946081129DAF1824E1).abv], debugDescription: "Expected to
decode String but found a number instead.", underlyingError: nil)):
file /Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-900.0.59/src/swift/stdlib/public/core/ErrorType.swift,
line 181
From: Youming Lin <ylin at us.ibm.com>
Date: Wednesday, August 30, 2017 at 2:35 PM
To: "Sneed, Brandon" <brsneed at ebay.com>
Cc: Tony Parker <anthony.parker at apple.com>, "swift-corelibs-dev at swift.org"
<swift-corelibs-dev at swift.org>, "swift-corelibs-dev-bounces at swift.org"
<swift-corelibs-dev-bounces at swift.org>
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
Brandon
I cooked up a simple example and it works as expected.
ylin at youming-mbpr:~/Swift/Configuration$ swift
Welcome to Apple Swift version 4.0-dev (LLVM 2dedb62a0b, Clang b9d76a314c,
Swift 0899bd328a). Type :help for assistance.
1> import Foundation
2> struct A: Codable {
3. var integer: Int
4.
5. public init(integer: Int) {
6. self.integer = integer
7. }
8.
9. public init(from decoder: Decoder) throws {
10. print("Custom decoder")
11. let container = try decoder.container(keyedBy: CodingKeys.self)
12. integer = try container.decode(Int.self, forKey: .integer)
13. }
14. }
15.
16. try! JSONDecoder().decode(A.self, from: JSONEncoder().encode(A(integer:
42)))
Custom decoder
$R0: A = {
integer = 42
}
You should be able to implement your custom init to convert the number into
a string and JSONDecoder should use that automatically.
Thanks,
Youming Lin
IBM Cloud, Swift at IBM, Kitura developer
Austin, TX
GitHub: @youming-lin
nactive hide details for "Sneed, Brandon" ---08/30/2017 04:14:57 PM---Tha
"Sneed, Brandon" ---08/30/2017 04:14:57 PM---Thanks Youming, Ok, thanks! I
did try that, but I can’t seem to figure out how to make it actually
From: "Sneed, Brandon" <brsneed at ebay.com>
To: Youming Lin <ylin at us.ibm.com>
Cc: Tony Parker <anthony.parker at apple.com>, "swift-corelibs-dev at swift.org"
<swift-corelibs-dev at swift.org>, "swift-corelibs-dev-bounces at swift.org"
<swift-corelibs-dev-bounces at swift.org>
Date: 08/30/2017 04:14 PM
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
Thanks Youming,
Ok, thanks! I did try that, but I can’t seem to figure out how to make it
actually get used in the decoding process. That being my preferred way, I
tried it first. I chalked it not working up to Swift not knowing which of
the 2 init(from decoder:) functions to call, mine or theirs. But, maybe
there’s something I’m missing here.
Any insight is appreciated.
Thanks!
Brandon Sneed
From: Youming Lin <ylin at us.ibm.com>
Date: Wednesday, August 30, 2017 at 1:18 PM
To: "Sneed, Brandon" <brsneed at ebay.com>
Cc: Tony Parker <anthony.parker at apple.com>, "swift-corelibs-dev at swift.org"
<swift-corelibs-dev at swift.org>, "swift-corelibs-dev-bounces at swift.org"
<swift-corelibs-dev-bounces at swift.org>
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
Brandon
>JSON’s types effectively end up matching specifically to primitives, of
which there is no mechanism to override the behavior of how a String gets
decoded for instance.
You can override the default behavior with your own custom init(from:)
implementation for your Codable struct:
https://developer.apple.com/documentation/swift/decodable/2894081-init
You can check Foundation source code (i.e., the URL struct) on how this can
be implemented.
Thanks,
Youming Lin
IBM Cloud, Swift at IBM, Kitura developer
Austin, TX
GitHub: @youming-lin
active hide details for "Sneed, Brandon via swift-corelibs-dev" ---08/30
"Sneed, Brandon via swift-corelibs-dev" ---08/30/2017 03:07:05 PM---Hi
Tony, I like the idea that the type itself is responsible for the
conversion. My own json encode
From: "Sneed, Brandon via swift-corelibs-dev"
<swift-corelibs-dev at swift.org>
To: Tony Parker <anthony.parker at apple.com>
Cc: "swift-corelibs-dev at swift.org" <swift-corelibs-dev at swift.org>
Date: 08/30/2017 03:07 PM
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
Sent by: swift-corelibs-dev-bounces at swift.org
Hi Tony,
I like the idea that the type itself is responsible for the conversion. My
own json encode/decode library worked this way and it was really great,
however in trying to leverage Swift4 into it, or to replace it, I just
don’t see how that’s possible given how it’s currently structured.
JSON’s types effectively end up matching specifically to primitives, of
which there is no mechanism to override the behavior of how a String gets
decoded for instance. The only way I can think of to accomplish that is to
create *another* type, JSONString for example, but since String is a
struct, I can’t subclass it, and instead need to have the real value buried
inside of it … it seems to start getting messy very quickly. It also adds
the obfuscation of dealing with yet another type, which I’m not against,
but just feels less than ideal.
Brandon Sneed
From: <anthony.parker at apple.com> on behalf of Tony Parker
<anthony.parker at apple.com>
Date: Wednesday, August 30, 2017 at 11:30 AM
To: "Sneed, Brandon" <brsneed at ebay.com>
Cc: Itai Ferber <iferber at apple.com>, "swift-corelibs-dev at swift.org"
<swift-corelibs-dev at swift.org>
Subject: Re: [swift-corelibs-dev] Adding type conversion capabilities to
JSON encode/decode
I’m still not convinced that we should actually provide such a strategy.
Conversions like those below seem like the domain of each type that is
being decoded. If, in a particular type, the “number” can be either a true
number or a string, then that type can try decoding it as one or the other
and fall back as required. That puts the responsibility of doing that kind
of conversion in the type itself.
JSON has very few types already. I’m not sure we want to blur the line
between numbers and strings automatically…
- Tony
On Aug 30, 2017, at 11:24 AM, Sneed, Brandon via
swift-corelibs-dev <swift-corelibs-dev at swift.org>
wrote:
Hi Itai,
No problem! Thanks for the heads up. Is there any
way I could be involved? Happy to do the work to
whatever guidance your team might have. I’m mostly
just interested in it being there soon, hence
volunteering.
Thanks!
Brandon Sneed
From: <iferber at apple.com> on behalf of Itai Ferber
<iferber at apple.com>
Date: Wednesday, August 30, 2017 at 11:22 AM
To: "Sneed, Brandon" <brsneed at ebay.com>
Cc: "swift-corelibs-dev at swift.org" <
swift-corelibs-dev at swift.org>
Subject: Re: [swift-corelibs-dev] Adding type
conversion capabilities to JSON encode/decode
Hi Brandon,
Thanks for looking at this! We’ve got plans
internally to potentially add a strategy to
JSONEncoder/JSONDecoder to allow lenient
conversions like this — i.e. implicitly stringify
numbers (or parse them from string input), among
some others.
This would be opt-in for consumers of JSONDecoder
while not requiring any special annotations on
Codable types.
— Itai
On 30 Aug 2017, at 10:59, Sneed, Brandon via
swift-corelibs-dev wrote:
Hi everyone,
Just throwing this out to see if anyone else is
working on this, or has opinions/suggestions on how
it’s implemented. I’d like to add this to the
Codable/JSONDecoder/JSONEncoder system if no one
else is working on it.
Type type conversion, I mean given this JSON
payload:
{
"name": "Endeavor”,
"abv": 8.9,
"brewery": "Saint Arnold”,
"style": "ipa"
}
and a struct defined as:
struct Beer: Codable {
let name: String
let abv: String
let brewery: String
let style: BeerStyle
}
Notice that “abv” is a number in the JSON, but a
String in the struct. I’d like to make it such that
I can let the system know it’s ok to convert it
from a number to a string as opposed to throwing an
exception. The benefits are:
1. It’s defensive; service types can change without
causing my application to crash.
2. It allows a developer to work with the types
they want to work with as opposed to what the
server provides, thus saving them time of writing a
custom encode/decode code for all members.
The argument against it that I’ve heard is
generally “it’s a service bug, make them fix it”,
which is valid but the reality is we’re not all in
control of the services we injest. The same type of
logic could be applied to a member name changing,
though I haven’t seen this happen often in
practice. I do see types in a json payload change
with some frequency though. I think much of the
reason stems from the fact that type conversion in
javascript is effectively free, ie: you ask for a
String, you get a String if possible.
To implement this type conversion in practice,
looking at it from the point of view using
Codable/JSON(en/de)coder, one way would be to make
it opt-in:
struct Beer: Codable, CodingConvertible {
let name: String
let abv: String
let brewery: String
let style: BeerStyle
}
I like this because looking at the struct, the
members still remain clear and relatively
unambiguous. The downside is it’s unknown which
member is likely to get converted. And since it’s
opt-in, conversion doesn’t happen if the
CodingConvertible conformance isn’t adhered to.
Another option would be to box each type, like so:
struct Beer: Codable {
let name: String
let abv: Convertible<String>
let brewery: String
let style: BeerStyle
}
This seems tedious for developers, but would show
which types are being converted. It does however
seriously weaken benefit #1 above.
Those example usages above aside, I do think it’d
be best if this conversion behavior was the default
and no end-developer changes required. I think that
could be done without impact to code that’s been
already been written against the JSON en/decode
bits.
I’m very open to alternatives, other ideas, or
anything else you might have to say on the subject.
Thanks for reading!
Brandon Sneed
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev at swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev at swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev at swift.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.swift.org_mailman_listinfo_swift-2Dcorelibs-2Ddev&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=gkRZBtsmKeGPCOlAIRJoOA&m=ViDSVPImta3StTVAcktby2PMF_-du5itzz47jo-tNHg&s=zRuNQ3NLxpfhFBewRTkoMWZnpvHlm6Ja-ot9_pwAgqI&e=
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20170830/68843d40/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20170830/68843d40/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 70731091.gif
Type: image/gif
Size: 106 bytes
Desc: not available
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20170830/68843d40/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 70431800.gif
Type: image/gif
Size: 107 bytes
Desc: not available
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20170830/68843d40/attachment-0002.gif>
More information about the swift-corelibs-dev
mailing list