[swift-evolution] Trial balloon: Ensure that String always contains valid Unicode

[Tony Parker]

> On Dec 19, 2015, at 7:59 PM, Dmitri Gribenko <gribozavr at gmail.com> wrote:
> 
> On Fri, Dec 18, 2015 at 1:47 PM, Paul Cantrell via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
> I was quite surprised to learn that it’s possible to create Swift strings that do not contain things other than valid Unicode characters. Is it feasible to guarantee that this cannot happen?
> 
> String.init(bytes:encoding:) is failable, and does in fact validate that the given bytes are decodable with the given encoding in most circumstances:
> 
>     // Returns nil
>     String(
>         bytes: [0xD8, 0x00] as [UInt8],
>         encoding: NSUTF8StringEncoding)
> 
> However, that initializer does not reject invalid surrogate characters in UTF-16:
> 
>     // Succeeds (wat?!)
>     let bogusStr = String(
>         bytes: [0xD8, 0x00] as [UInt8],
>         encoding: NSUTF16BigEndianStringEncoding)!
> 
> Adding this would be a useful guarantee, I support this.  The current behavior looks inconsistent to me.  OTOH, the current behavior of String(bytes:encoding:) mirrors the behavior of the NSString method, so this would create inconsistency.  But I think the extra guarantee is worth it.
> 
> Tony, what do you think?
> 

NSString deals with this issue more on the ‘get’ side. For example, CFStringGetBytes has a ‘lossByte’ for use in replacement when the requested encoding cannot represent something stored by the receiver string. Also, the abstract NSString interface can be extended to add additional encodings (which is why the string encoding values are not an enumeration).

- Tony

> Dmitri
> 
> -- 
> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com <mailto:gribozavr at gmail.com>>*/


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160104/44f96a9f/attachment.html>