<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I would presume that the index type will still be shared between String and SubString, will this mean that we will now be able to express index manipulation in StringProtocol?<div class=""><br class=""></div><div class="">I find StringProtocol a bit hard to deal with when attempting to make range conversions; it would be really nice if we could make this possible (or perhaps more intuitive... since, for the life of me I can’t figure out a way to generically convert indexes for StringProtocol adoption)</div><div class=""><br class=""></div><div class="">So lets say you have a function as such: </div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="color: #c42275" class="">func</span> foo<S: <span style="color: #6122ae" class="">StringProtocol</span>>(<span style="color: #c42275" class="">_</span> str: <span style="color: #6122ae" class="">S</span>, range: <span style="color: #6122ae" class="">Range</span><<span style="color: #6122ae" class="">S</span>.<span style="color: #6122ae" class="">Index</span>>) {</div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> range.lowerBound.samePosition(in: str.utf16)</div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">}</div><div style="margin: 0px; font-stretch: normal; font-size: 18px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="text-indent: -12px; margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><span style="font-family: Helvetica; font-size: 12px;" class="">results in the error </span><span style="text-indent: -12px;" class=""><font face="Menlo" class=""><span style="font-size: 11px;" class="">error: value of type 'S.Index' has no member ‘samePosition</span></font></span><font face="Menlo" class=""><span style="font-size: 11px;" class="">’</span></font></div><div style="text-indent: -12px; margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="text-indent: -12px; margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class="">This of course is an intended target of something that deals with strings and wants to deal with both strings and substrings uniformly since it is reasonable to pass either.</div><div style="text-indent: -12px; margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="text-indent: -12px; margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class="">In short: are StringProtocol accessors a consideration for conversion in this change?</div><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On May 27, 2017, at 10:40 AM, Dave Abrahams via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><br class="">Pretty version: <a href="https://github.com/dabrahams/swift-evolution/blob/string-index-overhaul/proposals/NNNN-string-index-overhaul.md" class="">https://github.com/dabrahams/swift-evolution/blob/string-index-overhaul/proposals/NNNN-string-index-overhaul.md</a><br class=""><br class="">----<br class=""><br class=""># String Index Overhaul<br class=""><br class="">* Proposal: [SE-NNNN](NNNN-string-index-overhaul.md)<br class="">* Authors: [Dave Abrahams](<a href="https://github.com/dabrahams" class="">https://github.com/dabrahams</a>)<br class="">* Review Manager: TBD<br class="">* Status: **Awaiting review**<br class="">* Pull Request Implementing This Proposal: <a href="https://github.com/apple/swift/pull/9806" class="">https://github.com/apple/swift/pull/9806</a> <br class=""><br class="">*During the review process, add the following fields as needed:*<br class=""><br class="">## Introduction<br class=""><br class="">Today `String` shares an `Index` type with its `CharacterView` but not<br class="">with its `UTF8View`, `UTF16View`, or `UnicodeScalarView`. This<br class="">proposal redefines `String.UTF8View.Index`, `String.UTF16View.Index`,<br class="">and `String.CharacterView.Index` as typealiases for `String.Index`,<br class="">and exposes a public `encodedOffset` property and initializer that can<br class="">be used to serialize and deserialize positions in a `String` or<br class="">`Substring`.<br class=""><br class="">Swift-evolution thread: [Discussion thread topic for that proposal](<a href="https://lists.swift.org/pipermail/swift-evolution/" class="">https://lists.swift.org/pipermail/swift-evolution/</a>)<br class=""><br class="">## Motivation<br class=""><br class="">The different index types are supported by a set of `Index`<br class="">initializers, which are failable whenever the source index might not<br class="">correspond to a position in the target view:<br class=""><br class="">```swift<br class="">if let j = String.UnicodeScalarView.Index(<br class=""> someUTF16Position, within: s.unicodeScalars) {<br class=""> ... <br class="">}<br class="">```<br class=""><br class="">The current API is as follows:<br class=""><br class="">```swift<br class="">public extension String.Index {<br class=""> init?(_: String.UnicodeScalarIndex, within: String)<br class=""> init?(_: String.UTF16Index, within: String)<br class=""> init?(_: String.UTF8Index, within: String)<br class="">}<br class=""><br class="">public extension String.UTF16View.Index {<br class=""> init?(_: String.UTF8Index, within: String.UTF16View)<br class=""> init(_: String.UnicodeScalarIndex, within: String.UTF16View)<br class=""> init(_: String.Index, within: String.UTF16View)<br class="">}<br class=""><br class="">public extension String.UTF8View.Index {<br class=""> init?(_: String.UTF16Index, within: String.UTF8View)<br class=""> init(_: String.UnicodeScalarIndex, within: String.UTF8View)<br class=""> init(_: String.Index, within: String.UTF8View)<br class="">}<br class=""><br class="">public extension String.UnicodeScalarView.Index {<br class=""> init?(_: String.UTF16Index, within: String.UnicodeScalarView)<br class=""> init?(_: String.UTF8Index, within: String.UnicodeScalarView)<br class=""> init(_: String.Index, within: String.UnicodeScalarView)<br class="">}<br class="">```<br class=""><br class="">These initializers are supplemented by a corresponding set of<br class="">convenience conversion methods:<br class=""><br class="">```swift<br class="">if let j = someUTF16Position.samePosition(in: s.unicodeScalars) {<br class=""> ... <br class="">}<br class="">```<br class=""><br class="">with the following API:<br class=""><br class="">```swift<br class="">public extension String.Index {<br class=""> func samePosition(in: String.UTF8View) -> String.UTF8View.Index<br class=""> func samePosition(in: String.UTF16View) -> String.UTF16View.Index<br class=""> func samePosition(<br class=""> in: String.UnicodeScalarView) -> String.UnicodeScalarView.Index<br class="">}<br class=""><br class="">public extension String.UTF16View.Index {<br class=""> func samePosition(in: String) -> String.Index?<br class=""> func samePosition(in: String.UTF8View) -> String.UTF8View.Index?<br class=""> func samePosition(<br class=""> in: String.UnicodeScalarView) -> String.UnicodeScalarView.Index?<br class="">}<br class=""><br class="">public extension String.UTF8View.Index {<br class=""> func samePosition(in: String) -> String.Index?<br class=""> func samePosition(in: String.UTF16View) -> String.UTF16View.Index?<br class=""> func samePosition(<br class=""> in: String.UnicodeScalarView) -> String.UnicodeScalarView.Index?<br class="">}<br class=""><br class="">public extension String.UnicodeScalarView.Index {<br class=""> func samePosition(in: String) -> String.Index?<br class=""> func samePosition(in: String.UTF8View) -> String.UTF8View.Index<br class=""> func samePosition(in: String.UTF16View) -> String.UTF16View.Index<br class="">}<br class="">```<br class=""><br class="">The result is a great deal of API surface area for apparently little<br class="">gain in ordinary code, that normally only interchanges indices among<br class="">views when the positions match up exactly (i.e. when the conversion is<br class="">going to succeed). Also, the resulting code is needlessly awkward.<br class=""><br class="">Finally, the opacity of these index types makes it difficult to record<br class="">`String` or `Substring` positions in files or other archival forms,<br class="">and reconstruct the original positions with respect to a deserialized<br class="">`String` or `Substring`.<br class=""><br class="">## Proposed solution<br class=""><br class="">All `String` views will use a single index type (`String.Index`), so<br class="">that positions can be interchanged without awkward explicit<br class="">conversions:<br class=""><br class="">```swift<br class="">let html: String = "See <a href=\"<a href="http://swift.org" class="">http://swift.org</a>\"><a href="http://swift.org" class="">swift.org</a></a>"<br class=""><br class="">// Search the UTF16, instead of characters, for performance reasons:<br class="">let open = "<".utf16.first!, close = ">".utf16.first!<br class="">let tagStart = s.utf16.index(of: open)<br class="">let tagEnd = s.utf16[tagStart...].index(of: close)<br class=""><br class="">// Slice the String with the UTF-16 indices to retrieve the tag.<br class="">let tag = html[tagStart...tagEnd]<br class="">```<br class=""><br class="">A property and an intializer will be added to `String.Index`, exposing<br class="">the offset of the index in code units (currently only UTF-16) from the<br class="">beginning of the string:<br class=""><br class="">```swift<br class="">let n: Int = html.endIndex.encodedOffset<br class="">let end = String.Index(encodedOffset: n)<br class="">assert(end == String.endIndex)<br class="">```<br class=""><br class=""># Comparison and Slicing Semantics<br class=""><br class="">When two indices being compared correspond to positions that are valid<br class="">in any single `String` view, comparison semantics are already fully<br class="">specified by the `Collection` requirements. Where no single `String`<br class="">view contains both index values, the indices compare unequal and<br class="">ordering is determined by comparison of `encodedOffsets`. These index<br class="">values are not totally ordered but do satisfy strict weak ordering<br class="">requirements, which is sufficient for algorithms such as `sort` to<br class="">exhibit sensible behavior. We might consider loosening the specified<br class="">requirements on these algorithms and on `Comparable` to support strict<br class="">weak ordering, but for now we can treat such index pairs as being<br class="">outside the domain of comparison, like any other indices from<br class="">completely distinct collections.<br class=""><br class="">An index that does not fall on an exact boundary in a given `String`<br class="">or `Substring` view will be “rounded down” to the nearest boundary<br class="">when used for slicing or range replacement. So, for example,<br class=""><br class="">```swift<br class="">let s = "e\u{301}galite\u{301}" // "égalité"<br class="">print(s[s.unicodeScalars.indices.dropFirst().first!...]) // "égalité"<br class="">print(s[..<s.unicodeScalars.indices.last!]) // "égalit"<br class="">```<br class=""><br class="">Replacing the failable APIs listed [above](#motivation) that detect<br class="">whether an index represents a valid position in a given view, and<br class="">enhancement that explicitly round index positions to nearby boundaries<br class="">in a given view, are left to a later proposal. For now, we do not<br class="">propose to remove the existing index conversion APIs.<br class=""><br class="">## Detailed design<br class=""><br class="">`String.Index` acquires an `encodedOffset` property and initializer:<br class=""><br class="">```swift<br class="">public extension String.Index {<br class=""> /// Creates a position corresponding to the given offset in a<br class=""> /// `String`'s underlying (UTF-16) code units.<br class=""> init(encodedOffset: Int)<br class=""><br class=""> /// The position of this index expressed as an offset from the<br class=""> /// beginning of the `String`'s underlying (UTF-16) code units.<br class=""> var encodedOffset: Int<br class="">}<br class="">```<br class=""><br class="">`Index` types of `String.UTF8View`, `String.UTF16View`, and<br class="">`String.UnicodeScalarView` are replaced by `String.Index`:<br class=""><br class="">```swift<br class="">public extension String.UTF8View {<br class=""> typealias Index = String.Index<br class="">}<br class="">public extension String.UTF16View {<br class=""> typealias Index = String.Index<br class="">}<br class="">public extension String.UnicodeScalarView {<br class=""> typealias Index = String.Index<br class="">}<br class="">```<br class=""><br class="">Because the index types are collapsing, index conversion methods and<br class="">initializers are reduced to the following:<br class=""><br class="">```swift<br class="">public extension String.Index {<br class=""> init?(_: String.Index, within: String)<br class=""> init?(_: String.Index, within: String.UTF8View)<br class=""> init?(_: String.Index, within: String.UTF16View)<br class=""> init?(_: String.Index, within: String.UnicodeScalarView)<br class=""><br class=""> func samePosition(in: String) -> String.Index?<br class=""> func samePosition(in: String.UTF8View) -> String.Index?<br class=""> func samePosition(in: String.UTF16View) -> String.Index?<br class=""> func samePosition(in: String.UnicodeScalarView) -> String.Index?<br class="">}<br class="">```<br class=""><br class="">## Source compatibility<br class=""><br class="">Because of the collapse of index<br class="">types, [existing non-failable APIs](#motivation) become failable. To<br class="">avoid breaking Swift 3 code, the following overloads of existing<br class="">functions are added, allowing the resulting optional indices to be<br class="">used where previously non-optional indices were used. These overloads<br class="">were driven by making the new APIs work with existing code, including<br class="">the Swift source compatibility test suite, and should be viewed as<br class="">migration aids only, rather than additions to the Swift 3 API.<br class=""><br class="">```swift<br class="">extension Optional where Wrapped == String.Index {<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional indices")<br class=""> public static func ..<(<br class=""> lhs: String.Index?, rhs: String.Index?<br class=""> ) -> Range<String.Index> {<br class=""> return lhs! ..< rhs!<br class=""> }<br class=""><br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional indices")<br class=""> public static func ...(<br class=""> lhs: String.Index?, rhs: String.Index?<br class=""> ) -> ClosedRange<String.Index> {<br class=""> return lhs! ... rhs!<br class=""> }<br class="">}<br class=""><br class="">// backward compatibility for index interchange. <br class="">extension String.UTF16View {<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(after i: Index?) -> Index {<br class=""> return index(after: i)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(<br class=""> _ i: Index?, offsetBy n: IndexDistance) -> Index {<br class=""> return index(i!, offsetBy: n)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional indices")<br class=""> public func distance(from i: Index?, to j: Index?) -> IndexDistance {<br class=""> return distance(from: i!, to: j!)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public subscript(i: Index?) -> Unicode.UTF16.CodeUnit {<br class=""> return self[i!]<br class=""> }<br class="">}<br class=""><br class="">extension String.UTF8View {<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(after i: Index?) -> Index {<br class=""> return index(after: i!)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(_ i: Index?, offsetBy n: IndexDistance) -> Index {<br class=""> return index(i!, offsetBy: n)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional indices")<br class=""> public func distance(<br class=""> from i: Index?, to j: Index?) -> IndexDistance {<br class=""> return distance(from: i!, to: j!)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public subscript(i: Index?) -> Unicode.UTF8.CodeUnit {<br class=""> return self[i!]<br class=""> }<br class="">}<br class=""><br class="">// backward compatibility for index interchange. <br class="">extension String.UnicodeScalarView {<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(after i: Index?) -> Index {<br class=""> return index(after: i)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public func index(_ i: Index?, offsetBy n: IndexDistance) -> Index {<br class=""> return index(i!, offsetBy: n)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional indices")<br class=""> public func distance(from i: Index?, to j: Index?) -> IndexDistance {<br class=""> return distance(from: i!, to: j!)<br class=""> }<br class=""> @available(<br class=""> swift, deprecated: 3.2, obsoleted: 4.0,<br class=""> message: "Any String view index conversion can fail in Swift 4; please unwrap the optional index")<br class=""> public subscript(i: Index?) -> Unicode.Scalar {<br class=""> return self[i!]<br class=""> }<br class="">}<br class="">```<br class=""><br class="">- **Q**: Will existing correct Swift 3 applications stop compiling due<br class=""> to this change?<br class=""><br class=""> **A**: it is possible but unlikely. The existing index conversion<br class=""> APIs are relatively rarely used, and the overloads listed above<br class=""> handle the common cases in Swift 3 compatibility mode.<br class=""><br class="">- **Q**: Will applications still compile but produce<br class=""> different behavior than they used to? <br class=""><br class=""> **A**: No.<br class=""><br class="">- **Q**: Is it possible to automatically migrate from the old syntax<br class=""> to the new syntax? <br class=""><br class=""> **A**: Yes, although usages of these APIs may be rare enough that it<br class=""> isn't worth the trouble.<br class=""><br class="">- **Q**: Can Swift applications be written in a common subset that works<br class=""> both with Swift 3 and Swift 4 to aid in migration?<br class=""><br class=""> **A**: Yes, the Swift 4 APIs will all be available in Swift 3 mode.<br class=""><br class="">## Effect on ABI stability<br class=""><br class="">This proposal changes the ABI of the standard library.<br class=""><br class="">## Effect on API resilience<br class=""><br class="">This proposal makes no changes to the resilience of any APIs.<br class=""><br class="">## Alternatives considered<br class=""><br class="">The only alternative considered was no action.<br class=""><br class=""><br class="">-- <br class="">-Dave<br class=""><br class="">_______________________________________________<br class="">swift-evolution mailing list<br class=""><a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-evolution<br class=""></div></div></blockquote></div><br class=""></div></div></body></html>