[swift-evolution] [Draft] Refining identifier and operator symbology (take 2)
Nevin Brackett-Rozinsky
nevin.brackettrozinsky at gmail.com
Tue Aug 8 13:09:55 CDT 2017
Is this proposal still on track, or are there other plans to address the
issue of operator and identifier characters in Swift?
Nevin
On Fri, Feb 17, 2017 at 12:50 AM, Xiaodi Wu via swift-evolution <
swift-evolution at swift.org> wrote:
> As Stage 2 of Swift 4 evolution starts now, I'd like to share a revised
> proposal in draft form.
>
> It proposes a source-breaking change for *rationalizing* which characters
> are permitted in identifiers and which in operators. It's justified for
> this phase of Swift 4 because:
>
> - Existing grammar, in permitting invisible characters without
> security-minded restrictions, can be *actively harmful.*
> - A rationalized approach is *superior* to the current approach: by
> referencing Unicode standards, Swift should be able to evolve in a
> backwards-compatible way alongside Unicode, and will benefit from the
> significant expertise of others outside the Swift community with respect to
> Unicode best practices.
> - The vast majority of existing code (including all of the standard
> library) should *require no migration* work at all
>
> *What's changed* since the last time:
>
> - In an earlier draft, we proposed some radical changes to align with
> available Unicode standards; in particular, since emoji represent a
> difficult issue, and no recommendations about "operator identifiers" have
> surfaced from Unicode, we proposed temporarily stripping them out. This was *very
> poorly received*. This revision uses Unicode categories to identify
> nearly all emoji and classify them as identifier characters (while
> excluding those that depict operators such as !), and it uses Unicode
> categories to identify over 900 operators that nearly all pass the
> subjective test of "operator-likeness."
>
> What this proposal *does not attempt* to do:
>
> - This document *does not* seek to stake out new ground as to what
> characters should be *added* to the set of valid identifiers and
> operators. Such additions to the grammar are properly separate discussions.
> This proposal is only an attempt at systemization and rationalization. Only
> one character is incidentally added to the list of valid characters (`\`),
> and it is on the basis of an explicit table in Unicode Technical Report 25
> regarding ASCII characters that are "mathematical."
>
> What feedback would be* most helpful*:
>
> - "Hey, this approach is so much more *clumsy* than my superior, more
> elegant category-based approach to identifying [operators/emoji], which is
> [insert here]."
> - "Hey, I disagree with the detailed design because it's got a *major
> security hole*, which is [insert here]."
> - "Hey, your proposal would break my *real-world* Swift code, which
> requires that character [X] be an [identifier/operator]."
>
> What would be *less helpful*:
>
> - "Hey, let's talk about how [specific character] should be an
> [identifier/operator]. We should add that character to the list of
> [identifiers/operators]. In fact, let's discuss [list] characters one by
> one."
>
> Acknowledgments:
> Thanks to co-authors of the previous take for their support for
> resurrecting this issue. Any brilliant ideas are undoubtedly theirs, and
> any botched efforts are certainly mine. Thanks also to Nevin
> Brackett-Rozinsky for helpful feedback.
>
> Link:
> https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651
>
>
> Rendered text:
>
> Refining identifier and operator symbology (take 2)
>
> - Proposal: SE-NNNN
> <https://gist.github.com/xwu/NNNN-refining-identifier-and-operator-symbology.md>
> - Authors: Xiaodi Wu <https://github.com/xwu>, Jacob Bandes-Storch
> <https://github.com/jtbandes>, Erica Sadun <https://github.com/erica>,
> Jonathan Shapiro, João Pinheiro <https://github.com/joaopinheiro>
> - Review Manager: TBD
> - Status: Awaiting review
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#introduction>
> Introduction
>
> This proposal refines and rationalizes Swift's identifier and operator
> symbology. Specifically, this proposal:
>
> - refines the set of valid identifier characters based on Unicode
> recommendations, with customizations principally to accommodate emoji;
> - refines the set of valid operator characters based on Unicode
> categories; and
> - changes rules as to where dots may appear in operators.
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#prior-discussion-threads-and-proposals>Prior
> discussion threads and proposals
>
> - Define backslash '\' as a operator-head in the swift grammar
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170130/031461.html>
> - Refining Identifier and Operator Symbology
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161017/028174.html> (a
> precursor to this document)
> - Proposal: Normalize Unicode identifiers
> <https://github.com/apple/swift-evolution/pull/531>
> - Lexical matters: identifiers and operators
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160926/027479.html>
> - Unicode identifiers & operators
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160912/027108.html>,
> with pre-proposal
> <https://gist.github.com/jtbandes/c0b0c072181dcd22c3147802025d0b59>
> - Proposal: Allow Single Dollar Sign as Valid Identifier
> <https://github.com/apple/swift-evolution/pull/354>
> - Free the '$' Symbol!
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151228/005133.html>
> - Request to add middle dot (U+00B7) as operator character?
> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151214/003176.html>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#motivation>
> Motivation
>
> Swift supports programmers from many languages and cultures. However, the
> current identifier and operator character sets do not conform to any
> Unicode standards, nor have they been rationalized in the language or
> compiler documentation. These deserve a well-considered, standards-based
> revision.
>
> As Chris Lattner has written:
>
> We need a token to be unambiguously an operator or identifier - we can
> have different rules for the leading and subsequent characters though.
>
> …our current operator space (particularly the Unicode segments covered) is
> not super well considered. It would be great for someone to take a more
> systematic pass over them to rationalize things.
>
> Identifiers, which serve as *names* for various entities, are linguistic
> in nature and must permit a variety of characters in order to properly
> serve non–English-speaking coders. This issue has been considered by the
> communities of many programming languages already, and the Unicode
> Consortium has published recommendations on how to choose identifier
> character sets. Swift should make an effort to conform to these
> recommendations.
>
> Operators, on the other hand, should be rare and carefully chosen because
> they suffer from limited discoverability and readability. They are by
> nature *symbols*, not names. This places a cognitive cost on users with
> respect to recall ("What is the operator that applies the behavior I
> need?") and recognition ("What does the operator in this code do?"). While
> almost every non-trivial program defines new identifiers, most programs do
> not define new operators.
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#inconsistency>
> Inconsistency
>
> Concrete discrepancies and edge cases motivate these proposed changes. For
> example:
>
> - The Greek question mark ; is a valid identifier.
> - Some *non-combining* diacritics ´ ¨ ꓻ are valid in identifiers.
> - Braille patterns ⠟, which are letter-like, are operator characters.
> - Other symbols such as ⚄ and ♄ are operator characters despite not
> being "operator-like."
> - Currency symbols are split across operators (¢ £ ¤ ¥) and
> identifiers (₪ € ₱ ₹ ฿ ...).
> - 🙂🤘▶️🛩 are identifiers, while ☹️✌️🔼✈️♠️ are operators.
> - A few characters 〡〢〣〤〥〦〧〨〩 〪 〫 〬 〭 〮 〯 are valid in both identifiers
> and operators.
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#invisible-distinctions>Invisible
> distinctions
>
> Identifiers that take advantage of Swift's Unicode support are not
> normalized. This allows different representations of the same characters to
> be considered distinct identifiers. For example:
>
> let Å = "Angstrom"
> let Å = "Latin Capital Letter A With Ring Above"
> let Å = "Latin Capital Letter A + Combining Ring Above"
>
> Non-printing characters such as ZERO WIDTH SPACE and ZERO WIDTH NON-JOINER
> are also accepted as valid identifier chracters without any restrictions.
>
> let ab = "ab"
> let ab = "a + ZERO WIDTH SPACE + b"
>
> func xy() { print("xy") }
> func xy() { print("x + ZERO WIDTH NON-JOINER + y") }
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#timeline>
> Timeline
>
> These matters should be considered in a near timeframe (Swift 4).
> Identifier and operator character sets are fundamental parts of Swift
> grammar, and changes are inevitably source-breaking.
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#non-goals>
> Non-goals
>
> The aim of this proposal is to rationalize the set of valid operator
> characters and the set of valid identifier characters using Unicode
> categories and specific Unicode recommendations where available. The
> smallest necessary customizations are made to increase backwards
> compatibility, but no attempt is made to expand Swift grammar or to
> "improve" Unicode. Specifically, the following questions are potential
> subjects of separate study, either within the purview of the Swift open
> source project or of the Unicode Consortium:
>
> -
>
> Expanding the set of valid operator or identifier characters. For
> example, $ is not currently a valid operator in Swift, there are no
> current Unicode recommendations regarding operators in programming
> languages, and $ is not enumerated among the list of "mathematical"
> characters in Unicode. Although is possible for Swift to customize its
> implementation of Unicode recommendations to add $ as a valid
> operator, that is an expansion of Swift grammar distinct from the task of
> rationalizing Swift symbology according to Unicode standards. Therefore,
> this document neither proposes nor opposes its addition. For similar
> reasons, this document refines the inclusion of emoji in identifiers based
> on Unicode categories, but it neither proposes nor opposes the inclusion of
> non-emoji pictographic symbols to the set of valid identifier characters.
> -
>
> Rectifying Unicode shortcomings. Although it is possible to discover
> shortcomings concerning particular characters in the current version of
> Unicode, no attempt is made to preempt the Unicode standardization process
> by "patching" such issues in the Swift grammar. For example, in the current
> version of Unicode, ⁗ QUADRUPLE PRIME is not deemed to be "mathematical"
> (even though ‴ TRIPLE PRIME *is* deemed to be "mathematical").
> Certainly, this issue would be appropriate to report to Unicode and may
> well be corrected in a future revision of the standard. However, as the
> Swift community is not congruent with the community of experts that
> specialize in Unicode, there is no rational basis to expect that Swift-only
> determinations of what Unicode "should have done" (without vetting through
> Unicode's standardization processes) are likely to result in a better
> outcome than the existing Unicode standard. Therefore, no attempt is made
> to augment the Unicode derived category Math with ⁗ QUADRUPLE PRIME in
> this proposal. Similarly, Unicode recommends certain normalization forms
> for identifiers in code, which are proposed here for adoption by Swift, but
> these normalization forms do not eliminate all possible combinations of
> "confusable" characters. This proposal does not attempt to invent an ad-hoc
> normalization form in an attempt to "improve" Unicode recommendations.
> -
>
> Implementing additional features. Innovative ideas such as mixfix operators
> are detailed below in *Future directions*. This proposal does not
> attempt to introduce any such features.
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#precedent-in-other-languages>Precedent
> in other languages
>
> Haskell distinguishes identifiers/operators by their general category
> <http://www.fileformat.info/info/unicode/category/index.htm> (for
> instance, "any Unicode lowercase letter" or "any Unicode symbol or
> punctuation"). Identifiers can start with any lowercase letter or _, and
> they may contain any letter, digit, ', or _. This includes letters like δ
> and Я, and digits like ٢.
>
> - Haskell Syntax Reference
> <https://www.haskell.org/onlinereport/syntax-iso.html>
> - Haskell Lexer
> <https://github.com/ghc/ghc/blob/714bebff44076061d0a719c4eda2cfd213b7ac3d/compiler/parser/Lexer.x#L1949-L1973>
>
> Scala similarly allows letters, numbers, $, and _ in identifiers,
> distinguishing by general categories Ll, Lu, Lt, Lo, and Nl. Operator
> characters include mathematical and other symbols (Sm and So) in addition
> to certain ASCII characters.
>
> - Scala Lexical Syntax
> <http://www.scala-lang.org/files/archive/spec/2.11/01-lexical-syntax.html#lexical-syntax>
>
> ECMAScript 2015 uses ID_Start and ID_Continue, as well as Other_ID_Start
> and Other_ID_Continue, for identifiers.
>
> - ECMAScript Specification: Names and Keywords
> <http://www.ecma-international.org/ecma-262/6.0/#sec-names-and-keywords>
>
> Python 3 uses XID_Start and XID_Continue.
>
> - The Python Language Reference: Identifiers and Keywords
> <https://docs.python.org/3/reference/lexical_analysis.html#grammar-token-identifier>
> - PEP 3131: Supporting Non-ASCII Identifiers
> <https://www.python.org/dev/peps/pep-3131/>
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#proposed-solution>Proposed
> solution
>
> Identifiers. Adopt recommendations made in UAX#31 Identifier and Pattern
> Syntax <http://unicode.org/reports/tr31/>, deriving the sets of valid
> identifier characters from ID_Start and ID_Continue. Adopt specific
> customizations principally to accommodate emoji. Consider two identifiers
> equivalent when they produce the same normalized form under Normalization
> Form C (NFC) <http://unicode.org/reports/tr15/>, as recommended in UAX#31
> for case-sensitive use cases.
> Is an identifierIs not an identifier
> Shall be an identifier 120,617 code points
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%0D%0A%26+%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%5D&g=&i=> 699
> emoji
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%0D%0A-%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%5D&g=&i=>
> Shall not be an identifier 846,137 unassigned code points;
> 4,929 other code points
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%0D%0A-%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%5D&g=&i=> *All
> other code points*
>
> Operators. No Unicode recommendation currently exists on the topic of
> "operator identifiers," although work is ongoing as part of a future update
> to UAX#31. The aim of the proposed definition presented in this document is
> to identify, using Unicode categories, a reasonable set of operators that
> (a) may be in current use in Swift code; and (b) are likely to be included
> in future versions of UAX#31. It is not intended to be a final judgment on
> all code points that should ever be valid in Swift operators, for which it
> is proposed that Swift await the recommendations of the Unicode Consortium.
>
> Therefore, adopt an approach to define the set of valid operator
> characters based primarily on the Unicode categories Math and Pattern_
> Syntax (an approach analogous to that which is used to define ID_Start
> and ID_Continue in Unicode recommendations), informed by UAX#25 Unicode
> Support for Mathematics <http://www.unicode.org/reports/tr25/>. Augment
> the set of valid operator characters with a number of currently valid Swift
> operator characters to increase backward compatibility. Consider two
> operators equivalent when they produce the same normalized form under Normalization
> Form KC (NFKC) <http://unicode.org/reports/tr15/>, as recommended in
> UAX#31 for case-insensitive use cases. Fullwidth variants such as FULLWIDTH
> HYPHEN-MINUS are equivalent to their non-fullwidth counterparts after
> normalization under NFKC (but not NFC).
> Is an operatorIs not an operator
> Shall be an operator 986 code points
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5b%5b%3aPattern%5fSyntax%3a%5d%20%26%20%5b%3aMath%3a%5d%0d%0a%2d%20%5b%3aBlock%3dGeometric%20Shapes%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Symbols%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Technical%3a%5d%0d%0a%5b%21%20%25%20%5c%26%20%2a%20%5c%2d%20%2f%20%3f%20%5c%5c%20%5c%5e%20%C2%A1%20%C2%A6%20%C2%A7%20%C2%B0%20%C2%B6%20%C2%BF%20%E2%80%A0%20%E2%80%A1%20%E2%80%A2%20%E2%80%B0%20%E2%80%B1%20%E2%80%BB%20%E2%80%BD%20%E2%81%82%20%E2%81%85%20%E2%81%86%20%E2%81%8A%20%E2%81%8B%20%E2%81%8C%20%E2%81%8D%20%E2%81%8E%20%E2%81%91%5d%5d%26%5b%5b%0d%0a%5b%2f%20%5c%2d%20%2b%20%21%20%2a%20%25%20%3c%2d%3e%20%5c%26%20%7c%20%5c%5e%20%7e%20%3f%5d%0d%0aU%2b00A1%2dU%2b00A7%0d%0aU%2b00A9%20U%2b00AB%0d%0aU%2b00AC%20U%2b00AE%0d%0aU%2b00B0%2dU%2b00B1%20U%2b00B6%20U%2b00BB%20U%2b00BF%20U%2b00D7%20U%2b00F7%0d%0aU%2b2016%2dU%2b2017%20U%2b2020%2dU%2b2027%0d%0aU%2b2030%2dU%2b203E%0d%0aU%2b2041%2dU%2b2053%0d%0aU%2b2055%2dU%2b205E%0d%0aU%2b2190%2dU%2b23FF%0d%0aU%2b2500%2dU%2b2775%0d%0aU%2b2794%2dU%2b2BFF%0d%0aU%2b2E00%2dU%2b2E7F%0d%0aU%2b3001%2dU%2b3003%0d%0aU%2b3008%2dU%2b3030%0d%0a%5d%0d%0a%5b%0d%0aU%2b0300%2dU%2b036F%0d%0aU%2b1DC0%2dU%2b1DFF%0d%0aU%2b20D0%2dU%2b20FF%0d%0aU%2bFE00%2dU%2bFE0F%0d%0aU%2bFE20%2dU%2bFE2F%0d%0aU%2bE0100%2dU%2bE01EF%0d%0a%5d%5d%5D>
> \
> Shall not be an operator 130 unassigned code points;
> 2,024 other code points
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5b%5b%0d%0a%5b%2f%20%5c%2d%20%2b%20%21%20%2a%20%25%20%3c%2d%3e%20%5c%26%20%7c%20%5c%5e%20%7e%20%3f%5d%0d%0aU%2b00A1%2dU%2b00A7%0d%0aU%2b00A9%20U%2b00AB%0d%0aU%2b00AC%20U%2b00AE%0d%0aU%2b00B0%2dU%2b00B1%20U%2b00B6%20U%2b00BB%20U%2b00BF%20U%2b00D7%20U%2b00F7%0d%0aU%2b2016%2dU%2b2017%20U%2b2020%2dU%2b2027%0d%0aU%2b2030%2dU%2b203E%0d%0aU%2b2041%2dU%2b2053%0d%0aU%2b2055%2dU%2b205E%0d%0aU%2b2190%2dU%2b23FF%0d%0aU%2b2500%2dU%2b2775%0d%0aU%2b2794%2dU%2b2BFF%0d%0aU%2b2E00%2dU%2b2E7F%0d%0aU%2b3001%2dU%2b3003%0d%0aU%2b3008%2dU%2b3030%0d%0a%5d%0d%0a%5b%0d%0aU%2b0300%2dU%2b036F%0d%0aU%2b1DC0%2dU%2b1DFF%0d%0aU%2b20D0%2dU%2b20FF%0d%0aU%2bFE00%2dU%2bFE0F%0d%0aU%2bFE20%2dU%2bFE2F%0d%0aU%2bE0100%2dU%2bE01EF%0d%0a%5d%5d-%5b%5b%3aPattern%5fSyntax%3a%5d%20%26%20%5b%3aMath%3a%5d%0d%0a%2d%20%5b%3aBlock%3dGeometric%20Shapes%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Symbols%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Technical%3a%5d%0d%0a%5b%21%20%25%20%5c%26%20%2a%20%5c%2d%20%2f%20%3f%20%5c%5c%20%5c%5e%20%C2%A1%20%C2%A6%20%C2%A7%20%C2%B0%20%C2%B6%20%C2%BF%20%E2%80%A0%20%E2%80%A1%20%E2%80%A2%20%E2%80%B0%20%E2%80%B1%20%E2%80%BB%20%E2%80%BD%20%E2%81%82%20%E2%81%85%20%E2%81%86%20%E2%81%8A%20%E2%81%8B%20%E2%81%8C%20%E2%81%8D%20%E2%81%8E%20%E2%81%91%5d%5d%5D> *All
> other code points*
>
> Dots. Adopt a rule to allow dots to appear in operators at any location,
> but only in runs of two or more. (Currently, dots must be leading.)
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#detailed-design>Detailed
> design
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#identifiers>
> Identifiers
>
> Swift identifier characters shall conform to UAX#31
> <http://unicode.org/reports/tr31/#Conformance> as follows:
>
> -
>
> UAX31-C1. <http://unicode.org/reports/tr31/#C1> The conformance
> described herein refers to the Unicode 9.0.0 version of UAX#31.
> -
>
> UAX31-C2. <http://unicode.org/reports/tr31/#C2> Swift shall observe
> the following requirements:
> -
>
> UAX31-R1. <http://unicode.org/reports/tr31/#R1> Swift shall augment
> the definition of "Default Identifiers" with the following profiles:
> 1.
>
> ID_Start and ID_Continue shall be used for Start and Continue,
> replacing XID_Start and XID_Continue. This excludes characters
> in Other_ID_Start and Other_ID_Continue.
> 2.
>
> _ 005F LOW LINE shall additionally be allowed as a Start
> character.
> 3.
>
> Certain emoji shall additionally be allowed as Start characters.
> A detailed design for emoji permitted in identifiers is given below.
> 4.
>
> UAX31-R1a. <http://unicode.org/reports/tr31/#R1a> The
> join-control characters ZWJ and ZWNJ are strictly limited to the special
> cases A1, A2, and B described in UAX#31.
> -
>
> UAX31-R4. <http://unicode.org/reports/tr31/#R4> Swift shall
> consider two identifiers equivalent when they produce the same normalized
> form under Normalization Form C (NFC)
> <http://unicode.org/reports/tr15/>, as recommended in UAX#31 for
> case-sensitive use cases.
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes>Grammar
> changes
>
> identifier-head → [:ID_Start:]
> identifier-head → _
> identifier-head → identifier-emoji
> identifier-character → identifier-head
> identifier-character → [:ID_Continue:]
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#operators>
> Operators
>
> Swift operator characters shall be determined as follows:
>
> -
>
> Valid operator characters shall consist of Pattern_Syntax code points
> with a derived property Math. However, the following blocks are
> excluded: Geometric Shapes, Miscellaneous Symbols, and Miscellaneous
> Technical. In UnicodeSet notation:
>
> [:Pattern_Syntax:] & [:Math:]
> - [:Block=Geometric Shapes:]
> - [:Block=Miscellaneous Symbols:]
> - [:Block=Miscellaneous Technical:]
>
> Math captures a fuller set of operators than is possible using Sm, and
> we avoid the inclusion of characters in So that are clearly not
> "operator-like" (such as Braille). Math code points in the excluded
> blocks include sign parts such as ⎲ SUMMATION TOP and tenuously
> "operator-like" code points such as ♠️ BLACK SPADE SUIT.
> -
>
> The set of valid operator characters shall be augmented with the
> following ASCII characters: !, %, &, *, -, /, ?, \, ^. These ASCII
> characters are required by the Swift standard library and/or considered
> "weakly mathematical" in UAX#25 <http://www.unicode.org/reports/tr25/>.
> -
>
> For increased compatibility with Swift 3, the set of valid operator
> characters shall be augmented with the following Latin-1 Supplement
> characters: ¡, ¦, §, °, ¶, ¿. For the same reason, augment the set of
> valid operator characters with the following General Punctuation
> characters: † DAGGER, ‡ DOUBLE DAGGER, • BULLET, ‰ PER MILLE SIGN, ‱ PER
> TEN THOUSAND SIGN, ※ REFERENCE MARK, ‽ INTERROBANG, ⁂ ASTERISM, ⁅ LEFT
> SQUARE BRACKET WITH QUILL, ⁆ RIGHT SQUARE BRACKET WITH QUILL, ⁊ TIRONIAN
> SIGN ET, ⁋ REVERSED PILCROW SIGN, ⁌ BLACK LEFTWARDS BULLET, ⁍ BLACK
> RIGHTWARDS BULLET, ⁎ LOW ASTERISK, ⁑ TWO ASTERISKS ALIGNED VERTICALLY.
> -
>
> Swift shall consider two operators equivalent when they produce the
> same normalized form under Normalization Form KC (NFKC)
> <http://unicode.org/reports/tr15/>, as recommended in UAX#31 for
> *case-insensitive* use cases. Crucially, fullwidth variants such as
> FULLWIDTH HYPHEN-MINUS are equivalent to their non-fullwidth counterparts
> after normalization under NFKC (but not NFC).
> -
>
> Certainly strongly mathematical arrows now have an *alternative* emoji
> presentation, and future versions of Unicode may add such an emoji
> presentation to any Swift operator character. Some but not all
> "environments" or applications (for instance, Safari but not TextWrangler)
> display the alternative emoji presentation at all times, and such
> discrepancies between applications are explicitly permitted by Unicode
> recommendations (see dicussion in *Emoji*). However, it would be
> highly unusual to define the set of valid operator characters based on an
> essentially arbitrary criterion as to whether an alternative emoji
> presentation is retroactively assigned to a code point, and codifying how
> IDEs display Unicode characters in Swift files is outside the scope of this
> proposal. Therefore, valid operator characters are defined without regard
> to the presence or absence of an alternative emoji presentation, and U+FE0E
> VARIATION SELECTOR-15 (text presentation selector) is *optionally* permitted
> to follow an operator character that has an alternative emoji presentation.
> Note that variation selectors are discarded by normalization.
>
> These revised rules
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%3APattern_Syntax%3A%5D+%26+%5B%3AMath%3A%5D%0D%0A-+%5B%3ABlock%3DGeometric+Shapes%3A%5D%0D%0A-+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%0D%0A-+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%0D%0A%5B%21+%25+%5C%26+*+%5C-+%2F+%3F+%5C%5C+%5C%5E+%C2%A1+%C2%A6+%C2%A7+%C2%B0+%C2%B6+%C2%BF+%E2%80%A0+%E2%80%A1+%E2%80%A2+%E2%80%B0+%E2%80%B1+%E2%80%BB+%E2%80%BD+%E2%81%82+%E2%81%85+%E2%81%86+%E2%81%8A+%E2%81%8B+%E2%81%8C+%E2%81%8D+%E2%81%8E+%E2%81%91%5D&g=&i=> produce
> a set of 987 code points for operator characters. Since ID_Start is
> derived in part by exclusion of Pattern_Syntax code points, it is assured
> that operator and identifier characters do not overlap (although this
> assurance does not extend to emoji, which require additional design as
> detailed below).
>
> All current restrictions on reserved tokens and operators remain. Swift
> reserves =, ->, //, /*, */, ., ?, prefix <, prefix &, postfix >, and
> postfix !.
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#dots>Dots
>
> Swift's existing rule for dots in operators is:
>
> If an operator doesn’t begin with a dot, it can’t contain a dot elsewhere.
>
> This proposal modifies the rule to:
>
> Dots may only appear in operators in sequences of two or more.
>
> Incorporating the "two-dot rule" offers the following benefits:
>
> -
>
> It avoids lexical complications arising from lone ..
> -
>
> The approach is conservative, erring on the side of overly
> restrictive. Dropping the rule in future (and thereby allowing single dots)
> may be possible.
> -
>
> It does not require special cases for existing infix dot operators in
> the standard library, ... (closed range) and ..<(half-open range). It
> leaves open the possibility of adding analogous half-open and fully-open
> range operators <..and <..<.
>
> Finally, this proposal *reserves* the .. operator for a possible "method
> cascade" syntax in the future as supported by Dart
> <http://news.dartlang.org/2012/02/method-cascades-in-dart-posted-by-gilad.html>
> .
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes-1>Grammar
> changes
>
> operator → operator-head operator-characters[opt]
>
> operator-head → [[:Pattern_Syntax:] & [:Math:] - [:Emoji:] - [:Block=Geometric Shapes:] - [:Block=Miscellaneous Symbols:] - [:Block=Miscellaneous Technical:]]
> operator-head → [[:Pattern_Syntax:] & [:Math:] & [:Emoji:] - [:Block=Geometric Shapes:] - [:Block=Miscellaneous Symbols:] - [:Block=Miscellaneous Technical:]] U+FE0E[opt]
> operator-head → ! | % | & | * | - | / | ? | \ | ^ | ¡ | ¦ | § | ° | ¶ | ¿
> operator-head → † | ‡ | • | ‰ | ‱ | ※ | ‽ | ⁂ | ⁅ | ⁆ | ⁊ | ⁋ | ⁌ | ⁍ | ⁎ | ⁑
> operator-head → operator-dot operator-dots
> operator-character → operator-head
> operator-characters → operator-character operator-character[opt]
>
> operator-dot → .
> operator-dots → operator-dot operator-dots[opt]
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#emoji>Emoji
>
> The inclusion of emoji among valid identifier characters, though highly
> desired, presents significant challenges:
>
> -
>
> Emoji characters are not displayed uniformly across different
> platforms.
> -
>
> Whether any particular character is presented as emoji or text depends
> on a matrix of considerations, including "environment" (e.g., Safari vs.
> XCode), presence or absence of a variant selector, and whether the
> character itself defaults to "emoji presentation" or "text presentation."
> This behavior is specifically documented in Unicode recommendations
> <http://unicode.org/reports/tr51/#Presentation_Style>.
> -
>
> Some emoji not classified as Math depict operators: ❗️❓➕➖➗✖️. A
> Unicode chart <http://unicode.org/emoji/charts/emoji-ordering.html> provides
> additional information by dividing emoji according to "rough categories,"
> but it warns that these categories "may change at any time, and should not
> be used in production."
> -
>
> Full emoji support would require allowing identifiers to contain
> zero-width joiner sequences that UAX#31 would forbid. Some normalization
> scheme would have to be devised to account for Unicode recommendations that
> 👩❤️👨 (U+1F469 U+200D U+2764 U+FE0F U+200D U+1F468) can be
> displayed as either 💑 (U+1F491) or, as a fallback, 👩❤️👨(U+1F469
> U+2764 U+FE0F U+1F468).
>
> For maximum consistency across platforms, valid emoji in Swift identifiers
> shall be determined using the following rules:
>
> -
>
> Emoji shall include code points with default emoji presentation (as
> opposed to text presentation), minus Emoji_Defectives and ID_Continue.
> Exclude Pattern_Syntax code points unless they are in the following
> blocks: Miscellaneous Symbols, Miscellaneous Technical.
> -
>
> Emoji shall include Emoji code points with default text presentation *when
> immediately followed by U+FE0F VARIATION SELECTOR-16 (emoji presentation
> selector)*, minus Emoji_Defectives and ID_Continue. Again, exclude
> Pattern_Syntax code points unless they are in the following blocks:
> Miscellaneous Symbols, Miscellaneous Technical. (Note that the emoji picker
> on Apple platforms--and, possibly, other platforms--automatically inserts
> U+FE0F VARIATION SELECTOR-16 when a user selects such code points; for
> instance, selecting ❤️ inserts U+2764 U+FE0F. Therefore, it is
> important that the invisible U+FE0F be permitted strictly in this use case.
> Note also that variation selectors are discarded by normalization.)
> -
>
> Emoji shall include Emoji_Flag_Sequences, Emoji_Keycap_Sequences, and
> (to the extent not already included) Emoji_Modifier_Sequences.
>
> These revised rules
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%3AEmoji_Flag_Sequences%3A%5D%0D%0A%5B%3AEmoji_Keycap_Sequences%3A%5D%0D%0A%5B%3AEmoji_Modifier_Sequences%3A%5D&g=&i=> produce
> a set of 1,625 code points or sequences, of which 98 are currently
> categorized as operator characters.
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes-2>Grammar
> changes
>
> identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] - [:ID_Continue:] - [:Pattern_Syntax:]]
> identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] - [:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous Symbols:]]
> identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] - [:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous Technical:]]
> identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] - [:Emoji_Presentation:] - [:ID_Continue:] - [:Pattern_Syntax:]] U+FE0F
> identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] - [:Emoji_Presentation:] - [:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous Symbols:]] U+FE0F
> identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] - [:Emoji_Presentation:] - [:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous Technical:]] U+FE0F
> identifier-emoji → [[:Emoji_Flag_Sequences:] [:Emoji_Keycap_Sequences:] [:Emoji_Modifier_Sequences:]]
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#source-compatibility>Source
> compatibility
>
> This change is source-breaking where developers have incorporated certain
> emoji in identifiers or certain non-ASCII characters in operators. This is
> unlikely to be a significant breakage for the majority of Swift code.
> Diagnostics for invalid characters are already produced today. We can
> improve them easily if needed.
>
> Maintaining source compatibility for Swift 3 should be easy: keep the old
> parsing and identifier lookup code.
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#effect-on-abi-stability>Effect
> on ABI stability
>
> This proposal does not affect the ABI format itself. Normalization of
> Unicode identifiers would affect the ABI of compiled modules. The standard
> library will not be affected; it uses ASCII symbols with no combining
> characters.
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#effect-on-api-resilience>Effect
> on API resilience
>
> This proposal doesn't affect API resilience.
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#alternatives-considered>Alternatives
> considered
>
> -
>
> Use NFKC instead of NFC for identifiers. The decision to use NFC is
> based on UAX#31, which states:
>
> Generally if the programming language has case-sensitive identifiers,
> then Normalization Form C is appropriate; whereas, if the programming
> language has case-insensitive identifiers, then Normalization Form KC is
> more appropriate.
>
> -
>
> Eliminate emoji from identifiers and restrict operator characters to a
> limited number of ASCII code points. This approach would be simpler, but
> feedback on Swift-Evolution has been overwhelmingly against such a change.
> -
>
> Hand-pick a set of "operator-like" characters to include. The proposal
> authors tried this painstaking approach and came up with a relatively
> agreeable set of about 650 code points
> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%21%5C%24%25%5C%26*%2B%5C-%2F%3C%3D%3E%3F%5C%5E%7C%7E%0D%0A%0D%0A%5Cu00AC%0D%0A%5Cu00B1%0D%0A%5Cu00B7%0D%0A%5Cu00D7%0D%0A%5Cu00F7%0D%0A%0D%0A%5Cu2208-%5Cu220D%0D%0A%5Cu220F-%5Cu2211%0D%0A%5Cu22C0-%5Cu22C3%0D%0A%5Cu2212-%5Cu221D%0D%0A%5Cu2238%0D%0A%5Cu223A%0D%0A%5Cu2240%0D%0A%5Cu228C-%5Cu228E%0D%0A%5Cu2293-%5Cu22A3%0D%0A%5Cu22BA-%5Cu22BD%0D%0A%5Cu22C4-%5Cu22C7%0D%0A%5Cu22C9-%5Cu22CC%0D%0A%5Cu22D2-%5Cu22D3%0D%0A%5Cu2223-%5Cu222A%0D%0A%5Cu2236-%5Cu2237%0D%0A%5Cu2239%0D%0A%5Cu223B-%5Cu223E%0D%0A%5Cu2241-%5Cu228B%0D%0A%5Cu228F-%5Cu2292%0D%0A%5Cu22A6-%5Cu22B9%0D%0A%5Cu22C8%0D%0A%5Cu22CD%0D%0A%5Cu22D0-%5Cu22D1%0D%0A%5Cu22D4-%5Cu22FF%0D%0A%5Cu22CE-%5Cu22CF%0D%0A%0D%0A%5Cu2A00-%5Cu2AFF%0D%0A%0D%0A%5Cu27C2%0D%0A%5Cu27C3%0D%0A%5Cu27C4%0D%0A%5Cu27C7%0D%0A%5Cu27C8%0D%0A%5Cu27C9%0D%0A%5Cu27CA%0D%0A%5Cu27CE-%5Cu27D7%0D%0A%5Cu27DA-%5Cu27DF%0D%0A%5Cu27E0-%5Cu27E5%0D%0A%0D%0A%5Cu29B5-%5Cu29C3%0D%0A%5Cu29C4-%5Cu29C9%0D%0A%5Cu29CA-%5Cu29D0%0D%0A%5Cu29D1-%5Cu29D7%0D%0A%5Cu29DF%0D%0A%5Cu29E1%0D%0A%5Cu29E2%0D%0A%5Cu29E3-%5Cu29E6%0D%0A%5Cu29FA%0D%0A%5Cu29FB%0D%0A%0D%0A%5Cu2308-%5Cu230B%0D%0A%5Cu2336-%5Cu237A%0D%0A%5Cu2395%5D>.
> Such a list can carefully avoid idiosyncrasies in the Unicode standard.
> However, a character-by-character inventory is unlikely to converge on
> consensus, as likely to introduce unintended Swift-specific idiosyncrasies
> as it is to avoid Unicode shortcomings, and inconsistent with the Unicode
> method of deriving such lists using categories.
> -
>
> Continue to allow single . in operators, perhaps even expanding the
> original rule to allow them anywhere (even if the operator does not begin
> with .).
>
> This would allow a wider variety of custom operators (for some
> interesting possibilities, see the operators in Haskell's Lens
> <https://github.com/ekmett/lens/wiki/Operators> package). However,
> there are a handful of potential complications:
> -
>
> Combining prefix or postfix operators with member access: foo*.bar would
> need to be parsed as foo *. barrather than (foo*).bar. Parentheses
> could be required to disambiguate.
> -
>
> Combining infix operators with contextual members: foo*.bar would
> need to be parsed as foo *. bar rather than foo * (.bar).
> Whitespace or parentheses could be required to disambiguate.
> -
>
> Hypothetically, if operators were accessible as members such as
> MyNumber.+, allowing operators with single .s would require
> escaping operator names (perhaps with backticks, such as
> MyNumber.`+`).
>
> This would also require operators of the form [!?]*\. (for example . ?.
> !. !!.) to be reserved, to prevent users from defining custom
> operators that conflict with member access and optional chaining.
>
> We believe that requiring dots to appear in groups of at least two,
> while in some ways more restrictive, will prevent a significant amount of
> future pain, and does not require special-case considerations such as the
> above.
>
>
> <https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#future-directions>Future
> directions
>
> While not within the scope of this proposal, the following considerations
> may provide useful context for the proposed changes. We encourage the
> community to pick up these topics when the time is right.
>
> -
>
> Introduce a syntax for method cascades. The Dart language supports method
> cascades
> <http://news.dartlang.org/2012/02/method-cascades-in-dart-posted-by-gilad.html>,
> whereby multiple methods can be called on an object within one expression:
> foo..bar()..baz() effectively performs foo.bar(); foo.baz(). This
> syntax can also be used with assignments and subscripts. Such a feature
> might be very useful in Swift; this proposal reserves the .. operator
> so that it may be added in the future.
> -
>
> Introduce "mixfix" operator declarations. Mixfix operators are based
> on pattern matching and would allow more than two operands. For example,
> the ternary operator ? : can be defined as a mixfix operator with
> three "holes": _ ? _ : _. Subscripts might be subsumed by mixfix
> declarations such as _ [ _ ]. Some holes could be made @autoclosure,
> and there might even be holes whose argument is represented as an AST,
> rather than a value or thunk, supporting advanced metaprogramming (for
> instance, F#'s code quotations
> <https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/code-quotations>).
> Should mixfix operators become supported, it would be sensible to add
> brackets to the set of valid operator characters.
> -
>
> Diminish or remove the lexical distinction between operators and
> identifiers. If precedence and fixity applied to traditional
> identifiers as well as operators, it would be possible to incorporate ASCII
> equivalents for standard operators (e.g. and for &&, to allow A and B).
> If additionally combined with mixfix operator support, this might enable
> powerful DSLs (for instance, C#'s LINQ
> <https://en.wikipedia.org/wiki/Language_Integrated_Query>).
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170808/d8c9bae2/attachment.html>
More information about the swift-evolution
mailing list