[swift-evolution] [Draft] Refining identifier and operator symbology (take 2)

Xiaodi Wu xiaodi.wu at gmail.com
Thu Feb 16 23:50:47 CST 2017


As Stage 2 of Swift 4 evolution starts now, I'd like to share a revised
proposal in draft form.

It proposes a source-breaking change for *rationalizing* which characters
are permitted in identifiers and which in operators. It's justified for
this phase of Swift 4 because:

- Existing grammar, in permitting invisible characters without
security-minded restrictions, can be *actively harmful.*
- A rationalized approach is *superior* to the current approach: by
referencing Unicode standards, Swift should be able to evolve in a
backwards-compatible way alongside Unicode, and will benefit from the
significant expertise of others outside the Swift community with respect to
Unicode best practices.
- The vast majority of existing code (including all of the standard
library) should *require no migration* work at all

*What's changed* since the last time:

- In an earlier draft, we proposed some radical changes to align with
available Unicode standards; in particular, since emoji represent a
difficult issue, and no recommendations about "operator identifiers" have
surfaced from Unicode, we proposed temporarily stripping them out.
This was *very
poorly received*. This revision uses Unicode categories to identify nearly
all emoji and classify them as identifier characters (while excluding those
that depict operators such as !), and it uses Unicode categories to
identify over 900 operators that nearly all pass the subjective test of
"operator-likeness."

What this proposal *does not attempt* to do:

- This document *does not* seek to stake out new ground as to what
characters should be *added* to the set of valid identifiers and operators.
Such additions to the grammar are properly separate discussions. This
proposal is only an attempt at systemization and rationalization. Only one
character is incidentally added to the list of valid characters (`\`), and
it is on the basis of an explicit table in Unicode Technical Report 25
regarding ASCII characters that are "mathematical."

What feedback would be* most helpful*:

- "Hey, this approach is so much more *clumsy* than my superior, more
elegant category-based approach to identifying [operators/emoji], which is
[insert here]."
- "Hey, I disagree with the detailed design because it's got a *major
security hole*, which is [insert here]."
- "Hey, your proposal would break my *real-world* Swift code, which
requires that character [X] be an [identifier/operator]."

What would be *less helpful*:

- "Hey, let's talk about how [specific character] should be an
[identifier/operator]. We should add that character to the list of
[identifiers/operators]. In fact, let's discuss [list] characters one by
one."

Acknowledgments:
Thanks to co-authors of the previous take for their support for
resurrecting this issue. Any brilliant ideas are undoubtedly theirs, and
any botched efforts are certainly mine. Thanks also to Nevin
Brackett-Rozinsky for helpful feedback.

Link:
https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651


Rendered text:

Refining identifier and operator symbology (take 2)

   - Proposal: SE-NNNN
   <https://gist.github.com/xwu/NNNN-refining-identifier-and-operator-symbology.md>
   - Authors: Xiaodi Wu <https://github.com/xwu>, Jacob Bandes-Storch
   <https://github.com/jtbandes>, Erica Sadun <https://github.com/erica>,
   Jonathan Shapiro, João Pinheiro <https://github.com/joaopinheiro>
   - Review Manager: TBD
   - Status: Awaiting review

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#introduction>
Introduction

This proposal refines and rationalizes Swift's identifier and operator
symbology. Specifically, this proposal:

   - refines the set of valid identifier characters based on Unicode
   recommendations, with customizations principally to accommodate emoji;
   - refines the set of valid operator characters based on Unicode
   categories; and
   - changes rules as to where dots may appear in operators.

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#prior-discussion-threads-and-proposals>Prior
discussion threads and proposals

   - Define backslash '\' as a operator-head in the swift grammar
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170130/031461.html>
   - Refining Identifier and Operator Symbology
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161017/028174.html>
(a
   precursor to this document)
   - Proposal: Normalize Unicode identifiers
   <https://github.com/apple/swift-evolution/pull/531>
   - Lexical matters: identifiers and operators
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160926/027479.html>
   - Unicode identifiers & operators
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160912/027108.html>,
   with pre-proposal
   <https://gist.github.com/jtbandes/c0b0c072181dcd22c3147802025d0b59>
   - Proposal: Allow Single Dollar Sign as Valid Identifier
   <https://github.com/apple/swift-evolution/pull/354>
   - Free the '$' Symbol!
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151228/005133.html>
   - Request to add middle dot (U+00B7) as operator character?
   <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151214/003176.html>

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#motivation>
Motivation

Swift supports programmers from many languages and cultures. However, the
current identifier and operator character sets do not conform to any
Unicode standards, nor have they been rationalized in the language or
compiler documentation. These deserve a well-considered, standards-based
revision.

As Chris Lattner has written:

We need a token to be unambiguously an operator or identifier - we can have
different rules for the leading and subsequent characters though.

…our current operator space (particularly the Unicode segments covered) is
not super well considered. It would be great for someone to take a more
systematic pass over them to rationalize things.

Identifiers, which serve as *names* for various entities, are linguistic in
nature and must permit a variety of characters in order to properly serve
non–English-speaking coders. This issue has been considered by the
communities of many programming languages already, and the Unicode
Consortium has published recommendations on how to choose identifier
character sets. Swift should make an effort to conform to these
recommendations.

Operators, on the other hand, should be rare and carefully chosen because
they suffer from limited discoverability and readability. They are by
nature *symbols*, not names. This places a cognitive cost on users with
respect to recall ("What is the operator that applies the behavior I
need?") and recognition ("What does the operator in this code do?"). While
almost every non-trivial program defines new identifiers, most programs do
not define new operators.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#inconsistency>
Inconsistency

Concrete discrepancies and edge cases motivate these proposed changes. For
example:

   - The Greek question mark ; is a valid identifier.
   - Some *non-combining* diacritics ´ ¨ ꓻ are valid in identifiers.
   - Braille patterns ⠟, which are letter-like, are operator characters.
   - Other symbols such as ⚄ and ♄ are operator characters despite not
   being "operator-like."
   - Currency symbols are split across operators (¢ £ ¤ ¥) and identifiers
   (₪ € ₱ ₹ ฿ ...).
   - 🙂🤘▶️🛩 are identifiers, while ☹️✌️🔼✈️♠️ are operators.
   - A few characters 〡〢〣〤〥〦〧〨〩 〪 〫 〬 〭 〮 〯 are valid in both identifiers
   and operators.

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#invisible-distinctions>Invisible
distinctions

Identifiers that take advantage of Swift's Unicode support are not
normalized. This allows different representations of the same characters to
be considered distinct identifiers. For example:

let Å = "Angstrom"
let Å = "Latin Capital Letter A With Ring Above"
let Å = "Latin Capital Letter A + Combining Ring Above"

Non-printing characters such as ZERO WIDTH SPACE and ZERO WIDTH NON-JOINER
are also accepted as valid identifier chracters without any restrictions.

let ab = "ab"
let a​b = "a + ZERO WIDTH SPACE + b"

func xy() { print("xy") }
func x‌y() { print("x + ZERO WIDTH NON-JOINER + y") }

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#timeline>
Timeline

These matters should be considered in a near timeframe (Swift 4).
Identifier and operator character sets are fundamental parts of Swift
grammar, and changes are inevitably source-breaking.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#non-goals>
Non-goals

The aim of this proposal is to rationalize the set of valid operator
characters and the set of valid identifier characters using Unicode
categories and specific Unicode recommendations where available. The
smallest necessary customizations are made to increase backwards
compatibility, but no attempt is made to expand Swift grammar or to
"improve" Unicode. Specifically, the following questions are potential
subjects of separate study, either within the purview of the Swift open
source project or of the Unicode Consortium:

   -

   Expanding the set of valid operator or identifier characters. For
   example, $ is not currently a valid operator in Swift, there are no
   current Unicode recommendations regarding operators in programming
   languages, and $ is not enumerated among the list of "mathematical"
   characters in Unicode. Although is possible for Swift to customize its
   implementation of Unicode recommendations to add $ as a valid operator,
   that is an expansion of Swift grammar distinct from the task of
   rationalizing Swift symbology according to Unicode standards. Therefore,
   this document neither proposes nor opposes its addition. For similar
   reasons, this document refines the inclusion of emoji in identifiers based
   on Unicode categories, but it neither proposes nor opposes the inclusion of
   non-emoji pictographic symbols to the set of valid identifier characters.
   -

   Rectifying Unicode shortcomings. Although it is possible to discover
   shortcomings concerning particular characters in the current version of
   Unicode, no attempt is made to preempt the Unicode standardization process
   by "patching" such issues in the Swift grammar. For example, in the current
   version of Unicode, ⁗ QUADRUPLE PRIME is not deemed to be "mathematical"
   (even though ‴ TRIPLE PRIME *is* deemed to be "mathematical").
   Certainly, this issue would be appropriate to report to Unicode and may
   well be corrected in a future revision of the standard. However, as the
   Swift community is not congruent with the community of experts that
   specialize in Unicode, there is no rational basis to expect that Swift-only
   determinations of what Unicode "should have done" (without vetting through
   Unicode's standardization processes) are likely to result in a better
   outcome than the existing Unicode standard. Therefore, no attempt is made
   to augment the Unicode derived category Math with ⁗ QUADRUPLE PRIME in
   this proposal. Similarly, Unicode recommends certain normalization forms
   for identifiers in code, which are proposed here for adoption by Swift, but
   these normalization forms do not eliminate all possible combinations of
   "confusable" characters. This proposal does not attempt to invent an ad-hoc
   normalization form in an attempt to "improve" Unicode recommendations.
   -

   Implementing additional features. Innovative ideas such as mixfix operators
   are detailed below in *Future directions*. This proposal does not
   attempt to introduce any such features.

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#precedent-in-other-languages>Precedent
in other languages

Haskell distinguishes identifiers/operators by their general category
<http://www.fileformat.info/info/unicode/category/index.htm> (for instance,
"any Unicode lowercase letter" or "any Unicode symbol or punctuation").
Identifiers can start with any lowercase letter or _, and they may contain
any letter, digit, ', or _. This includes letters like δ and Я, and digits
like ٢.

   - Haskell Syntax Reference
   <https://www.haskell.org/onlinereport/syntax-iso.html>
   - Haskell Lexer
   <https://github.com/ghc/ghc/blob/714bebff44076061d0a719c4eda2cfd213b7ac3d/compiler/parser/Lexer.x#L1949-L1973>

Scala similarly allows letters, numbers, $, and _ in identifiers,
distinguishing by general categories Ll, Lu, Lt, Lo, and Nl. Operator
characters include mathematical and other symbols (Sm and So) in addition
to certain ASCII characters.

   - Scala Lexical Syntax
   <http://www.scala-lang.org/files/archive/spec/2.11/01-lexical-syntax.html#lexical-syntax>

ECMAScript 2015 uses ID_Start and ID_Continue, as well as Other_ID_Start
 and Other_ID_Continue, for identifiers.

   - ECMAScript Specification: Names and Keywords
   <http://www.ecma-international.org/ecma-262/6.0/#sec-names-and-keywords>

Python 3 uses XID_Start and XID_Continue.

   - The Python Language Reference: Identifiers and Keywords
   <https://docs.python.org/3/reference/lexical_analysis.html#grammar-token-identifier>
   - PEP 3131: Supporting Non-ASCII Identifiers
   <https://www.python.org/dev/peps/pep-3131/>

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#proposed-solution>Proposed
solution

Identifiers. Adopt recommendations made in UAX#31 Identifier and Pattern
Syntax <http://unicode.org/reports/tr31/>, deriving the sets of valid
identifier characters from ID_Start and ID_Continue. Adopt specific
customizations principally to accommodate emoji. Consider two identifiers
equivalent when they produce the same normalized form under Normalization
Form C (NFC) <http://unicode.org/reports/tr15/>, as recommended in UAX#31
for case-sensitive use cases.
Is an identifierIs not an identifier
Shall be an identifier 120,617 code points
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%0D%0A%26+%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%5D&g=&i=>
699
emoji
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%0D%0A-%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%5D&g=&i=>
Shall not be an identifier 846,137 unassigned code points;
4,929 other code points
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%5Ba-zA-Z%0D%0A_%0D%0A%5Cu00A8%0D%0A%5Cu00AA%0D%0A%5Cu00AD%0D%0A%5Cu00AF%0D%0A%5Cu00B2-%5Cu00B5%0D%0A%5Cu00B7-%5Cu00BA%0D%0A%5Cu00BC-%5Cu00BE%0D%0A%5Cu00C0-%5Cu00D6%0D%0A%5Cu00D8-%5Cu00F6%0D%0A%5Cu00F8-%5Cu00FF%0D%0A%5Cu0100-%5Cu02FF%0D%0A%5Cu0370-%5Cu167F%0D%0A%5Cu1681-%5Cu180D%0D%0A%5Cu180F-%5Cu1DBF%0D%0A%5Cu1E00-%5Cu1FFF%0D%0A%5Cu200B-%5Cu200D%0D%0A%5Cu202A-%5Cu202E%0D%0A%5Cu203F-%5Cu2040%0D%0A%5Cu2054%0D%0A%5Cu2060-%5Cu206F%0D%0A%5Cu2070-%5Cu20CF%0D%0A%5Cu2100-%5Cu218F%0D%0A%5Cu2460-%5Cu24FF%0D%0A%5Cu2776-%5Cu2793%0D%0A%5Cu2C00-%5Cu2DFF%0D%0A%5Cu2E80-%5Cu2FFF%0D%0A%5Cu3004-%5Cu3007%0D%0A%5Cu3021-%5Cu302F%0D%0A%5Cu3031-%5Cu303F%0D%0A%5Cu3040-%5CuD7FF%0D%0A%5CuF900-%5CuFD3D%0D%0A%5CuFD40-%5CuFDCF%0D%0A%5CuFDF0-%5CuFE1F%0D%0A%5CuFE30-%5CuFE44%0D%0A%5CuFE47-%5CuFFFD%0D%0A%5CU00010000-%5CU0001FFFD%0D%0A%5CU00020000-%5CU0002FFFD%0D%0A%5CU00030000-%5CU0003FFFD%0D%0A%5CU00040000-%5CU0004FFFD%0D%0A%5CU00050000-%5CU0005FFFD%0D%0A%5CU00060000-%5CU0006FFFD%0D%0A%5CU00070000-%5CU0007FFFD%0D%0A%5CU00080000-%5CU0008FFFD%0D%0A%5CU00090000-%5CU0009FFFD%0D%0A%5CU000A0000-%5CU000AFFFD%0D%0A%5CU000B0000-%5CU000BFFFD%0D%0A%5CU000C0000-%5CU000CFFFD%0D%0A%5CU000D0000-%5CU000DFFFD%0D%0A%5CU000E0000-%5CU000EFFFD%5D%0D%0A%5B0-9%0D%0A%5Cu0300-%5Cu036F%0D%0A%5Cu1DC0-%5Cu1DFF%0D%0A%5Cu20D0-%5Cu20FF%0D%0A%5CuFE20-%5CuFE2F%5D%5D%0D%0A-%5B%5B%3AID_Continue%3A%5D%0D%0A_%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Flag_Sequences%3A%5D+%5B%3AEmoji_Keycap_Sequences%3A%5D+%5B%3AEmoji_Modifier_Sequences%3A%5D%5D%5D%5D&g=&i=>
*All
other code points*

Operators. No Unicode recommendation currently exists on the topic of
"operator identifiers," although work is ongoing as part of a future update
to UAX#31. The aim of the proposed definition presented in this document is
to identify, using Unicode categories, a reasonable set of operators that
(a) may be in current use in Swift code; and (b) are likely to be included
in future versions of UAX#31. It is not intended to be a final judgment on
all code points that should ever be valid in Swift operators, for which it
is proposed that Swift await the recommendations of the Unicode Consortium.

Therefore, adopt an approach to define the set of valid operator characters
based primarily on the Unicode categories Math and Pattern_Syntax (an
approach analogous to that which is used to define ID_Start and ID_Continue in
Unicode recommendations), informed by UAX#25 Unicode Support for Mathematics
<http://www.unicode.org/reports/tr25/>. Augment the set of valid operator
characters with a number of currently valid Swift operator characters to
increase backward compatibility. Consider two operators equivalent when
they produce the same normalized form under Normalization Form KC (NFKC)
<http://unicode.org/reports/tr15/>, as recommended in UAX#31 for
case-insensitive use cases. Fullwidth variants such as FULLWIDTH
HYPHEN-MINUS are equivalent to their non-fullwidth counterparts after
normalization under NFKC (but not NFC).
Is an operatorIs not an operator
Shall be an operator 986 code points
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5b%5b%3aPattern%5fSyntax%3a%5d%20%26%20%5b%3aMath%3a%5d%0d%0a%2d%20%5b%3aBlock%3dGeometric%20Shapes%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Symbols%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Technical%3a%5d%0d%0a%5b%21%20%25%20%5c%26%20%2a%20%5c%2d%20%2f%20%3f%20%5c%5c%20%5c%5e%20%C2%A1%20%C2%A6%20%C2%A7%20%C2%B0%20%C2%B6%20%C2%BF%20%E2%80%A0%20%E2%80%A1%20%E2%80%A2%20%E2%80%B0%20%E2%80%B1%20%E2%80%BB%20%E2%80%BD%20%E2%81%82%20%E2%81%85%20%E2%81%86%20%E2%81%8A%20%E2%81%8B%20%E2%81%8C%20%E2%81%8D%20%E2%81%8E%20%E2%81%91%5d%5d%26%5b%5b%0d%0a%5b%2f%20%5c%2d%20%2b%20%21%20%2a%20%25%20%3c%2d%3e%20%5c%26%20%7c%20%5c%5e%20%7e%20%3f%5d%0d%0aU%2b00A1%2dU%2b00A7%0d%0aU%2b00A9%20U%2b00AB%0d%0aU%2b00AC%20U%2b00AE%0d%0aU%2b00B0%2dU%2b00B1%20U%2b00B6%20U%2b00BB%20U%2b00BF%20U%2b00D7%20U%2b00F7%0d%0aU%2b2016%2dU%2b2017%20U%2b2020%2dU%2b2027%0d%0aU%2b2030%2dU%2b203E%0d%0aU%2b2041%2dU%2b2053%0d%0aU%2b2055%2dU%2b205E%0d%0aU%2b2190%2dU%2b23FF%0d%0aU%2b2500%2dU%2b2775%0d%0aU%2b2794%2dU%2b2BFF%0d%0aU%2b2E00%2dU%2b2E7F%0d%0aU%2b3001%2dU%2b3003%0d%0aU%2b3008%2dU%2b3030%0d%0a%5d%0d%0a%5b%0d%0aU%2b0300%2dU%2b036F%0d%0aU%2b1DC0%2dU%2b1DFF%0d%0aU%2b20D0%2dU%2b20FF%0d%0aU%2bFE00%2dU%2bFE0F%0d%0aU%2bFE20%2dU%2bFE2F%0d%0aU%2bE0100%2dU%2bE01EF%0d%0a%5d%5d%5D>
\
Shall not be an operator 130 unassigned code points;
2,024 other code points
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5b%5b%0d%0a%5b%2f%20%5c%2d%20%2b%20%21%20%2a%20%25%20%3c%2d%3e%20%5c%26%20%7c%20%5c%5e%20%7e%20%3f%5d%0d%0aU%2b00A1%2dU%2b00A7%0d%0aU%2b00A9%20U%2b00AB%0d%0aU%2b00AC%20U%2b00AE%0d%0aU%2b00B0%2dU%2b00B1%20U%2b00B6%20U%2b00BB%20U%2b00BF%20U%2b00D7%20U%2b00F7%0d%0aU%2b2016%2dU%2b2017%20U%2b2020%2dU%2b2027%0d%0aU%2b2030%2dU%2b203E%0d%0aU%2b2041%2dU%2b2053%0d%0aU%2b2055%2dU%2b205E%0d%0aU%2b2190%2dU%2b23FF%0d%0aU%2b2500%2dU%2b2775%0d%0aU%2b2794%2dU%2b2BFF%0d%0aU%2b2E00%2dU%2b2E7F%0d%0aU%2b3001%2dU%2b3003%0d%0aU%2b3008%2dU%2b3030%0d%0a%5d%0d%0a%5b%0d%0aU%2b0300%2dU%2b036F%0d%0aU%2b1DC0%2dU%2b1DFF%0d%0aU%2b20D0%2dU%2b20FF%0d%0aU%2bFE00%2dU%2bFE0F%0d%0aU%2bFE20%2dU%2bFE2F%0d%0aU%2bE0100%2dU%2bE01EF%0d%0a%5d%5d-%5b%5b%3aPattern%5fSyntax%3a%5d%20%26%20%5b%3aMath%3a%5d%0d%0a%2d%20%5b%3aBlock%3dGeometric%20Shapes%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Symbols%3a%5d%0d%0a%2d%20%5b%3aBlock%3dMiscellaneous%20Technical%3a%5d%0d%0a%5b%21%20%25%20%5c%26%20%2a%20%5c%2d%20%2f%20%3f%20%5c%5c%20%5c%5e%20%C2%A1%20%C2%A6%20%C2%A7%20%C2%B0%20%C2%B6%20%C2%BF%20%E2%80%A0%20%E2%80%A1%20%E2%80%A2%20%E2%80%B0%20%E2%80%B1%20%E2%80%BB%20%E2%80%BD%20%E2%81%82%20%E2%81%85%20%E2%81%86%20%E2%81%8A%20%E2%81%8B%20%E2%81%8C%20%E2%81%8D%20%E2%81%8E%20%E2%81%91%5d%5d%5D>
*All
other code points*

Dots. Adopt a rule to allow dots to appear in operators at any location,
but only in runs of two or more. (Currently, dots must be leading.)
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#detailed-design>Detailed
design
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#identifiers>
Identifiers

Swift identifier characters shall conform to UAX#31
<http://unicode.org/reports/tr31/#Conformance> as follows:

   -

   UAX31-C1. <http://unicode.org/reports/tr31/#C1> The conformance
   described herein refers to the Unicode 9.0.0 version of UAX#31.
   -

   UAX31-C2. <http://unicode.org/reports/tr31/#C2> Swift shall observe the
   following requirements:
   -

      UAX31-R1. <http://unicode.org/reports/tr31/#R1> Swift shall augment
      the definition of "Default Identifiers" with the following profiles:
      1.

         ID_Start and ID_Continue shall be used for Start and Continue,
         replacing XID_Start and XID_Continue. This excludes characters in
         Other_ID_Start and Other_ID_Continue.
         2.

         _ 005F LOW LINE shall additionally be allowed as a Start character.
         3.

         Certain emoji shall additionally be allowed as Start characters. A
         detailed design for emoji permitted in identifiers is given below.
         4.

         UAX31-R1a. <http://unicode.org/reports/tr31/#R1a> The join-control
         characters ZWJ and ZWNJ are strictly limited to the special
cases A1, A2,
         and B described in UAX#31.
         -

      UAX31-R4. <http://unicode.org/reports/tr31/#R4> Swift shall consider
      two identifiers equivalent when they produce the same normalized
form under Normalization
      Form C (NFC) <http://unicode.org/reports/tr15/>, as recommended in
      UAX#31 for case-sensitive use cases.

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes>Grammar
changes

identifier-head → [:ID_Start:]
identifier-head → _
identifier-head → identifier-emoji
identifier-character → identifier-head
identifier-character → [:ID_Continue:]

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#operators>
Operators

Swift operator characters shall be determined as follows:

   -

   Valid operator characters shall consist of Pattern_Syntax code points
   with a derived property Math. However, the following blocks are
   excluded: Geometric Shapes, Miscellaneous Symbols, and Miscellaneous
   Technical. In UnicodeSet notation:

   [:Pattern_Syntax:] & [:Math:]
   - [:Block=Geometric Shapes:]
   - [:Block=Miscellaneous Symbols:]
   - [:Block=Miscellaneous Technical:]

   Math captures a fuller set of operators than is possible using Sm, and
   we avoid the inclusion of characters in So that are clearly not
   "operator-like" (such as Braille). Math code points in the excluded
   blocks include sign parts such as ⎲ SUMMATION TOP and tenuously
   "operator-like" code points such as ♠️ BLACK SPADE SUIT.
   -

   The set of valid operator characters shall be augmented with the
   following ASCII characters: !, %, &, *, -, /, ?, \, ^. These ASCII
   characters are required by the Swift standard library and/or considered
   "weakly mathematical" in UAX#25 <http://www.unicode.org/reports/tr25/>.
   -

   For increased compatibility with Swift 3, the set of valid operator
   characters shall be augmented with the following Latin-1 Supplement
   characters: ¡, ¦, §, °, ¶, ¿. For the same reason, augment the set of
   valid operator characters with the following General Punctuation
   characters: † DAGGER, ‡ DOUBLE DAGGER, • BULLET, ‰ PER MILLE SIGN, ‱ PER
   TEN THOUSAND SIGN, ※ REFERENCE MARK, ‽ INTERROBANG, ⁂ ASTERISM, ⁅ LEFT
   SQUARE BRACKET WITH QUILL, ⁆ RIGHT SQUARE BRACKET WITH QUILL, ⁊ TIRONIAN
   SIGN ET, ⁋ REVERSED PILCROW SIGN, ⁌ BLACK LEFTWARDS BULLET, ⁍ BLACK
   RIGHTWARDS BULLET, ⁎ LOW ASTERISK, ⁑ TWO ASTERISKS ALIGNED VERTICALLY.
   -

   Swift shall consider two operators equivalent when they produce the same
   normalized form under Normalization Form KC (NFKC)
   <http://unicode.org/reports/tr15/>, as recommended in UAX#31 for
   *case-insensitive* use cases. Crucially, fullwidth variants such as
   FULLWIDTH HYPHEN-MINUS are equivalent to their non-fullwidth counterparts
   after normalization under NFKC (but not NFC).
   -

   Certainly strongly mathematical arrows now have an *alternative* emoji
   presentation, and future versions of Unicode may add such an emoji
   presentation to any Swift operator character. Some but not all
   "environments" or applications (for instance, Safari but not TextWrangler)
   display the alternative emoji presentation at all times, and such
   discrepancies between applications are explicitly permitted by Unicode
   recommendations (see dicussion in *Emoji*). However, it would be highly
   unusual to define the set of valid operator characters based on an
   essentially arbitrary criterion as to whether an alternative emoji
   presentation is retroactively assigned to a code point, and codifying how
   IDEs display Unicode characters in Swift files is outside the scope of this
   proposal. Therefore, valid operator characters are defined without regard
   to the presence or absence of an alternative emoji presentation, and U+FE0E
   VARIATION SELECTOR-15 (text presentation selector) is *optionally* permitted
   to follow an operator character that has an alternative emoji presentation.
   Note that variation selectors are discarded by normalization.

These revised rules
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%3APattern_Syntax%3A%5D+%26+%5B%3AMath%3A%5D%0D%0A-+%5B%3ABlock%3DGeometric+Shapes%3A%5D%0D%0A-+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%0D%0A-+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%0D%0A%5B%21+%25+%5C%26+*+%5C-+%2F+%3F+%5C%5C+%5C%5E+%C2%A1+%C2%A6+%C2%A7+%C2%B0+%C2%B6+%C2%BF+%E2%80%A0+%E2%80%A1+%E2%80%A2+%E2%80%B0+%E2%80%B1+%E2%80%BB+%E2%80%BD+%E2%81%82+%E2%81%85+%E2%81%86+%E2%81%8A+%E2%81%8B+%E2%81%8C+%E2%81%8D+%E2%81%8E+%E2%81%91%5D&g=&i=>
produce
a set of 987 code points for operator characters. Since ID_Start is derived
in part by exclusion of Pattern_Syntax code points, it is assured that
operator and identifier characters do not overlap (although this assurance
does not extend to emoji, which require additional design as detailed
below).

All current restrictions on reserved tokens and operators remain. Swift
reserves =, ->, //, /*, */, ., ?, prefix <, prefix &, postfix >, and
postfix !.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#dots>Dots

Swift's existing rule for dots in operators is:

If an operator doesn’t begin with a dot, it can’t contain a dot elsewhere.

This proposal modifies the rule to:

Dots may only appear in operators in sequences of two or more.

Incorporating the "two-dot rule" offers the following benefits:

   -

   It avoids lexical complications arising from lone ..
   -

   The approach is conservative, erring on the side of overly restrictive.
   Dropping the rule in future (and thereby allowing single dots) may be
   possible.
   -

   It does not require special cases for existing infix dot operators in
   the standard library, ... (closed range) and ..<(half-open range). It
   leaves open the possibility of adding analogous half-open and fully-open
   range operators <..and <..<.

Finally, this proposal *reserves* the .. operator for a possible "method
cascade" syntax in the future as supported by Dart
<http://news.dartlang.org/2012/02/method-cascades-in-dart-posted-by-gilad.html>
.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes-1>Grammar
changes

operator → operator-head operator-characters[opt]

operator-head → [[:Pattern_Syntax:] & [:Math:] - [:Emoji:] -
[:Block=Geometric Shapes:] - [:Block=Miscellaneous Symbols:] -
[:Block=Miscellaneous Technical:]]
operator-head → [[:Pattern_Syntax:] & [:Math:] & [:Emoji:] -
[:Block=Geometric Shapes:] - [:Block=Miscellaneous Symbols:] -
[:Block=Miscellaneous Technical:]] U+FE0E[opt]
operator-head → ! | % | & | * | - | / | ? | \ | ^ | ¡ | ¦ | § | ° | ¶ | ¿
operator-head → † | ‡ | • | ‰ | ‱ | ※ | ‽ | ⁂ | ⁅ | ⁆ | ⁊ | ⁋ | ⁌ | ⁍ | ⁎ | ⁑
operator-head → operator-dot operator-dots
operator-character → operator-head
operator-characters → operator-character operator-character[opt]

operator-dot → .
operator-dots → operator-dot operator-dots[opt]

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#emoji>Emoji

The inclusion of emoji among valid identifier characters, though highly
desired, presents significant challenges:

   -

   Emoji characters are not displayed uniformly across different platforms.
   -

   Whether any particular character is presented as emoji or text depends
   on a matrix of considerations, including "environment" (e.g., Safari vs.
   XCode), presence or absence of a variant selector, and whether the
   character itself defaults to "emoji presentation" or "text presentation."
   This behavior is specifically documented in Unicode recommendations
   <http://unicode.org/reports/tr51/#Presentation_Style>.
   -

   Some emoji not classified as Math depict operators: ❗️❓➕➖➗✖️. A Unicode
   chart <http://unicode.org/emoji/charts/emoji-ordering.html> provides
   additional information by dividing emoji according to "rough categories,"
   but it warns that these categories "may change at any time, and should not
   be used in production."
   -

   Full emoji support would require allowing identifiers to contain
   zero-width joiner sequences that UAX#31 would forbid. Some normalization
   scheme would have to be devised to account for Unicode recommendations that
   👩‍❤️‍👨 (U+1F469 U+200D U+2764 U+FE0F U+200D U+1F468) can be displayed
   as either 💑 (U+1F491) or, as a fallback, 👩❤️👨(U+1F469 U+2764 U+FE0F
   U+1F468).

For maximum consistency across platforms, valid emoji in Swift identifiers
shall be determined using the following rules:

   -

   Emoji shall include code points with default emoji presentation (as
   opposed to text presentation), minus Emoji_Defectives and ID_Continue.
   Exclude Pattern_Syntax code points unless they are in the following
   blocks: Miscellaneous Symbols, Miscellaneous Technical.
   -

   Emoji shall include Emoji code points with default text presentation *when
   immediately followed by U+FE0F VARIATION SELECTOR-16 (emoji presentation
   selector)*, minus Emoji_Defectives and ID_Continue. Again, exclude
   Pattern_Syntax code points unless they are in the following blocks:
   Miscellaneous Symbols, Miscellaneous Technical. (Note that the emoji picker
   on Apple platforms--and, possibly, other platforms--automatically inserts
   U+FE0F VARIATION SELECTOR-16 when a user selects such code points; for
   instance, selecting ❤️ inserts U+2764 U+FE0F. Therefore, it is important
   that the invisible U+FE0F be permitted strictly in this use case. Note also
   that variation selectors are discarded by normalization.)
   -

   Emoji shall include Emoji_Flag_Sequences, Emoji_Keycap_Sequences, and
   (to the extent not already included) Emoji_Modifier_Sequences.

These revised rules
<http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+-+%5B%3APattern_Syntax%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Symbols%3A%5D%5D%0D%0A%5B%5B%3AEmoji%3A%5D+-+%5B%3AEmoji_Defectives%3A%5D+-+%5B%3AEmoji_Presentation%3A%5D+-+%5B%3AID_Continue%3A%5D+%26+%5B%3APattern_Syntax%3A%5D+%26+%5B%3ABlock%3DMiscellaneous+Technical%3A%5D%5D%0D%0A%5B%3AEmoji_Flag_Sequences%3A%5D%0D%0A%5B%3AEmoji_Keycap_Sequences%3A%5D%0D%0A%5B%3AEmoji_Modifier_Sequences%3A%5D&g=&i=>
produce
a set of 1,625 code points or sequences, of which 98 are currently
categorized as operator characters.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#grammar-changes-2>Grammar
changes

identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] -
[:ID_Continue:] - [:Pattern_Syntax:]]
identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] -
[:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous
Symbols:]]
identifier-emoji → [[:Emoji_Presentation:] - [:Emoji_Defectives:] -
[:ID_Continue:] & [:Pattern_Syntax:] & [:Block=Miscellaneous
Technical:]]
identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] -
[:Emoji_Presentation:] - [:ID_Continue:] - [:Pattern_Syntax:]] U+FE0F
identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] -
[:Emoji_Presentation:] - [:ID_Continue:] & [:Pattern_Syntax:] &
[:Block=Miscellaneous Symbols:]] U+FE0F
identifier-emoji → [[:Emoji:] - [:Emoji_Defectives:] -
[:Emoji_Presentation:] - [:ID_Continue:] & [:Pattern_Syntax:] &
[:Block=Miscellaneous Technical:]] U+FE0F
identifier-emoji → [[:Emoji_Flag_Sequences:]
[:Emoji_Keycap_Sequences:] [:Emoji_Modifier_Sequences:]]

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#source-compatibility>Source
compatibility

This change is source-breaking where developers have incorporated certain
emoji in identifiers or certain non-ASCII characters in operators. This is
unlikely to be a significant breakage for the majority of Swift code.
Diagnostics for invalid characters are already produced today. We can
improve them easily if needed.

Maintaining source compatibility for Swift 3 should be easy: keep the old
parsing and identifier lookup code.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#effect-on-abi-stability>Effect
on ABI stability

This proposal does not affect the ABI format itself. Normalization of
Unicode identifiers would affect the ABI of compiled modules. The standard
library will not be affected; it uses ASCII symbols with no combining
characters.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#effect-on-api-resilience>Effect
on API resilience

This proposal doesn't affect API resilience.
<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#alternatives-considered>Alternatives
considered

   -

   Use NFKC instead of NFC for identifiers. The decision to use NFC is
   based on UAX#31, which states:

   Generally if the programming language has case-sensitive identifiers,
   then Normalization Form C is appropriate; whereas, if the programming
   language has case-insensitive identifiers, then Normalization Form KC is
   more appropriate.

   -

   Eliminate emoji from identifiers and restrict operator characters to a
   limited number of ASCII code points. This approach would be simpler, but
   feedback on Swift-Evolution has been overwhelmingly against such a change.
   -

   Hand-pick a set of "operator-like" characters to include. The proposal
   authors tried this painstaking approach and came up with a relatively
   agreeable set of about 650 code points
   <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%21%5C%24%25%5C%26*%2B%5C-%2F%3C%3D%3E%3F%5C%5E%7C%7E%0D%0A%0D%0A%5Cu00AC%0D%0A%5Cu00B1%0D%0A%5Cu00B7%0D%0A%5Cu00D7%0D%0A%5Cu00F7%0D%0A%0D%0A%5Cu2208-%5Cu220D%0D%0A%5Cu220F-%5Cu2211%0D%0A%5Cu22C0-%5Cu22C3%0D%0A%5Cu2212-%5Cu221D%0D%0A%5Cu2238%0D%0A%5Cu223A%0D%0A%5Cu2240%0D%0A%5Cu228C-%5Cu228E%0D%0A%5Cu2293-%5Cu22A3%0D%0A%5Cu22BA-%5Cu22BD%0D%0A%5Cu22C4-%5Cu22C7%0D%0A%5Cu22C9-%5Cu22CC%0D%0A%5Cu22D2-%5Cu22D3%0D%0A%5Cu2223-%5Cu222A%0D%0A%5Cu2236-%5Cu2237%0D%0A%5Cu2239%0D%0A%5Cu223B-%5Cu223E%0D%0A%5Cu2241-%5Cu228B%0D%0A%5Cu228F-%5Cu2292%0D%0A%5Cu22A6-%5Cu22B9%0D%0A%5Cu22C8%0D%0A%5Cu22CD%0D%0A%5Cu22D0-%5Cu22D1%0D%0A%5Cu22D4-%5Cu22FF%0D%0A%5Cu22CE-%5Cu22CF%0D%0A%0D%0A%5Cu2A00-%5Cu2AFF%0D%0A%0D%0A%5Cu27C2%0D%0A%5Cu27C3%0D%0A%5Cu27C4%0D%0A%5Cu27C7%0D%0A%5Cu27C8%0D%0A%5Cu27C9%0D%0A%5Cu27CA%0D%0A%5Cu27CE-%5Cu27D7%0D%0A%5Cu27DA-%5Cu27DF%0D%0A%5Cu27E0-%5Cu27E5%0D%0A%0D%0A%5Cu29B5-%5Cu29C3%0D%0A%5Cu29C4-%5Cu29C9%0D%0A%5Cu29CA-%5Cu29D0%0D%0A%5Cu29D1-%5Cu29D7%0D%0A%5Cu29DF%0D%0A%5Cu29E1%0D%0A%5Cu29E2%0D%0A%5Cu29E3-%5Cu29E6%0D%0A%5Cu29FA%0D%0A%5Cu29FB%0D%0A%0D%0A%5Cu2308-%5Cu230B%0D%0A%5Cu2336-%5Cu237A%0D%0A%5Cu2395%5D>.
   Such a list can carefully avoid idiosyncrasies in the Unicode standard.
   However, a character-by-character inventory is unlikely to converge on
   consensus, as likely to introduce unintended Swift-specific idiosyncrasies
   as it is to avoid Unicode shortcomings, and inconsistent with the Unicode
   method of deriving such lists using categories.
   -

   Continue to allow single . in operators, perhaps even expanding the
   original rule to allow them anywhere (even if the operator does not begin
   with .).

   This would allow a wider variety of custom operators (for some
   interesting possibilities, see the operators in Haskell's Lens
   <https://github.com/ekmett/lens/wiki/Operators> package). However, there
   are a handful of potential complications:
   -

      Combining prefix or postfix operators with member access: foo*.bar would
      need to be parsed as foo *. barrather than (foo*).bar. Parentheses
      could be required to disambiguate.
      -

      Combining infix operators with contextual members: foo*.bar would
      need to be parsed as foo *. bar rather than foo * (.bar). Whitespace
      or parentheses could be required to disambiguate.
      -

      Hypothetically, if operators were accessible as members such as
      MyNumber.+, allowing operators with single .s would require escaping
      operator names (perhaps with backticks, such as MyNumber.`+`).

   This would also require operators of the form [!?]*\. (for example . ?.
   !.  !!.) to be reserved, to prevent users from defining custom operators
   that conflict with member access and optional chaining.

   We believe that requiring dots to appear in groups of at least two,
   while in some ways more restrictive, will prevent a significant amount of
   future pain, and does not require special-case considerations such as the
   above.

<https://gist.github.com/xwu/d2c2bb7097b0b5a4e9985aae737a2651#future-directions>Future
directions

While not within the scope of this proposal, the following considerations
may provide useful context for the proposed changes. We encourage the
community to pick up these topics when the time is right.

   -

   Introduce a syntax for method cascades. The Dart language supports method
   cascades
   <http://news.dartlang.org/2012/02/method-cascades-in-dart-posted-by-gilad.html>,
   whereby multiple methods can be called on an object within one expression:
   foo..bar()..baz() effectively performs foo.bar(); foo.baz(). This syntax
   can also be used with assignments and subscripts. Such a feature might be
   very useful in Swift; this proposal reserves the .. operator so that it
   may be added in the future.
   -

   Introduce "mixfix" operator declarations. Mixfix operators are based on
   pattern matching and would allow more than two operands. For example, the
   ternary operator ? : can be defined as a mixfix operator with three
   "holes": _ ? _ : _. Subscripts might be subsumed by mixfix declarations
   such as _ [ _ ]. Some holes could be made @autoclosure, and there might
   even be holes whose argument is represented as an AST, rather than a value
   or thunk, supporting advanced metaprogramming (for instance, F#'s code
   quotations
   <https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/code-quotations>).
   Should mixfix operators become supported, it would be sensible to add
   brackets to the set of valid operator characters.
   -

   Diminish or remove the lexical distinction between operators and
   identifiers. If precedence and fixity applied to traditional identifiers
   as well as operators, it would be possible to incorporate ASCII equivalents
   for standard operators (e.g. and for &&, to allow A and B). If
   additionally combined with mixfix operator support, this might enable
   powerful DSLs (for instance, C#'s LINQ
   <https://en.wikipedia.org/wiki/Language_Integrated_Query>).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170216/06224157/attachment-0001.html>


More information about the swift-evolution mailing list