[swift-evolution] define backslash '\' as a operator-head in the swift grammar

[Nevin Brackett-Rozinsky]

Given that the definition of operator characters in Swift is “part of
Swift”, it follows that any change to that set will go through the Swift
Evolution process, which includes discussion on this list. I do not see a
reasonable way to skip that, nor would I consider it desirable to do so.
Discussing on this list which characters should be operators in Swift is
*prima facie* a necessary part of changing them.

In light of the fact that our current operator-character situation needs to
be fixed—meaning we will have to decide which characters should and should
not be allowed in operators—and seeing as any proposal to solve the problem
will be discussed on this list, the natural conclusion is that discussing
which characters should be operators is *exactly* what we ought to do.

Moreover, we are trying to decide which operators are right for *Swift*,
thus our goals include expressivity, usefulness, and enjoyability. Our
goals do *not* include being maximally conservative, and indeed I view
parsimony in operator-character availability as undesirable.

We should aim to choose the *right* set of operator characters, for some
Swifty and opinionated value of “right”. If the recommendations that
Unicode settles on strike us as right for Swift, that is great. But those
recommendations do not exist yet, and so far it sounds like they could well
be defined in a way which is *not* right for Swift.

Perhaps Unicode will add a new category for operators, and they will go
character-by-character to put certain glyphs in it. But if they don’t, then
as you say the existing categories are not sufficient for Swift’s purposes,
so their recommendations will not solve our problem. And if they do, we
still would need to verify that their decision matches our use-case.

The past proposal to gut non-ASCII operators from Swift was rightly
blocked, for it would have damaged the language. My follow-up with a list
of 1,020 characters to use as operators has not seen much discussion. If
there were any controversial characters in it or omissions from it, I hope
someone would raise an objection.

In any case, I think that discussing which characters are right for Swift
operators is an entirely proper and expected use of this mailing list, and
I suggest we do so during the Swift 4 timeframe because it affects source
compatibility.

Nevin


On Mon, Feb 6, 2017 at 7:05 PM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:

> Indeed, I'd be thrilled to revise and reintroduce a proposal to update
> identifier and operator characters, if co-authors of that proposal are OK
> with it. Agree entirely with the principle you enunciate; that is exactly
> how we tried to approach it previously. I take it that you agree that a
> comprehensive revision of identifier characters cannot happen without
> thinking about operator characters, and vice versa, and I believe we came
> to that conclusion as well.
>
> Two difficulties arose. Jonathan Shapiro indicated that the Unicode
> approach would invariably be to specify which characters are valid
> identifiers, etc. not by code points or planes but by "categories," for
> which there are none for operators but also for which there are problematic
> ones for emoji (the numerals 0-9 belong to the emoji category, for
> instance; there is another emoji_presentation category which has its own
> tricks). Therefore, we considered that the most conservative way to
> classify "definite operators," "definite identifiers," and everything else
> was to leave emoji out (UAX31 does not include them as identifier
> characters) and to roll back operator characters to the ASCII range.
> However, both temporary measures were soundly rejected.
>
> There are a number of security issues to be ironed out with respect to
> emoji. For instance, emoji modifiers such as skin tones and genders allow
> different identifiers to be made that are visually identical or nearly so.
> (And yes, I'm aware that you or I could come up with an ad-hoc solution to
> this particular problem with emoji, but the issue is identifying all such
> relevant problems and coming up with solutions that jibe with future
> Unicode solutions to the same problems.) It would have been nice to leave
> this out entirely for now until further guidance from Unicode experts, but
> the community made it clear that emoji were a sine qua non.
>
> Likewise, every discussion on operator characters thus far has devolved
> into a series of replies nominating specific characters for inclusion. I
> continue to believe a character-by-character debate on this mailing list
> would be exceptionally poor form. My point in replying to this thread is
> only that we don't need to get into any of this (Unicode operators,
> canonicalization forms, emoji, etc.--much as I would love to) in order to
> consider the isolated addition of an ASCII-range operator \. That idea
> seems very reasonable, independently useful, and limited in scope.
>
>
> On Mon, Feb 6, 2017 at 15:28 Nevin Brackett-Rozinsky via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>> Although there was, as you say, some push-back against revamping our set
>> of operator characters, there was also substantial push-forward. Many
>> people want to resolve the problematic situation we currently have
>> regarding the designation of operators and identifiers.
>>
>> And indeed, cutting back to ASCII-only operators would have been an
>> abominable choice. However waiting for the Unicode Consortium to draft
>> guidelines for operator characters means prolonging our existing
>> predicament. Additionally, in the discussion last fall it was mentioned
>> that Unicode personnel are aware of what we are doing with Swift operators,
>> and that our decisions may help to inform their classification of operator
>> characters:
>>
>> On Fri, Oct 21, 2016 at 5:38 PM, Jonathan S. Shapiro <jonathan.s.shapiro@
>> gmail.com> wrote:
>>
>> That's a feasible way to go, but keep in mind that the UAX31 changes are
>> being co-designed with and informed by the current discussion. There are a
>> bunch of things that have come up here that will allow UAX31 to side-step
>> some "might have happened" mistakes, so this discussion has been very
>> useful.
>>
>> The Swift community can and should make its own decision about whether to
>> remain engaged. The risk of disengagement is that messy compatibility
>> issues will probably have to be faced later that we can easily head-off now.
>>
>>
>> Given all these considerations, I think the principled approach is for us
>> to move forward with a 3-part categorization of characters into operators,
>> identifiers, and unspecified (to be determined). That way we need not
>> harangue ourselves over every controversial glyph, and may instead quickly
>> determine those characters which should definitely be operators and those
>> which should definitely be identifiers, while saving the difficult
>> decisions until such time as Unicode produces recommendations and/or we
>> decide to undertake a more comprehensive review.
>>
>> Nevin
>>
>>
>> On Sun, Feb 5, 2017 at 8:29 PM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>
>> IIRC, where we left the discussion last time was that there was work not
>> yet complete within Unicode on delineating identifier and operator
>> characters. As there was broad agreement to align identifier characters
>> with Unicode standards, and since the strict separation between identifiers
>> and operators means that no character should belong to both, there was
>> hesitation to declare an operator what Unicode may later deem to be an
>> identifier.
>>
>> There was strenuous objection to temporarily cutting back operators to
>> the ASCII range until Unicode completes its work, but also pushback in
>> going character-by-character above the ASCII range.
>>
>> In any case, \ seems perfectly reasonable as an additional operator
>> character that doesn't have to wait for Unicode.
>>
>> On Sun, Feb 5, 2017 at 19:02 T.J. Usiyan via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>> +1 from me.
>>
>> I hope that operators get more work soon, especially with regard to math.
>>
>> On Sun, Feb 5, 2017 at 5:09 PM, Nevin Brackett-Rozinsky via
>> swift-evolution <swift-evolution at swift.org> wrote:
>>
>> +1 as well. I also support adding these four symbols: ⅀ ؆ ؇ ⅋ as
>> operators.
>>
>> There was substantial discussion last fall about revamping operators in
>> Swift, with the primary goal of removing characters that should not be in
>> the set. I went through the Unicode tables and compiled a list of 1,020
>> characters that I think definitely should be operators [list of operator
>> characters
>> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5B%3ASm%3A%5D%0D%0A%0D%0A-%5Cp%7BBlock%3DSuperscripts+And+Subscripts%7D%0D%0A-%5Cp%7BBlock%3DMiscellaneous+Technical%7D%0D%0A-%5Cp%7BBlock%3DGeometric+Shapes%7D%0D%0A-%5Cp%7BBlock%3DMiscellaneous+Symbols%7D%0D%0A-%5Cp%7BBlock%3DAlphabetic+Presentation+Forms%7D%0D%0A-%5Cp%7BBlock%3DSmall+Form+Variants%7D%0D%0A-%5Cp%7BBlock%3DHalfwidth+And+Fullwidth+Forms%7D%0D%0A-%5Cp%7BBlock%3DMathematical+Alphanumeric+Symbols%7D%0D%0A-%5Cp%7BBlock%3DArabic+Mathematical+Alphabetic+Symbols%7D%0D%0A-%5Cp%7Bsubhead%3DVariant+letterforms+and+symbols%7D%0D%0A-%5Cp%7Bsubhead%3DLetterlike+symbol%7D%0D%0A%0D%0A%5Cp%7BBlock%3DArrows%7D%0D%0A%5B%2F+%3D+%5C-+%2B+%21+*+%25+%3C+%3E+%5C%26+%7C+%5C%5E+~+%3F%5D%0D%0A%5B%C2%A1+%C2%A2+%C2%A3+%C2%A4+%C2%A5+%C2%A6+%C2%A7+%C2%A9+%C2%AB+%C2%AC+%C2%AE+%C2%B0+%C2%B1+%C2%B6+%C2%BB+%C2%BF%5D+-+%5B%C2%A2+%C2%A3+%C2%A4+%C2%A5+%C2%A9+%C2%AE%5D%0D%0A%5Cp%7Bsubhead%3DGeneral+punctuation%7D+-+%5BU%2B203F+U%2B2040+U%2B2045+U%2B2046+U%2B2054%5D%0D%0A%5Cp%7Bsubhead%3DDouble+punctuation+for+vertical+text%7D%0D%0A%5Cp%7Bsubhead%3DArchaic+punctuation%7D+-+%5BU%2B2E31+U%2B2E33+U%2B2E34+U%2B2E3F%5D%0D%0AU%2B214B%5D&g=&i=>
>> ]
>>
>> The effect of that would be to make 1,628 characters no longer usable as
>> operators [list of non-operator characters
>> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%2F+%3D+%5C-+%2B+%21+*+%25+%3C+%3E+%5C%26+%7C+%5C%5E+~+%3F%0D%0AU%2B00A1+-+U%2B00A7%0D%0AU%2B00A9+U%2B00AB+U%2B00AC+U%2B00AE%0D%0AU%2B00B0+-+U%2B00B1%0D%0AU%2B00B6+U%2B00BB+U%2B00BF+U%2B00D7+U%2B00F7%0D%0AU%2B2016+-+U%2B2017%0D%0AU%2B2020+-+U%2B2027%0D%0AU%2B2030+-+U%2B203E%0D%0AU%2B2041+-+U%2B2053%0D%0AU%2B2055+-+U%2B205E%0D%0AU%2B2190+-+U%2B23FF%0D%0AU%2B2500+-+U%2B2775%0D%0AU%2B2794+-+U%2B2BFF%0D%0AU%2B2E00+-+U%2B2E7F%0D%0AU%2B3001+-+U%2B3003%0D%0AU%2B3008+-+U%2B3030%5D%0D%0A%0D%0A-%5B%5B%3ASm%3A%5D%0D%0A-%5Cp%7BBlock%3DSuperscripts+And+Subscripts%7D%0D%0A-%5Cp%7BBlock%3DMiscellaneous+Technical%7D%0D%0A-%5Cp%7BBlock%3DGeometric+Shapes%7D%0D%0A-%5Cp%7BBlock%3DMiscellaneous+Symbols%7D%0D%0A-%5Cp%7BBlock%3DAlphabetic+Presentation+Forms%7D%0D%0A-%5Cp%7BBlock%3DSmall+Form+Variants%7D%0D%0A-%5Cp%7BBlock%3DHalfwidth+And+Fullwidth+Forms%7D%0D%0A-%5Cp%7BBlock%3DMathematical+Alphanumeric+Symbols%7D%0D%0A-%5Cp%7BBlock%3DArabic+Mathematical+Alphabetic+Symbols%7D%0D%0A-%5Cp%7Bsubhead%3DVariant+letterforms+and+symbols%7D%0D%0A-%5Cp%7Bsubhead%3DLetterlike+symbol%7D%0D%0A%5Cp%7BBlock%3DArrows%7D%0D%0A%5B%2F+%3D+%5C-+%2B+%21+*+%25+%3C+%3E+%5C%26+%7C+%5C%5E+~+%3F%5D%0D%0A%5B%C2%A1+%C2%A2+%C2%A3+%C2%A4+%C2%A5+%C2%A6+%C2%A7+%C2%A9+%C2%AB+%C2%AC+%C2%AE+%C2%B0+%C2%B1+%C2%B6+%C2%BB+%C2%BF%5D+-+%5B%C2%A2+%C2%A3+%C2%A4+%C2%A5+%C2%A9+%C2%AE%5D%0D%0A%5Cp%7Bsubhead%3DGeneral+punctuation%7D+-+%5BU%2B203F+U%2B2040+U%2B2045+U%2B2046+U%2B2054%5D%0D%0A%5Cp%7Bsubhead%3DDouble+punctuation+for+vertical+text%7D%0D%0A%5Cp%7Bsubhead%3DArchaic+punctuation%7D+-+%5BU%2B2E31+U%2B2E33+U%2B2E34+U%2B2E3F%5D%0D%0AU%2B214B%5D&g=&i=>
>> ]
>>
>> However, my strategy was to be conservative in accepting operators. There
>> are several Unicode blocks which contain some additional characters which
>> we may want to have as operators [list of characters in those blocks
>> <http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%5Cp%7BBlock%3DMiscellaneous+Technical%7D%0D%0A%5Cp%7BBlock%3DOptical+Character+Recognition%7D%0D%0A%5Cp%7BBlock%3DBox+Drawing%7D%0D%0A%5Cp%7BBlock%3DBlock+Elements%7D%0D%0A%5Cp%7BBlock%3DGeometric+Shapes%7D%0D%0A%5Cp%7BBlock%3DMiscellaneous+Symbols%7D%0D%0A%5Cp%7BBlock%3DDingbats%7D%0D%0A%5Cp%7BBlock%3DBraille%7D%0D%0A%5Cp%7BBlock%3DMiscellaneous+Symbols+And+Arrows%7D%0D%0A%5Cp%7BBlock%3DYijing+Hexagram+Symbols%7D%0D%0A%5Cp%7BBlock%3DMusical+Symbols%7D%0D%0A%5Cp%7BBlock%3DAncient+Greek+Musical+Notation%7D%0D%0A%5Cp%7BBlock%3DTai+Xuan+Jing+Symbols%7D%0D%0A%5Cp%7BBlock%3DMahjong+Tiles%7D%0D%0A%5Cp%7BBlock%3DDomino+Tiles%7D%0D%0A%5Cp%7BBlock%3DPlaying+Cards%7D%0D%0A%5Cp%7BBlock%3DOrnamental+Dingbats%7D%0D%0A%5Cp%7BBlock%3DAlchemical+Symbols%7D%0D%0A%5Cp%7BBlock%3DGeometric+Shapes+Extended%7D%0D%0A%5Cp%7BBlock%3DSupplemental+Arrows+C%7D%5D%0D%0A&g=&i=>
>> ]
>>
>> I did not include the backslash because I decided not to mess with the
>> choice of ASCII operators, however I do support making backslash an
>> operator. I am not sure about currency symbols.
>>
>> Nevin
>>
>>
>> On Sun, Feb 5, 2017 at 1:20 PM, Dave Abrahams via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>>
>> on Sun Feb 05 2017, Nicolas Fezans <swift-evolution at swift.org> wrote:
>>
>> > Dear all,
>> >
>> > This is a rather simple proposal to add '\' (backslash character) as a
>> > valid operator-head in the swift grammar.
>> >
>> > One argument for it, is that there exist a backslash operator in the
>> > MATLAB/Scilab/Octave languages. In this languages A\B solves the linear
>> > system A*X = B for X (or the least square problem associated to it if
>> the
>> > system of equations is overdetermined). I am doing some numerical
>> > computation in Swift and it would be nice to be able to declare the same
>> > operator name for this functionality.
>> >
>> > I might have missed some arguments for not adding them, but I seems to
>> me
>> > that until now the \ character is only used inside of string literals.
>> If
>> > that is the case, both uses should never generate a conflict or be
>> > ambiguous, isn't it? (String literals keep their interpretation of \
>> and \
>> > used otherwise within the swift code will be interpreted as an operator
>> or
>> > as the beginning of an operator)
>> >
>> > I am curious to see what will be the feedback on this.
>>
>> +1 if it doesn't clash with the grammar.
>>
>> --
>> -Dave
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170206/42899d16/attachment.html>