[swift-evolution] A path forward on rationalizing unicode identifiers and operators

[Chris Lattner]

On Sep 30, 2017, at 7:10 PM, Jonathan Hull <jhull at gbis.com> wrote:
> I have a technical question on this:
> 
> Instead of parsing these into identifiers & operators, would it be possible to parse these into 3 categories: Identifiers, Operators, and Ambiguous?
> 
> The ambiguous category would be disallowed for the moment, as you say.  But since they are rarely used, maybe we can allow a declaration (similar to how we define operators) that effectively pulls it into one of the other categories (not in terms of tokenization, but in terms of how it can be used in Swift).  

This is commonly requested, but the third category isn’t practical.  

Swift statically partitions characters between identifiers and operators to make it possible to parse a Swift source file without parsing all of its dependencies.  If you could have directives that change this, it would be difficult or perhaps impossible to parse a file that used these characters without parsing/reading the transitive closure of dependent modules.

This is important for compile speed and some tooling, and is an area that C gets wrong - its grammar requires all headers to be parsed in order to distinguish between type names and normal identifiers.

-Chris