[swift-dev] Breaking change in lexing operators next to comments

[Jesse Rusak]

Any other comments about this? Can someone from the core team make a call on this, or should we ask for comments on swift-evolution?

> On Dec 15, 2015, at 7:15 PM, Jesse Rusak <me at jesserusak.com> wrote:
> 
>>> On Dec 14, 2015, at 9:42 PM, Chris Lattner via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
> 
>>> There are two defensible models here:
>>> 
>>> 1) comments should be treated as whitespace.
>>> 2) comments should be treated as if they were not present.
>>> 
>>> The later model seems more ideal to me (because you can put whitespace on either side of the comment after all), but I don’t have a strong opinion about that.  What do others think?
> 
>> 
>> On Dec 15, 2015, at 1:08 AM, John Calsbeek via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
>> 
>> If you treat comments as though they are not present, you can no longer reason locally about whitespace on either side of an operator. Straw example:
>> 
>> foo/* insert an
>> excerpt from War
>> and Peace here */!
>> 
>> I need to scan to the other side of the comment to determine if ! is preceded by whitespace.
>> 
>> There is already a list of situations in which some token is treated as whitespace for the purpose of operators in The Swift Programming Language:
>> 
>> For the purposes of these rules, the characters (, [, and { before an operator, the characters ), ], and } after an operator, and the characters ,, ;, and : are also considered whitespace.
>> 
>> There is one caveat to the rules above. If the ! or ? predefined operator has no whitespace on the left, it is treated as a postfix operator, regardless of whether it has whitespace on the right. To use the ? as the optional-chaining operator, it must not have whitespace on the left. To use it in the ternary conditional (? :) operator, it must have whitespace around both sides.
>> 
>> Given that, it seems more natural to me to define comments as “treated-as-whitespace” in the same way.
>> 
>> “Treated as not present” is also not quite the right way to word the opposite case, since comments would still separate tokens. Say you had an automated tool that deletes comments (perhaps unlikely, but let’s roll with it). “Treated as not present” says you should completely delete the comment, but that doesn’t actually work since it could still cause two separate tokens to be glued together. “Treated as whitespace” just means that you have to replace the comment with at least one character of whitespace.
> 
> FWIW, I agree with John about this. I think either model is reasonable but treating comments as whitespace is better because:
> 
> * The Swift language reference already states the general rule that comments are whitespace; I think it’s better to apply this throughout than change it.
> 
> * I think it’s easier to explain that "comments are whitespace" than "comments are treated as not present except they separate tokens”.
> 
> * The non-local effects John describes are mildly awkward for human readers and in the lexer. (I think we’d have to walk backwards through slash-star comments to determine if we have space to the left of an operator.)
> 
> - Jesse

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20151217/4cd2b8d5/attachment.html>