[swift-dev] undefining macros after including Tokens.def

[Micah Hainline]

I'm looking to spruce some of this up, now that I've taken the time to
understand what people are using it for. I've broken it down locally
to have all the macro functions that expanded KEYWORD directly go
through an intermediary for consistency, and added a SWIFT_KEYWORD
that expands to KEYWORD. No more defining SIL_KEYWORD first just to
eliminate SIL from KEYWORD.

While doing so, I added another macro EXPR_KEYWORD for everything that
was under the comment "Expression keywords". That caused errors when I
put that new macro around __FILE__, and that made me wonder why in the
heck __FILE__ is included in the keywords. What I got when I did that
was a token pasting error in the many places we make things like
tok::kw_##KW. Suddenly it's token pasting to create
kw_"/User/micah/swift/blah/blah/something.cpp" and failing.

I'm sure I'm just not very good with macros and a long-term whiz at it
would laugh, but I'm confused about what we're trying to do here. Why
would we put __FILE__ into the keywords list to begin with?

On Wed, Dec 28, 2016 at 5:10 PM, Jacob Bandes-Storch <jtbandes at gmail.com> wrote:
> I am not the authority here, but based on what I've seen this sounds good to
> me. I was once discouraged from adding an #undef in client code because
> Tokens.def does it already. Mentioning that behavior in the comments
> certainly can't hurt.
>
>
> On Wed, Dec 28, 2016 at 2:37 PM, Micah Hainline via swift-dev
> <swift-dev at swift.org> wrote:
>>
>> I'm still wrapping my head around this, but we're doing some heavy
>> macro programming in swift/Parse/Tokens.def. We define macros such as
>> KEYWORD and then include the file, which allows different things to
>> happen based on what macros we've defined beforehand.
>>
>> Because it's using macros, subsequent includes of Tokens.def will have
>> those same old macros defined unless they are subsequently undefined.
>> That is actually happening in the bottom of the Tokens.def now, but
>> apparently that wasn't always the case and there is some leftover code
>> floating around that tries to deal with it. Sometimes (see
>> SyntaxModel.cpp lines 98-100) we then #undef the macro afterward, in
>> effect manually cleaning up after ourselves. In Lexer.cpp line 546 we
>> define KEYWORD just to redefine it to be empty on line 602 presumably
>> to avoid the previous definition stepping on our toes in the next
>> couple of lines. Of course, the include already cleaned that up.
>>
>> I think we should go through and try to be more consistent with this.
>> Tokens.def should have a comment block explaining exactly which macros
>> will be checked and just specifying they'll all be undefined
>> afterward, and all include usages should get rid of empty defines and
>> undefines. Does that make sense?
>> _______________________________________________
>> swift-dev mailing list
>> swift-dev at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev
>
>