<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">… response inline<div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On May 2, 2016, at 2:23 PM, John Holdsworth <<a href="mailto:mac@johnholdsworth.com" class="">mac@johnholdsworth.com</a>> wrote:</div><div class=""><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""></div><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">I'm having trouble getting the `e` modifier to work as advertised, at least for the sequence `\\`. For example, `print(e"\\\\")` prints two backslashes, and `print(e"\\\")` seems to try to escape the string literal. I'm currently envisioning `e` as disabling *all* backslash escapes, so these behaviors wouldn't be appropriate. It also looks like interpolation is still enabled in `e` strings.</div><div class=""><div class=""><br class="">Since other things like `print(e"\w+")` work just fine, I'm guessing this is a bug in the proposal's sketches (not being clear enough about the expected behavior), not your code.<br class=""><br class="">I've written a gist with some tests to show how I expect things to work:<br class=""><br class=""><span class="Apple-tab-span" style="white-space: pre;"> </span><a href="https://gist.github.com/brentdax/be3c032bc7e0c101d7ba8b72cd1a692e" class="">https://gist.github.com/brentdax/be3c032bc7e0c101d7ba8b72cd1a692e</a></div></div></blockquote><div class=""><br class=""></div>The problem here is that I’ve not implemented unescaped literals fully as it would require changes outside the lexer.</div><div class="">This is because the string is first lexed and tokenised by one piece of code <span class="" style="font-family: Menlo; font-size: 11px; font-variant-ligatures: no-common-ligatures; color: rgb(79, 129, 135);">Lexer</span><span class="" style="font-family: Menlo; font-size: 11px; font-variant-ligatures: no-common-ligatures;">::lexStringLiteral </span>but later</div><div class="">on in the code generation phase it generates the actual literal in a function <span class="" style="font-family: Menlo; font-size: 11px; font-variant-ligatures: no-common-ligatures; color: rgb(79, 129, 135);">Lexer</span><span class="" style="font-family: Menlo; font-size: 11px; font-variant-ligatures: no-common-ligatures;">::getEncodedStringSegment.</span></div><div class="">This is passed the same string from the source file but does not know what modifiers should be applied. As a result</div><div class="">normal escapes are still processed. All the “e” flag does is silence the error for invalid escapes during tokenising.</div></div></blockquote><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""></div><div class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Lexer just lays ropes around certain areas to tell what's where. sometimes this is not enough for extra semantics. this is the reason why i went down the path of a custom string_multiline_literal token. It looks like you might want to consider that path too. <span class="" style="background-color: rgba(255, 255, 255, 0);">If you do, you might </span><span class="" style="background-color: rgba(255, 255, 255, 0);">consider the merits of suggesting that half the work be put in place now, allowing both our experimentations (and other more sophisticated) to lean on it, as an alternative to just </span>directly adding extra conditional code in the default lexer code.</div></div></blockquote><div><br class=""></div><div>Not sure what you mean here. It’s the modifiers that have a greater effect on lexing, not whether a string is multi-line. IMO it’s </div><div>probably best to avoid creating a separate string_multiline_literal token as that would require visiting the grammar everywhere</div><div>a string could occur. If you want to see what I mean I’ve committed a change which uses 3 extra bits to the Token structure to</div><div>carry modifiers applied from the lexing stage to code generation so non-escaping strings can finally be handled correctly.</div><div><br class=""></div><div><a href="https://github.com/apple/swift/pull/2275" class="">https://github.com/apple/swift/pull/2275</a></div><div>new toolchain: <a href="http://johnholdsworth.com/swift-LOCAL-2016-05-04-a-osx.tar.gz" class="">http://johnholdsworth.com/swift-LOCAL-2016-05-04-a-osx.tar.gz</a></div><div><br class=""></div><div>The following now holds</div><div><br class=""></div><div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; color: rgb(209, 47, 27);" class=""><span style="font-variant-ligatures: no-common-ligatures; color: #000000" class=""> </span><span style="font-variant-ligatures: no-common-ligatures; color: #3d1d81" class="">assert</span><span style="font-variant-ligatures: no-common-ligatures; color: #000000" class="">( e</span><span style="font-variant-ligatures: no-common-ligatures" class="">"\w\d+\(author)\n"</span><span style="font-variant-ligatures: no-common-ligatures; color: #000000" class=""> == </span><span style="font-variant-ligatures: no-common-ligatures" class="">"<a href="smb://w//d+//(author)//n" class="">\\w\\d+\\(author)\\n</a>"</span><span style="font-variant-ligatures: no-common-ligatures; color: #000000" class=""> );</span></div><div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> </span><span style="font-variant-ligatures: no-common-ligatures; color: #3d1d81" class="">assert</span><span style="font-variant-ligatures: no-common-ligatures" class="">( r</span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">"\w\d+</span><span style="font-variant-ligatures: no-common-ligatures" class="">\</span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">(</span><span style="font-variant-ligatures: no-common-ligatures" class="">author</span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">)\n"</span><span style="font-variant-ligatures: no-common-ligatures" class=""> == </span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">"<a href="smb://w//d+" class="">\\w\\d+</a></span><span style="font-variant-ligatures: no-common-ligatures" class="">\</span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">(</span><span style="font-variant-ligatures: no-common-ligatures" class="">author</span><span style="font-variant-ligatures: no-common-ligatures; color: #d12f1b" class="">)\n"</span><span style="font-variant-ligatures: no-common-ligatures" class=""> ); // previous implementation</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class="">John</span></div></div><br class=""></div><br class=""></div></body></html>