[swift-evolution] Strings in Swift 4

Tue Jan 24 16:38:02 CST 2017

> On Jan 24, 2017, at 7:52 AM, Matthew Johnson via swift-evolution <swift-evolution at swift.org> wrote:
> 
>> 
>> On Jan 24, 2017, at 2:05 AM, Chris Eidhof via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> I agree that being able to implement parsers in a nice way can be a huge step forward in being really good at string processing.
>> 
>> There are a couple of possibilities that come to mind directly:
>> 
>> 1. Build parsers right into the language (like Perl 6 grammars)
>> 2. Provide a parser combinator language (e.g. https://github.com/davedufresne/SwiftParsec <https://github.com/davedufresne/SwiftParsec>). 
>> 3. Rely on external tools like bison/yacc/etc.
>> 4. Make it easy for people to write hand-written parsers (e.g. by providing an NSScanner alternative).
>> 
>> Some obvious drawbacks of each approach:
>> 
>> 1. Lots of work, probably hard to get right?
>> 2. Only way to do this, afaik, is using lots of functional programming which might scare people off. Also probably it's hard to get performance as fast as 1.
> 
> FWIW, it is quite possible to do things very similar to parser combinators without functional programming.  What you need is a way to create and compose small parser fragments, ideally an EDSL approaching something like EBNF that allows users to build a grammar out of the parser fragments, and a way to execute / interpret the resulting grammar during parsing.  
> 
> The functional approach would not be the most idiomatic approach in Swift and as you note, it probably wouldn’t have the performance a more idiomatic approach could achieve (too much copying).
> 
> My intuition is that a hybrid 1 / 2 approach might be best: do as much as possible in the library and let the design drive new language enhancements where necessary.

Yes, the first program I wrote in Swift was a packrat parser (now used in several of my projects).  It uses custom operators to let you write the grammar directly in swift, which is nice because it can be combined directly with code which gets run when a rule is matched.  

One of the most time consuming parts of that was writing a custom string scanner.  I would really like to see support for basic string scanning, including a standard library implementation of something like NSCharacterSet, built into the new String.  Or rather, I would like features built in which make building a string scanner trivial and lightweight.  Basically just a couple of methods saying things like: given an index and a character set, return a range including the contiguous group of characters in that set directly after the index (with options for case/diacritic sensitivity, etc…).  With that and better slices, it should be fairly easy to build a lightweight/efficient scanner... 

>> 3. No clear integrated way to do this
>> 4. You still have to know how to write a parser.
>> 
>> I would think that 4. would be a good step forward, and 1/2 would definitely benefit from this.
>> 
>> Also, I'd love to have this functionality on sequence/collection types, rather than Strings. For example, it can be tremendously helpful to parse a binary format using proper parsers. Or maybe you would want to use an event-driven XML parser as "tokenizer" and parse that. Plenty of cool possibilities.
>> 
>> On Tue, Jan 24, 2017 at 8:46 AM, Russ Bishop via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>>> On Jan 23, 2017, at 2:27 PM, Joe Groff via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>>> 
>>>> On Jan 23, 2017, at 2:06 PM, Ben Cohen via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>> 
>>>> 
>>>>> On Jan 23, 2017, at 7:49 AM, Joshua Alvarado <alvaradojoshua0 at gmail.com <mailto:alvaradojoshua0 at gmail.com>> wrote:
>>>>> 
>>>>> Taken from NSHipster <http://nshipster.com/nsregularexpression/>:
>>>>> Happily, on one thing we can all agree. In NSRegularExpression, Cocoa has the most long-winded and byzantine regular expression interface you’re ever likely to come across.
>>>>> 
>>>>> There is no way to achieve the goal of being better at string processing than Perl without regular expressions being addressed. It just should not be ignored. 
>>>> 
>>>> 
>>>> We’re certainly not ignoring the importance of regexes. But if there’s a key takeaway from your experiences with NSRegularExpression, it’s that a good regex implementation matters, a lot. That’s why we don’t want to rush one in alongside the rest of the overhaul of String. Instead, we should take our time to make it really great, and building on a solid foundation of a good String API that’s already in place should help ensure that.
>>> 
>>> I do think that there's some danger to focusing too narrowly on regular expressions as they appear in languages today. I think the industry has largely moved on to fully-structured formats that require proper parsing beyond what traditional regexes can handle. The decades of experience with Perl shows that making regexes too easy to use without an easy ramp up to more sophisticated string processing leads to people cutting corners trying to make regex-based designs kind-of work. The Perl 6 folks recognized this and developed their "regular expression" support into something that supported arbitrary grammars; I think we'd do well to start at that level by looking at what they've done.
>>> 
>>> -Joe
>>> 
>> 
>> I fully agree. I think we could learn something from Perl 6 grammars. As PCREs are to languages without regex, Perl 6 grammars are to languages with PCREs. 
>> 
>> A lot of really crappy user interfaces and bad tools come down to half-assed parsers; maybe we can do better? (Another argument against rushing it).
>> 
>> 
>> Russ
>> 
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>> 
>> 
>> 
>> 
>> -- 
>> Chris Eidhof
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170124/2e36f9fa/attachment.html>