[swift-evolution] Large Proposal: Non-Standard Libraries

Wed Nov 8 23:42:20 CST 2017

These are some really interesting analogies.  How would you imagine the community “governance” of these “plugins” (which I assume would be libraries or packages) to be managed?  What does it mean for the “full community” to manage them, and provide the rough guarantees you suggest?

I like the general premise here: support a library ecosystem for Swift, with possibly layers of expectations and guarantees on the quality of those libraries and the features they support.

A related question, what can we do to how the communication discussion via Swift.org <http://swift.org/> can be used to facilitate building such a library ecosystem?  Kelvin Ma made a great point about shouting out on the mailing list for a project and getting no real response.  I’m getting optimistic that we can move to a forum soon; could we use that forum in a way to help facilitate something like what you are proposing? 

> On Nov 8, 2017, at 4:30 AM, Wallacy via swift-evolution <swift-evolution at swift.org> wrote:
> 
> @Félix Fischer
> 
> Yes, I wanted to say central index. I chose the wrong term.
> 
> Actually I could have summed it up like this:
> - Central Index
> - SwiftPM to download/search using this index like npm/nuget
> - GStreamer organization Style.
> 
> I do not agree with Ted that only a few projects should be ranked, everyone, as it is in npm should be available. Only be graded according to recommendations.
> 
> For exemple this is what GStreamer do:
> 
> gst-plugins-base
> a small and fixed set of plug-ins, covering a wide range of possible types of elements; these are continuously kept up-to-date with any core changes during the development series.
> 
> We believe distributors can safely ship these plug-ins
> People writing elements should base their code on these elements
> These elements come with examples, documentation, and regression tests
> gst-plugins-good
> a set of plug-ins that we consider to have good quality code, correct functionality, our preferred license (LGPL for the plug-in code, LGPL or LGPL-compatible for the supporting library).
> 
> We believe distributors can safely ship these plug-ins
> People writing elements should base their code on these elements
> gst-plugins-ugly
> a set of plug-ins that have good quality and correct functionality, but distributing them might pose problems. The license on either the plug-ins or the supporting libraries might not be how we'd like. The code might be widely known to present patent problems.
> 
> Distributors should check if they want/can ship these plug-ins
> People writing elements should base their code on these elements
> gst-plugins-bad
> a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors.
> 
> If the plug-ins break, you can't complain - instead, you can fix the problem and send us a patch, or bribe someone into fixing them for you
> New contributors can start here for things to work on
> 
> For example, let's say that the proposal on type Result <T> is not approved.
> 
> The community cam build a Result<T> "project" ate "swift-extension-base" and everyone at "base" level must agree to use then (or improve then) to avoid one of the concerns at Result<T> proposal (several projects build the own)...
> 
> So in Swift side maybe the indexer (and the community) can classify projects in this way:
> 
> swift-extension-base  // (Swift Core Team maybe can manager this one)
> 
> a small and fixed set of features, covering a small range of possible types of elements; these are continuously kept up-to-date with any core changes during the development series.
> 
> We believe distributors can safely ship these features
> People writing elements should base their code on these elements
> These elements come with examples, documentation, and regression tests
> swift-extension-good // (Community can manage this, and Swift Core Team can do some advices)  
> a set of features that we consider to have good quality code, correct functionality, our preferred license (Swift Licence), and covering a wide range features.
> 
> We believe distributors can safely ship these plug-ins
> People writing elements should base their code on these elements
> swift-extension-ugly // (Full Community)  
> a set of features  that have good quality and correct functionality, but distributing them might pose problems. The license on either the or the supporting libraries might not be how we'd like. The code might be widely known to present patent problems.
> 
> Distributors should check if they want/can ship these plug-ins
> People writing elements should base their code on these elements
> swift-extension-bad // (Full Community)  
> a set of features that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use.
> 
> If the features break, you can't complain - instead, you can fix the problem and send us a patch, or bribe someone into fixing them for you
> New contributors can start here for things to work on
> 
> I just change plugins to feature to make more sense here... And at "base" level we can just drop very small projects, and make few guarantees that will always work. And at good level we can drop bigger projects like Alamofire, Perfect etc, because of the nature of those projects, we know that will get a good support but we cant guarantee that documentation and tests and language updates will always be pair with the rest.
> 
> And at "bad" level, will be like NPM is today... just a easy way to obtain some 3rd party code but no guarantees at all!
> 
> We heve IBM Swift Package Catalog (https://packagecatalog.com <https://packagecatalog.com/>) but i think we can do better.
> 
> Also, if we (and Apple) do nothing, the community someway will do! It's like Brew/Macports at some point people will build the tools to make the ecosystem easier. Is better make something oficial (like npm/nuget) of course.
> 
> A "Non-Standard Libraries" its a non start for me, but good indexer, a good integration with SwiftPM, a oficial help/support of the Swift Core Team, and a set of rules/patterns to rule this (and other) projects is a good feature to have.
> 
> Em ter, 7 de nov de 2017 às 21:16, Alejandro Martinez via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> escreveu:
> I’m with Ted on this one. 
> I would also love to see the Swift ecosystem grow but I think that it has to happen with SPM. With improvements on SPM (as discussed in other threads) and having a proper index (imho Cocoapods webpage is the best one out there, with stats about docs, unit testing, downloads...) that makes finding libraries easier. 
> But also a discussion forum where great libs can be shared and highlighted to the rest of the community and big community efforts coordinated. If you look at Rust is a great example, their forums attract the whole community (sadly not something the mailing list does) and big and important projects have come up from the community that have made a huge impact. With that in place and some libs out there maturing I’m sure that in the future the conversation to include battle tested code into an oficial distribution would be easier.
> 
> Sent from my iPad
> 
> On 7 Nov 2017, at 21:58, Ted Kremenek via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
> 
>> Hi Dave,
>> 
>> Thanks for bringing up this topic.  This has been kicked around a little, and we’re still exploring different models on how to extend Swift.
>> 
>> The server APIs work group is one operational model for the community to build out a new set of core libraries.  That work group was formed out of the observation that there were different implementations of the same thing being created by different Swift on server efforts, and that those different efforts should pool those resources together and standardize on some fundamentals.
>> 
>> That analogy could work here.  However, it also has possible major downsides.  It can be heavyweight, and also artificially raise the importance of certain people’s library efforts over others by creating a formal work group whose efforts are expected to eventually be incorporated into the core Swift distribution.  It would also force a discussion up front of what even makes sense to incorporate into the core Swift distribution, which might be artificially constraining the exploration of the set of libraries to implement.
>> 
>> I need to think about your idea more, but my first reaction is that my preferred route is that the community builds these libraries, ideally using Swift Packages, and through trial and use we evaluate those libraries and then decide if they should be incorporated as part of the core distribution.  There’s many factors involved in deciding if they can be incorporated into the core distribution, but all of those could be informed by actually building libraries and see them getting used.
>> 
>> We could also figure out a way to possibly highlight these efforts to the Swift community, maybe on swift-evolution or other means — but all with the expectation that these libraries are not *necessarily* going to be standardized as part of the core swift distribution.  I don’t think that’s all that bad either; not every library will make sense to incorporate into the core Swift distribution (e.g., they are highly domain specific) but still supporting their development would be beneficial to the community.
>> 
>> Note that any change going into the Swift core distribution inevitably involves swift-evolution and the core team.  Changes going into the core Swift distribution must meet a very high bar as far as implementation and design, the confidence we have into committing to specific APIs (especially since we’ll have ABI stability in Swift 5), and other factors.  Thus this will not really alleviate much burden on the Core team or on the community — one of the core goals of your proposal.  Further, incorporating all those concerns up front when building libraries might be counterproductive.
>> 
>> FWIW, Ben Cohen and I have been talking about possibly using Swift packages as a way to seed out experimental ideas for extensions to the Standard Library.  This would allow ideas to be trialed by real usage (a complaint I’ve seen about some changes we’ve made to Swift in the past).  Users could build things on top of those libraries, knowing they are available as packages, and if an API “graduates” to being part of the Standard Library the user can then depend upon it being available there.  If it never graduates, however, the package remains around.
>> 
>> One thing that resonates me in what you propose is about having a “well-organized effort” whose aim is to make these libraries feel cohesive and implemented well.  However, given that many of these naturally fall somewhere in the spectrum of responsibilities within the Standard Library and Foundation, I think it is self-misleading to think that the Core Team or others would not be involved in these efforts if the intention is to possibly incorporate these one day into the core Swift distribution.  There also may be other ways to achieve that level of cohesion in API design, such as through community discussion (possibly via the Swift.org <http://swift.org/> mailing lists/forums).  This discussion would not necessarily be the same as the path to “ratification” of a library into core Swift, but a step that could benefit many library authors (including considering ideas one day incorporated into the core swift).  With such a mechanism many library authors could benefit even if they do not intend to have the library as part of the core distribution.
>> 
>> My apologies that these are all hand-wavy ideas, but I’m trying to paint a possible alternative way to achieve your goal of more library evolution for Swift without having such a formal work group that may be too heavy weight or too narrowly focused.
>> 
>> Ted
>> 
>>> On Nov 7, 2017, at 9:54 AM, Dave DeLong via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>> Hi Swift-Evolution,
>>> 
>>> The Standard Library's goal is to be small and targeted. However, many aspects of Apple-provided frameworks need or offer opportunities for improvement or wholesale replacement. These enhancements lie beyond the scope of the Standard Library.
>>> 
>>> To address this, we'd like to propose the idea of a "Non-Standard Library"; this would be a library that ships with a regular installation of Swift, but is not imported into .swift files by the compiler, unless explicitly requested by the developer.
>>> 
>>> We are proposing a well-organized effort to parallel the Standard Library without putting additional implementation responsibilities onto the core team. This effort would mitigate what we see as platform-independent requirements that provide native Swift implementations that aren't burdened by Apple history.
>>> 
>>> Mission Statement
>>> 
>>> We propose to create an open-sourced "Non-Standard Library" effort that coexists with, coordinates with, and is blessed by the open source Swift development community. The "Non-Standard Library" will offer a well-curated, high-value collection of routines, types, and documentation to support Swift development on all platforms.
>>> 
>>> Goals
>>> 
>>> The main goals of this effort would be:
>>> Alleviate pressure on the Core Team while providing the developer community with functionality that normally falls under Apple's internal development umbrella.
>>> Deliver authoritative, reliable, and regularly updated libraries avoiding issues faced by third-party dependencies.
>>> Provide oversight, organization, and full community involvement to ensure its components are worthy, maintained, and passing a bar of need and excellence.
>>> Suggested Libraries
>>> 
>>> There are several areas we think that are ripe for improvement and implementation as a non-standard library, such as (but not limited to):
>>> A BigNum library
>>> A full-featured Random library
>>> A simplified date/time library
>>> A library for manipulating paths (that is not based on URL or String)
>>> An expanded Swift-native Geometry library and so forth.
>>> The scope and extent of the sublibraries would be proposed and debated on a parallel Non-Standard Library Evolution development list.
>>> 
>>> Coordination
>>> 
>>> This effort would be fully coordinated with the Swift development team, respecting the team's existing commitments and responsibilities. We would request an Apple body to act as an official coordinator, enabling both oversight of this effort and to expose Apple-sourced suggestions to the Non-Standard community for action.
>>> 
>>> Next Steps
>>> 
>>> To proceed, we need a general community consensus that this effort is worth the significant overhead it would involve.
>>> 
>>> We must then:
>>> Select a project lead, who appoints technical leaders from the community.
>>> Recruit a core team, a small group responsible for strategic direction, pulled from experienced members well versed in contributing to Swift-dev.
>>> Establish a Non-Standard Swift Evolution process, based on the one that is currently followed by Swift Evolution. Following SE practices will guarantee a consistent and excellent language experience for those developers including the Non-Standard library into their projects.
>>> Build a Non-Standard Swift Evolution repository home.
>>> Adopt a code license, based on Swift's current licensing.
>>> Define the community working group rules and code of conduct.
>>> Establish a mailing list and/or forum.
>>> Begin the selection of target sublibraries and populating them by recruiting committers and contributors.
>>> Alternative Names
>>> 
>>> Suggested names for this effort include the following.
>>> Extended-Standard Library
>>> Community-Standard Library
>>> Non-Standard Library
>>> We are not married to any of these names and welcome alternative suggestions.
>>> 
>>> Measures of Success
>>> 
>>> A successful Non-Standard Library will provide a stable and incrementally grown ecology of Swift frameworks that developers can depend upon with a minimum of turbulence and regret cycles. For that reason, the most successful content will be core functionality, with significant field testing prior to adoption. We recommend that NSSE follow a model of provisionary adoption and refinement before deployment to ensure any adopting code base will not be affected by later module changes.
>>> 
>>> Thanks,
>>> 
>>> Dave DeLong and Erica Sadun
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171108/13806197/attachment-0001.html>