<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 13, 2015, at 3:05 PM, Drew Crawford <<a href="mailto:drew@sealedabstract.com" class="">drew@sealedabstract.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="" style="font-family: HelveticaNeue;">Yup. I can't say I've had the time to seriously study cargo and the state of its ecosystem, but I am aware of it. I'd appreciate pointers if there are specific things you are thinking about.</div></blockquote><br class=""></div><div class="">Well, the bullet list of learning from cargo:</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class="">Enforce a consistent file structure on sourcecode (which we do)</li><li class="">Consider using [swift] as a scripting language for external builds rather than bash</li><ul class=""><li class="">swift as manifest seems more powerful than is strictly necessary, but I do see some advantages too</li></ul></ul></div></div></div></blockquote><div><br class=""></div><div>I can see how this might be useful, but don't really see what it buys you if that just turns around and calls something as open ended as configure. Can you say more about why this is a good thing?</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><ul class="MailOutline"><li class="">Let the user choose static link, dynamic link, debug, release, --with-openssl, --target=iOS, etc. at the CLI</li></ul></div></div></div></blockquote><div><br class=""></div>Yup, Max and I have talked about this, I assume things like this will come eventually. I agree cargo's system here looks pretty well done, although I would also like to know more about how it is working in practice.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><ul class="MailOutline"><li class="">Have a synonym for "make check"</li></ul></div></div></div></blockquote><div><br class=""></div>Part of current testing proposal.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><ul class="MailOutline"><li class="">Have a .lock file that records the dependency revs actually used</li></ul></div></div></div></blockquote><div><br class=""></div>Yup, this is part of the plan.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><ul class="MailOutline"><li class="">Do something sane when the dependency repository is a git URL to an auto tools project</li></ul></div></div></div></blockquote><div><br class=""></div>I'm less interested in trying to do something automatic here. If someone comes up with something that "just works" in a huge % of cases once the rest of things are designed I would gladly take it. If it doesn't "just work" most of the time, I would much rather force someone to write the adaptor package if it means that all consumers get a more robust development experience, that seems like a better tradeoff.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">I would +1 most of your proposal (prefer llbuild, scripting escape hatch, tagged sources, etc.) but I forsee a potential problem on this point:</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class="">One of my personal beliefs is that part of the reason software packaging & integration has historically been so messy is because no one has been able to be a "force for change" and cause projects to adapt. For C/C++/Obj-C projects, there are a lot of "unnecessary" variations in the project structure which add complexity for very little overall gain. </blockquote></div><div class=""><br class=""></div><div class="">While this is true, this seems like an <a href="https://xkcd.com/927/" class="">https://xkcd.com/927/</a> kind of statement. I think this project has the potential for moral authority (or "force for change" as you put it), but the exact magnitude of the moral authority is unknown and I estimate it lower than perhaps you do.</div><div class=""><br class=""></div><div class="">I think swift-pm derives its moral authority from two facts:</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class="">That it's an official, <a href="http://swift.org/" class="">swift.org</a> project, like Rust's cargo, go get, etc.</li><li class="">That people want a build system for Linux now and so they have to rethink their packaging anyway</li></ul><div class=""><br class=""></div></div><div class="">However these two facts are only relevant for **swift** projects. That is to say, libpng already builds on Linux and is not actively shopping for a new build system. Nor do they care about packaging with the Approved Method™ from <a href="http://swift.org/" class="">swift.org</a>, a language they do not even use. And when push comes to shove, swift projects need libpng more badly than they need the Approved Packaging Method™.</div><div class=""><br class=""></div><div class="">tl;dr it is actually the C dependencies here which have the moral authority. Any conflict between what swift-pm wants and what libpng wants will be resolved in favor of libpng. You cannot tell swift developers to go apply pressure to libpng to switch build systems; we lack standing.</div></div></div></blockquote><div><br class=""></div>I am generally a believer in "if you build it *better*, they will come". My goals here are not to encourage change by being a moral authority, but by simply providing a better solution to real problems than already exists, even for projects that don't care about Swift.</div><div><br class=""></div><div>I've been a C/C++ developer a lot longer than I have been a Swift developer, and I think it is a real active pain point in those communities that there is no easy way to glue packages together. So what I am really shooting for here is a system that is so good, its worth it for a project like libpng to support just so that it is easier for their C clients to use.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class="" style="font-family: HelveticaNeue;">Well, we don't really support any of these features yet so our current advisement would be "we need to design this feature". :)</div></blockquote><br class=""></div><div class="">Well yes, but in the context of the above, I'm suggesting that the C dependency story and courting libpng etc (if that is in fact important) is more important than nearly any other problem. I can live for a few months without unit tests, configuration options, build environment isolation, versioning, etc. I can PR those as I need them, it is mostly obvious what is the right patch to write, from the README alone.</div><div class=""><br class=""></div><div class="">I can't build without libsodium. I can't build without libpng. So forget about months, I can't get ten minutes into being a user even on one platform without making a decision on how to build C dependencies. And it is much less obvious what the right thing to PR here is.</div></div></div></blockquote><div><br class=""></div>Well, for now the answer is that you have to install your C dependencies with a system package manager and then use the modules maps feature. We know there are issues even with that approach (like the portability of the module maps), and that seems like one area to tackle with individual PRs and more focused design discussions. If there are specific issues blocking this approach for "just getting it working" lets discuss that separately.</div><div><br class=""></div><div>I agree this is important and will be actively working on this area, starting with a proposal for building C family sources which are part of the package. That should go some way to supporting more integration with C based stuff, and will also hopefully start to clarify what 2.b would look like.</div><div><br class=""></div><div> - Daniel</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">I think it is potentially something like your 2b, but it seems presently underspecified ("We need to be very careful about doing #2.b in a way that doesn't restrict our ability to evolve") and I'm uncertain if there's anything concrete that can be done to get it into an implementable state.</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class="">On Dec 13, 2015, at 1:55 PM, Daniel Dunbar <<a href="mailto:daniel_dunbar@apple.com" class="">daniel_dunbar@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><br class="Apple-interchange-newline">On Dec 12, 2015, at 3:46 PM, Drew Crawford <<a href="mailto:drew@sealedabstract.com" class="">drew@sealedabstract.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><blockquote type="cite" class=""><span class="" style="font-family: HelveticaNeue;">1. Support a great development experience for building projects from source, where we build everything into the sandbox for that project. We would like to eventually support being able to build more complex dependencies (like external C projects) as part of a package build.</span></blockquote><blockquote type="cite" class=""><span class="" style="font-family: HelveticaNeue; float: none; display: inline !important;">2. Leave deployment and installation of packages to system package managers (i.e., someone else).</span></blockquote><div class=""><br class=""></div>Okay, this is good information. The output of the package manager is a .a (or .so, .dylib, Mach-O executable, etc.) and it is your job to install that to a path on your system, or not. This seems like a good model. I might suggest updating the README in some way that explains this.</div></div></blockquote><div class=""><br class=""></div>Good point, we don't really have any docs on the output. I don't have time to tackle this currently, but if patches here would be welcome. I believe Max also has some plans to make the build products more obvious (not hidden in the .build dir), the current system hasn't really been "designed" yet.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">This is actually vaguely analogous to Rust's cargo, so I would suggest looking there for design concepts, as they are quite a bit further ahead of us on this problem.</div></div></div></blockquote><div class=""><br class=""></div>Yup. I can't say I've had the time to seriously study cargo and the state of its ecosystem, but I am aware of it. I'd appreciate pointers if there are specific things you are thinking about.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><blockquote type="cite" class=""><span class="" style="font-family: HelveticaNeue; display: inline !important;">If you look at how easy and portable it is to install projects with complex dependencies in ecosystems like Python + virtualenv, that is the kind of environment we want for Swift and eventually for the C family dependencies.</span></blockquote><br class=""></div><div class="">I'm (sadly) intimately familiar. It is not really all that easy, primarily because of binary hell, as anyone who has googled how to install pycairo can attest. I really want to do better on this problem than Python does.</div></div></div></blockquote><div class=""><br class=""></div>I think it depends on the project. For web stacks it can be awesome to pull down a dependency tree w/ C extensions for PostgreSQL, Redis, etc. and a lot of stuff "just works". For more complicated C projects I agree it isn't a slam dunk.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">Here's a concrete question: how would we advise a hypothetical swiftcairo project to package themselves in the package manager? Should they:</div></div></blockquote><div class=""><br class=""></div>Well, we don't really support any of these features yet so our current advisement would be "we need to design this feature". :)</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">1. Call out to a shell script that does `brew install cairo` on OSX, `apt-get install cairo` on Debian, etc.</div></div></div></blockquote><div class=""><br class=""></div>Currently, this is the only supported thing. But I want to do better than this...</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">2. Print some error telling them to come back when they've installed cairo somewhere on their system. Good luck, kid</div><div class="">3. Write a shell script that downloads and builds cairo from source including all its dependencies</div><div class="">4. Create a tree in the swift package manager like</div><div class=""><br class=""></div><blockquote class="" style="margin: 0px 0px 0px 40px; border: none; padding: 0px;"><div class=""><font face="Menlo" class="">swiftcairo <-- cairo <-------pixman</font></div><div class=""><font face="Menlo" class=""> <-------freetype</font></div><div class=""><font face="Menlo" class=""> <-------fontconfig</font></div><div class=""><font face="Menlo" class=""> <-------libpng</font></div><div class=""><font face="Menlo" class=""> <-------glib</font></div></blockquote><div class=""><div class=""> Where each of these "packages" are mostly a shell script that runs a lot of `curl -O` `tar xf` `configure` `make` etc.</div><div class="">5. Do we try and second-guess what libpng's autotools want to do? (Bad, bad, idea. Build the way libpng decided to build, don't try to guess based on the arrangement of .c files in a folder)</div><div class="">6. Other ideas?</div><div class=""><br class=""></div><div class="">A vision for the pathological case like Cairo will illuminate the simpler cases. IMO it would be nice if "depends: swiftcairo" would 'just work', but that may be out of scope.</div></div></div></div></blockquote><div class=""><br class=""></div>My proposal is:</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">1. We include some native support for C/C++ libraries. This should be "well designed", but ideally would scale up to supporting very support complex projects (like `swiftc`). I care about this part deeply because there is a lot of awesome stuff I want to do in llbuild + clang, and that depends on being able to get the project onto those tools. We will by necessity need to start with a small limited scope and feature creep our way forward.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">2. We provide an adaption mechanism, to allow projects that can't yet be built by #1 to still integrate directly into the ecosystem. This can take two forms:</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">2.a. For projects which *could* conform to #1, but the only problem is the upstream project doesn't want to change (or hasn't yet), then we could support an adaption mechanism which just took in the upstream project and overlayed the appropriate Package.swift definitions that we need to use to build it. This means that swiftpm + llbuild would have a full understanding of the build process, which is the ideal scenario for the ensuring a good user experience.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">2.b. For projects which are just "too complicated" to do #2.a, we would allow a form of your #3 (a shell script which downloads and builds). The way I envision this working is that:</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">- I'd like to avoid the script actually downloading anything, I think we should still try and use our model for pulling the sources. This may mean someone maintains a tagged version of an adaptor package that includes the source of the upstream project + the adaptor pieces.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">- We would require a very strict interface between that script and the rest of the package manager. Essentially, we would provide the script with its inputs, and we would expect it to produce, as an output, the exact kind of build layout that we ourselves would have done for project in case #2.a. In practice, those scripts would usually look something like " configure && make && ... move the built products around a bunch to conform to the expected layout ...". We would run the script in appropriate dependency order, then ingest / audit all of the outputs, and from then on it would integrate just like #2.a.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">We need to be very careful about doing #2.b in a way that doesn't restrict our ability to evolve...</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">One thing I would like to call out is that I don't feel like we need to directly support the total amount of complexity that exists in the world w.r.t. C family projects. I want use to help make the world more sane by coming up with sensible conventions and models for C family projects, and then apply pressure in other places to reduce unnecessary complexity or to converge on semi-standard solutions. I outlined a bit more of my philosophy here in:</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""> <a href="https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20151130/000027.html" class="">https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20151130/000027.html</a></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><span class="" style="font-family: HelveticaNeue; display: inline !important;"> (and if you give me pointers to the full tree I would be interested in checking them out as test cases), this feature may be enough that you could in theory get your Swift package to also build the necessary C libraries (with some adaption work). Stay tuned...</span></blockquote><br class=""></div><div class=""><a href="https://code.sealedabstract.com/drewcrawford/NaOH" class="">NaOH</a> is a good test case. It's a framework that statically links libsodium. My current (terrible) workflow is:</div><div class=""><br class=""></div><div class="">1. Download <a href="https://github.com/jedisct1/libsodium/releases" class="">libsodium-xx.tar.gz</a> from GitHub</div><div class="">2. ./configure && make</div><div class="">3. Move binaries + .h around to where Xcode can find them</div><div class="">4. Check into source control</div><div class="">5. Build</div><div class=""><br class=""></div><div class="">Would love to automate this and get it building with the package manager. There's actually been an <a href="https://code.sealedabstract.com/drewcrawford/NaOH/issues/1" class="">issue open</a> since pretty much day 1 to get this working. That is the problem that motivated me to post. </div></div></div></div></blockquote><div class=""><br class=""></div>Cool, thanks for the pointer I will check it out.</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""> - Daniel</div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><div class=""><br class=""></div><div class=""><br class=""><div class=""><br class=""></div><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Dec 12, 2015, at 11:02 AM, Daniel Dunbar <<a href="mailto:daniel_dunbar@apple.com" class="">daniel_dunbar@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">I would say that our main goals are two fold:</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">1. Support a great development experience for building projects from source, where we build everything into the sandbox for that project. We would like to eventually support being able to build more complex dependencies (like external C projects) as part of a package build.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">2. Leave deployment and installation of packages to system package managers (i.e., someone else).</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">As a good example of #1, it would be awesome if in the future that even a project like the Swift compiler itself -- with its complicated dependency tree and build process -- could be configured and built with the package manager.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">The goals here are mostly oriented around how do we give the best possible developer experience for constructing source code. I strongly believe that the "container" / "sandbox" model is the right way to go in order to build tools which give reproducible results, allow for easy sharing and deployment, and encourage modular development. If you look at how easy and portable it is to install projects with complex dependencies in ecosystems like Python + virtualenv, that is the kind of environment we want for Swift and eventually for the C family dependencies.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">Goal #2 of trying to stay out of the system package manager business (installing built software) is more out of trying to maintain a clear focus and emphasis than anything else. We already have a ton of work to do on the package manager even within that scope, so it just makes sense for now to leave this problem to existing projects better suited to the task.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">FWIW, I am working on a proposal for our initial support for *building* C family sources as part of Swift packages. Depending on exactly how complicated your dependencies are (and if you give me pointers to the full tree I would be interested in checking them out as test cases), this feature may be enough that you could in theory get your Swift package to also build the necessary C libraries (with some adaption work). Stay tuned...</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">On Dec 12, 2015, at 5:04 AM, Drew Crawford via swift-build-dev <<a href="mailto:swift-build-dev@swift.org" class="">swift-build-dev@swift.org</a>> wrote:<br class=""><br class="">I noticed the README for the package manager seems to take a strong view of source distribution:<br class=""><br class=""><blockquote type="cite" class="">Packages are distributed and consumed as source code, rather than pre-compiled binaries.<br class="">Although it requires additional computational resources, this approach guarantees that developers can adopt new features on platforms they support, without being reliant on vendors to supply updated dependencies. This also has the advantage of allowing tools to do things like automated testing and API analysis of package dependencies.<br class=""></blockquote><br class="">I maintain a lot of Swift software with C dependencies. For example, I develop Swift frameworks that ship with libsodium statically linked. AFAIK the new Foundation will have various C dynamic and static dependencies. Etc.<br class=""><br class="">What is the vision for distributing this software with the package manager?<br class=""><br class="">1. Will it barf if the user does not have the C dependencies already installed on their machine?<br class=""> 1a. What about iOS, where there is no concept of this? Can I not build iOS libraries using the package manager?<br class=""></blockquote><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">Yes, it would barf. We are open to improving the reliability of the "system modules" mechanism to be more robust. We do need to support interoperating with installed libraries, and it would be great to make this as smooth as possible.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">2. Are all binary packages forbidden? Is there any mechanism like homebrew's bottles for distributing software that is arcane to compile?<br class=""></blockquote><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">We don't have such a mechanism, and as I said the goal of the package manager is to build your project, not distribute it as installable packages. I can imagine adding mechanisms to *help you create* an easily installable product, but I think the actual installation of that product should be left to other tools.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">3. Can I write a script that tries to resolve various C dependencies my project needs to link with?<br class=""></blockquote><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">Not currently. What exactly are you envisioning here?</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">4. Is it a design goal to distribute Foundation via the package manager?<br class=""></blockquote><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">I'm not sure what you mean. Foundation itself is a special case, it is something we consider part of the Swift corelibs and will ship with the Swift compiler.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">We *would* like to be able to build the Foundation project with the package manager, but there are a lot of things that need to get sorted out before that can happen.</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">HTH,</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">- Daniel</span><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: HelveticaNeue; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">_______________________________________________<br class="">swift-build-dev mailing list<br class=""><a href="mailto:swift-build-dev@swift.org" class="">swift-build-dev@swift.org</a><br class=""><a href="https://lists.swift.org/mailman/listinfo/swift-build-dev" class="">https://lists.swift.org/mailman/listinfo/swift-build-dev</a></blockquote></div></blockquote></div></div></div></div></div></div></blockquote></div></div></blockquote></div><br class=""></div></div></blockquote></div><br class=""></body></html>