[swift-build-dev] SwiftPM recompilation - Packages folder

[Ricardo N Olivieri]

Got it, thanks Daniel. Just in case we get the cycles to work on a PR, 
could you point us to the file (or set of files) in the 
https://github.com/apple/swift-package-manager repo that we'd need to 
modify so that the swift-build-tool only looks at the time/size metadata? 
I must admit I am not quite familiar yet with the internals of SwiftPM.

Regards,
   Ricardo Olivieri
   Software Engineer
   IBM Swift Engineering at IBM Cloud


From:   Daniel Dunbar <daniel at zuster.org>
To:     Ricardo N Olivieri/Austin/IBM at IBMUS
Cc:     swift-build-dev at swift.org
Date:   11/21/2016 04:41 PM
Subject:        Re: [swift-build-dev] SwiftPM recompilation - Packages 
folder
Sent by:        daniel.dunbar at gmail.com



On Mon, Nov 21, 2016 at 2:37 PM, Ricardo N Olivieri <
ricardo.olivieri at us.ibm.com> wrote:
Thanks Daniel for the response. That's probably the reason (i.e. metadata 
changes in the file inode) for the behavior we are seeing in SwiftPM.

The use case for us is as follows: 

We'd like to improve even further the performance of the IBM Bluemix 
buildpack for Swift and one way we could accomplish this is by not 
recompiling the Packages folder as long as the content of that folder has 
not changed. Without getting into too many details about cloud foundry 
buildpacks, for the buildpack to reuse the contents of the Packages folder 
in future re-deployments of the same application to the cloud, the 
buildpack has to copy the Packages folder a different folder outside the 
repo's folder and then copy it back again when the developer redeploys 
his/her application. This is when we noticed that despite the contents of 
the Packages folder had not changed, SwiftPM still recompiled all of the 
dependencies under the Packages folder.

Makes sense, thanks!

Is there an ETA for using content-based signatures instead? 

No, unfortunately.

If it is important for your use case, I'd be ok with accepting a PR to 
change swift-build-tool to only look at the time/size information to 
workaround this in the meantime.

 - Daniel
 



Regards,
  Ricardo Olivieri
  Software Engineer
  IBM Swift Engineering at IBM Cloud


From:        Daniel Dunbar <daniel at zuster.org>
To:        Ricardo N Olivieri/Austin/IBM at IBMUS
Cc:        swift-build-dev at swift.org
Date:        11/21/2016 03:58 PM
Subject:        Re: [swift-build-dev] SwiftPM recompilation - Packages 
folder
Sent by:        daniel.dunbar at gmail.com




swift-build-tool will currently rebuild any file if it detects a change to 
the file inode or device -- my guess is that is what you are seeing here.

My intention is to eventually move to using content-based signatures to 
determine the rebuild conditions (for tools where we know that is safe), 
which would prevent the rebuild in this situation.

Is there a particularly reason it is important to you that this not 
rebuild?

 - Daniel

On Mon, Nov 21, 2016 at 1:54 PM, Ricardo N Olivieri via swift-build-dev <
swift-build-dev at swift.org> wrote:
Hello all, 


I was wondering if someone could shed some light on how SwiftPM determines 
if it needs to compile again any of the source code under the Packages 
folder. I had thought that the modified timestamp attribute of any source 
code files under the Packages folder was the key for determining if any of 
the Swift package dependencies should be recompiled. However, it seems 
SwiftPM uses additional criteria to make this determination. Here’s the 
test we executed: 


1) At T0, we executed swift build. As expected all Swift package 
dependencies were cloned and compiled along with the application’s code. 



2) At T1, we executed swift build again and nothing was recompiled, as 
expected since no code changes were made. 


3) At T2, we made a copy of the Packages folder and place this copy in a 
location outside the repo’s directory. We then deleted the Packages folder 
from the repo’s directory, and, finally, we copied the Packages folder 
back to its original location in the repo’s directory. 


4) At T3, we executed again swift build and noticed that SwiftPM 
recompiled each one of the Swift package dependencies in the Packages 
folder, even though no code changes were made. We also double checked that 
the modified and creation timestamps for the Packages folder and folders 
and files in it were not changed. After doing so, we concluded that though 
copying the Packages folder back to the application’s repo did keep the 
original creation and modified timestamp values of all files and folders, 
SwiftPM still recompiled all of the dependencies in the Packages folder. 


Does anyone know why we are seeing in SwiftPM the behavior described 
above? 


Regards,
  Ricardo Olivieri


_______________________________________________
swift-build-dev mailing list
swift-build-dev at swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20161122/3ff7835b/attachment.html>