update package deps for custom project root

[vanvoorden]

One small improvement to package manifest:

• We follow the pattern from swift-corelibs-foundation¹ to compute the location of dependencies:
  • SWIFTCI_USE_LOCAL_DEPS=1 moves us up one directory when looking for dependencies.
  • Missing or empty SWIFTCI_USE_LOCAL_DEPS pulls dependencies from GitHub.

This also keeps us consistent with how we compute a local path for Benchmarks.²

Tested locally from my machine:

$ cd ~/Developer/swift-foundation
$ swift build
...
Build complete!

$ SWIFTCI_USE_LOCAL_DEPS=~/Developer swift build
...
Build complete!

$ SWIFTCI_USE_LOCAL_DEPS=~ swift build
...
error: the package at '/Users/rick/swift-collections' cannot be accessed (Error Domain=NSCocoaErrorDomain Code=260 "The folder “swift-collections” doesn’t exist." UserInfo={NSUserStringVariant=(
    Folder
), NSFilePath=/Users/rick/swift-collections, NSURL=file:///Users/rick/swift-collections, NSUnderlyingError=0x6000020b0e40 {Error Domain=NSOSStatusErrorDomain Code=-43 "fnfErr: File not found"}})
error: ExitCode(rawValue: 1)

Footnotes

1. https://github.com/swiftlang/swift-corelibs-foundation/blob/swift-6.1.2-RELEASE/Package.swift#L155-L181 ↩

2. https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Package.swift#L62 ↩

[vanvoorden]

https://github.com/swiftlang/swift/blob/swift-6.1.2-RELEASE/utils/swift_build_support/swift_build_support/products/swiftfoundationtests.py#L83

@jmschonfeld I found one place we run swift-foundation tests with SWIFTCI_USE_LOCAL_DEPS=1… can you think of any place we are trying to run swift-foundation tests with some arbitrary custom SWIFTCI_USE_LOCAL_DEPS=path? Or could you think of any more places this change might lead to problems?

[jmschonfeld]

@swift-ci please test

[jmschonfeld]

https://github.com/swiftlang/swift/blob/swift-6.1.2-RELEASE/utils/swift_build_support/swift_build_support/products/swiftfoundationtests.py#L83

@jmschonfeld I found one place we run swift-foundation tests with SWIFTCI_USE_LOCAL_DEPS=1… can you think of any place we are trying to run swift-foundation tests with some arbitrary custom SWIFTCI_USE_LOCAL_DEPS=path? Or could you think of any more places this change might lead to problems?

All of our automated testing should be using SWIFTCI_USE_LOCAL_DEPS=1 - that's what build.ps1 does for Windows CI and what that python file does for Linux CI. The ability to specify a path instead was something we just introduced for benchmarks so I don't think there's anything concrete that provides that today (aside from providing it via the command line locally itself)

[jmschonfeld]

We actually want to keep this at 1.1.0 instead of main since these versions should match what we use for building the toolchain. When building the swift toolchain, the build script builds against swift-syntax and swift-foundation-icu's main branch, but it builds against a tag of swift-collections since that package is developed outside of the toolchain and tagged every so often. When building a toolchain, this JSON file determines what is checked out: https://github.com/swiftlang/swift/blob/3bacfa7fb974e89ad7278f727ccd54a6643b87b6/utils/update_checkout/update-checkout-config.json#L142 (i.e. it determines what branch/tag is checked out and then specified via SWIFTCI_USE_LOCAL_DEPS when running in CI) so ideally these dependencies (for local package development outside of the toolchain) will use the same-ish dependencies

[vanvoorden]

@jmschonfeld Ahh… interesting! I can change this back. Is the idea to keep this version here and the version from update-checkout-config.json bound together manually? We don't have any easy way to keep that version defined in just one place and then queried dynamically?

[vanvoorden]

Does this all then imply that we don't currently have a Foundation CI job that builds against the main commit on Swift-Collections? If in theory a bad commit did land on Collections that broke something in Foundation we might not know about it until Collections pushes a new version tag?

[jmschonfeld]

Yes unfortunately these versions need to be kept in sync manually. update-checkout-config.json is considered the source of truth as it defines what refs of each project are checked out when building a toolchain. We also provide the dependencies in our SwiftPM and CMake manifests as conveniences to quickly check out / build dependencies locally without building the entire toolchain. Unfortunately there aren't any mechanisms today (and it would not be possible to build one for the SwiftPM manifest at least) that could read the dependency information from an external source. While this is a manual process, these versions are not typically updated frequently and the maintenance burden has been minimal so far.

And you're correct that if an issue landed on main of swift-collections that broke something in Foundation we may not find it until qualifying a new swift-collections tag. This is the case for many of the open source packages that are used by the toolchain. Foundation (being a public component of the toolchain) is always built in tangent with the toolchain but many other packages like swift-collections, swift-algorithms, swift-argument-parser, etc. all use stable tags instead of the unstable main branch when building in the toolchain.

[vanvoorden]

@jmschonfeld That all makes sense. Thanks!

[jmschonfeld]

@swift-ci please test