Rewrote motiviation

Author: dpaoliello

text/3529-cargo-path-bases.md

@@ -12,59 +12,109 @@ turn enable base-relative `path` dependencies.
 # Motivation
 [motivation]: #motivation
 
-While developing locally, users may wish to specify many `path`
-dependencies that all live in the same local directory. If that local
-directory is not a short distance from the `Cargo.toml`, this can get
-unwieldy. They may end up with a `Cargo.toml` that contains
+As a project grows in size, it becomes necessary to split it into smaller
+sub-projects, architected into layers with well-defined boundaries.
+
+One way to enforce these boundaries is to use different Git repos (aka
+"multi-repo"). Cargo has good support for multi-repo projects using either `git`
+dependencies, or developers can use private registries if they want to
+explicitly publish code or need to preprocess their sub-projects (e.g.,
+generating code) before they can be consumed.
+
+If all of the code is kept in a single Git repo (aka "mono-repo"), then these
+boundaries must be enforced a different way: either leveraging tooling during
+the build to check layering, or requiring that sub-projects explicitly publish
+and consume from some intermediate directory. Cargo has poor support for
+mono-repos: the only viable mechanism is `path` dependencies, but these require
+relative paths (which makes refactoring and moving sub-projects very difficult)
+and don't work at all if the mono-repo requires publishing and consuming from an
+intermediate directory (as this may very per host, or per target being built).
+
+This RFC proposes a mechanism to specify `base` directories in `Config.toml` or
+`Cargo.toml` files which can be used to prepend `path` dependencies. This allows
+mono-repos to specify dependencies relative to their root directory, which
+allows the consuming project to be moved freely (no relative paths to update)
+and a simple find-and-replace to handle a producing project being moved.
+Additionally, a host-specific or target-specific intermediate directory may be
+specified as a `base`, allowing code to be consumed from there using `path`
+dependencies.
+
+### Example
+
+If we had a sub-project that depends on three others:
+
+* `foo` which is in a different layer of the mono-repo.
+* `bar_with_generated` that must be consumed from an intermediate directory
+because it contains target-specific generated code.
+* `baz` which is in the current layer.
+
+We may have a `Cargo.toml` snippet that looks like this:
 
 ```toml
-foo = { path = "/home/jon/dev/rust/foo" }
-bar = { path = "/home/jon/dev/rust/bar" }
-baz = { path = "/home/jon/dev/rust/ws/baz" }
+[dependencies]
+foo = { path = "../../../other_layer/foo" }
+bar_with_generated = { path = "../../../../intermediates/x86_64/Debug/third_layer/bar_with_generated" }
+baz = { path = "../baz" }
 ```
 
-This is not only frustrating to type out, but also requires many changes
-should any component of the path change. For example, if `foo`, `bar`,
-and `ws/baz` were to move under a sub-directory of `libs`, all the paths
-would have to be updated. If they are used in more than one local
-project, each project would have to be updated.
+This has many issues:
+
+* Moving the current sub-project may require changing all of these relative
+paths.
+* `bar_with_generated` will only work if we're building x86_64 Debug.
+* `bar_with_generated` assumes that the `intermediates` directory is a sibling
+to our source directory, and not somewhere else completely (e.g., a different
+drive for performance reasons).
+* Moving `foo` or `baz` requires searching the code for each possible relative
+path (e.g., `../../../other_layer/foo` and `../foo`) and may be error prone if
+there is some other sub-project in directory with the same name.
 
-As related issue arises in contexts where an external build system may
-make certain dependencies available through vendoring. Such a build
-system might place vendored packages under some complex path under a
-build-root, like
+Instead, if we could specify these `base` directories in a `Config.toml` (which
+may be generated by an external build system which in turn invokes Cargo):
 
+```toml
+[base-paths]
+sources = "/home/user/dev/src"
+intermediates = "/home/user/dev/intermediates/x86_64/Debug"
 ```
-/home/user/workplace/feature-1/build/first-party-package/first-party-package-1.0/x86_64/dev/build/private/rust-vendored/
+
+Then the `Cargo.toml` can use those `base` directories and avoid relative paths:
+
+```toml
+[dependencies]
+foo = { path = "other_layer/foo", base = "sources" }
+bar_with_generated = { path = "third_layer/bar_with_generated", base = "intermediates" }
+baz = { path = "this_layer/baz", base = "sources" }
 ```
 
-If a developer wishes to use such an auto-vendored dependency, a
-contract must be established with the build system about exactly where
-vendred dependencies will end up. And since that path may not be near
-the project's `Cargo.toml`, the user's `Cargo.toml` may end up with
-either an absolute path or a long relative path, both of which may not
-work on other hosts, and thus cannot be checked in (or must be
-overwritten in-place by the build system).
-
-The proposed mechanism aims to simplify both of these use-cases by
-introducing named "base" paths in the Cargo configuration
-(`.cargo/config.toml`). Path dependencies can then be given relative to
-those base path names, which can be set either by a local developer in
-their user-wide configuration (`~/.cargo/config.toml`), or by an
-external build system in a project-wide configuration file.
-
-This effectively makes a "group" of path dependencies available at some
-undisclosed location to `Cargo.toml`, which then only has to know the
-layout to path dependencies _within_ that directory, and not the path
-_to_ that directory.
+Which resolves the issues we previously had:
+
+* The current project can be moved without modifying the `Cargo.toml` at all.
+* `bar_with_generated` works for all targets (assuming the `Config.toml` is
+generated).
+* The `intermediates` directory can be placed anywhere.
+* Moving `foo` or `baz` only requires searching for the canonical form relative
+to the `base` directory.
+
+## Other uses
+
+The ability to use `base` directories for `path` dependencies is convenient for
+developers who are using a large number of `path` dependencies within the same
+root directory. Instead of repeating the same path fragment many times in their
+`Cargo.toml`, they can instead specify it once in a `Config.toml` as a `base`
+directory, then use that `base` directory in each of their `path` dependencies.
+
+Cargo can also provide built-in base paths, for example `workspace` to point to
+the root directory of the workspace. This allows workspace members to reference
+each other without first needing to `../` their way back to the workspace root.
 
 # Guide-level explanation
 [guide-level-explanation]: #guide-level-explanation
 
 If you often use path dependencies that live in a particular location,
 or if you want to avoid putting long paths in your `Cargo.toml`, you can
-define path _base directories_ in your [Cargo
-configuration](https://doc.rust-lang.org/cargo/reference/config.html).
+define path _base directories_ in your Cargo [manifest](https://doc.rust-lang.org/cargo/reference/manifest.html)
+or [configuration](https://doc.rust-lang.org/cargo/reference/config.html).
 Your path dependencies can then be specified relative to those
 directories.
 
@@ -74,7 +124,7 @@ For example, say you have a number of projects checked out in
 `~/.cargo/config.toml`:
 
 ```toml
-[base_path]
+[base-paths]
 dev = "/home/user/dev/rust/libraries/"
 ```
 
@@ -93,60 +143,71 @@ the path must exist on any other host where you want to use the same
 # Reference-level explanation
 [reference-level-explanation]: #reference-level-explanation
 
-## Configuration
+## Specifying Dependencies
 
-`[base_path]`
+### Base Paths
 
-* Type: string
-* Default: see below
-* Environment: `CARGO_BASE_PATH_<name>`
+A `path` dependency may optionally specify a base path by setting the `base` key
+to the name of a base path from the `[base-paths]` table in either the
+[manifest](https://doc.rust-lang.org/cargo/reference/manifest.html) or
+[configuration](https://doc.rust-lang.org/cargo/reference/config.html#base-paths)
+or one of the [built-in base paths](#built-in-base-paths). The value of that
+base path is prepended to the `path` value to produce the actual location where
+Cargo will look for the dependency.
 
-The `[base_path]` table defines a set of path prefixes that can be used to
-prepend the locations of `path` dependencies. Each key in the table is the name
-of the base path and the value is the actual file system path. These base paths
-can be used in a `path` dependency by setting its `base` key to the name of the
-base path to use.
+For example, if the Cargo.toml contains:
 
 ```toml
-[base_path]
-dev = "/home/user/dev/rust/libraries/"
+[dependencies]
+foo = { path = "foo", base = "dev" }
 ```
 
-The "dev" base path may then be referenced in a `Cargo.toml`:
+Given a `[base-paths]` table in the configuration that contains:
 
 ```toml
-[dependencies]
-foo = { path = "foo", base = "dev" }
+[base-paths]
+dev = "/home/user/dev/rust/libraries/"
 ```
 
-To produce a `path` dependency `foo` located at
+This will produce a `path` dependency `foo` located at
 `/home/user/dev/rust/libraries/foo`.
 
+If the base path is not found in any `[base-paths]` table or one of the built-in
+base paths then Cargo will generate an error.
 
-## Specifying Dependencies
+If the name of a base path is specified in both the manifest and configuration,
+then the value in the manifest is preferred.
 
-A `path` dependency may optionally specify a base path by setting the `base` key
-to the name of a base path from the `[base_path]` table in the configuration.
-The value of that base path in the configuration is prepended to the `path`
-value to produce the actual location where Cargo will look for the dependency.
+The name of a base path must use only [alphanumeric](https://doc.rust-lang.org/std/primitive.char.html#method.is_alphanumeric)
+characters or `-` or `_`, and cannot be empty.
 
-If the base path is not found in the `[base_path]` table then Cargo will
-generate an error.
+#### Built-in base paths
 
-```toml
-[dependencies]
-foo = { path = "foo", base = "dev" }
-```
+Cargo provides implicit base paths that can be used without the need to specify
+them in a `[base-paths]` table.
 
-Given a `[base_path]` table in the configuration that contains:
+* `workspace` - If a project is [a workspace or workspace member](https://doc.rust-lang.org/cargo/reference/workspaces.html)
+then this base path is defined as the path to the directory containing the root
+Cargo.toml of the workspace.
 
-```toml
-[base_path]
-dev = "/home/user/dev/rust/libraries/"
-```
+If one of these built-in base paths is also specified in the manifest or
+configuration, then that value is preferred over the built-in value.
 
-Will then produce a `path` dependency `foo` located at
-`/home/user/dev/rust/libraries/foo`.
+## The Manifest Format
+
+[`[base-paths]`](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#base-paths) - Base paths for path dependencies.
+
+## Configuration
+
+`[base-paths]`
+
+* Type: string
+* Default: see below
+* Environment: `CARGO_BASE_PATHS_<name>`
+
+The `[base-paths]` table defines a set of path prefixes that can be used to
+prepend the locations of `path` dependencies. See the [specifying dependencies](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#base-paths)
+documentation for more information.
 
 # Drawbacks
 [drawbacks]: #drawbacks