Skip to content

feat: add subpath support for monorepo sparse checkout#47

Open
ChrisLally wants to merge 1 commit intovercel-labs:mainfrom
ChrisLally:feat/subpath-sparse-rust
Open

feat: add subpath support for monorepo sparse checkout#47
ChrisLally wants to merge 1 commit intovercel-labs:mainfrom
ChrisLally:feat/subpath-sparse-rust

Conversation

@ChrisLally
Copy link
Copy Markdown

Description

Fixes #27

Ports the sparse-checkout and monorepo subpath behavior from vercel-labs/opensrc#28 to the Rust CLI.

Summary

Adds support for fetching only a subdirectory from a GitHub (or other git host) monorepo instead of the full repo. Uses git sparse-checkout with --filter=blob:none so only the requested path is downloaded.

Sparse checkout under ~/.opensrc/repos/github.com/vercel/ai/main

^ Screenshot: opensrc path vercel/ai/packages/ai — cache directory ~/.opensrc/repos/github.com/vercel/ai/main with only the packages/ai tree checked out.

Supported syntax

opensrc path owner/repo/path/to/dir
opensrc path github:owner/repo/path/to/dir
opensrc path owner/repo/packages/pkg@main
opensrc path https://github.com/owner/repo/tree/main/packages/some-package
opensrc path vercel/ai/packages/ai

Implementation

  • Parsing (core/registries/repo.rs): RepoSpec / ResolvedRepo carry an optional subpath; repo specs with three or more path segments (and URL/tree forms) populate it.
  • Fetch (core/git.rs): When a subpath is present, clone uses --filter=blob:none --sparse and git sparse-checkout set <subpath> instead of a full checkout.
  • Cache (core/cache.rs): Repos live under ~/.opensrc/repos/<host>/<owner>/<repo>/<ref>/ (or $OPENSRC_HOME/repos/... if OPENSRC_HOME is set); the working tree only contains the sparse path.
  • CLI (commands/path.rs, commands/remove.rs): opensrc path resolves and prints the absolute directory; remove / sources.json handling understands repo specs with subpaths.

Why sparse checkout

  • Avoids cloning large monorepos when only one package is needed.
  • Uses Git’s built-in sparse checkout instead of many GitHub API calls.
  • Works with any git host (GitHub, GitLab, Bitbucket).

Testing

  • Unit tests in core/registries/repo.rs (and related) for subpath parsing across owner/repo/..., github:, and URL/tree forms.
  • cargo test --manifest-path packages/opensrc/cli/Cargo.toml from the repo root.
  • Manually tested with e.g. opensrc path vercel/ai/packages/ai.

Breaking changes

None. Existing owner/repo usage is unchanged; subpath support is additive.

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 11, 2026

@ChrisLally is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

Comment on lines +312 to +313
if resolved.subpath.is_some() && repo_path.exists() {
let _ = fs::remove_dir_all(&repo_path);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if resolved.subpath.is_some() && repo_path.exists() {
let _ = fs::remove_dir_all(&repo_path);
if let Some(ref sp) = resolved.subpath {
if repo_path.exists() {
// If the requested subpath already exists on disk, reuse it instead
// of deleting the entire repo directory (which would destroy files
// checked out for other subpaths of the same repo/ref).
if repo_path.join(sp).exists() {
return FetchResult {
package: format!("{}/{}", resolved.display_name, sp),
version: resolved.git_ref.clone(),
path: format!(
"{}/{}",
get_repo_relative_path(&resolved.display_name, &resolved.git_ref),
sp
),
success: true,
error: None,
registry: None,
};
}
// Subpath not present in the existing clone — must re-clone.
// This will remove files from any previously sparse-checked-out
// subpath; callers should handle stale sources.json entries.
let _ = fs::remove_dir_all(&repo_path);
}

Fetching a different subpath of the same repo at the same ref deletes and re-clones the shared directory, destroying previously fetched subpath files while their sources.json entries remain as dangling references.

Fix on Vercel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support subdirectory fetching for GitHub repos

1 participant