Skip to content

feat: nupm registry subcommand #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4d9d899
init commit
mkatychev Jun 13, 2025
573100b
changed const
mkatychev Jun 13, 2025
5409a71
get_index impl
mkatychev Jun 13, 2025
3bcf1e9
removed TODOs for callout of registry subcommand
mkatychev Jun 13, 2025
87ed5d1
Merge branch 'main' into feat/registry-submodule
mkatychev Jun 20, 2025
26c3444
made nupm config declarative
mkatychev Jun 24, 2025
67c2826
Merge branch 'main' into feat/registry-submodule
mkatychev Jun 24, 2025
fce3057
updated NUPM_ variables to match nested structure
mkatychev Jun 24, 2025
354dc8e
typos
mkatychev Jun 24, 2025
2efd004
updated initial test variables
mkatychev Jun 24, 2025
5e96f1a
fixed registry typo
mkatychev Jun 24, 2025
0821712
renamed tmp to temp
mkatychev Jun 24, 2025
445be0f
revert temp
mkatychev Jun 24, 2025
1f12490
fixed tmp-dir variable resolution issue
mkatychev Jun 26, 2025
f33f29e
updated with workaround
mkatychev Jun 26, 2025
7d3a1cf
updated test cases for exitsting registry subcommands
mkatychev Jun 26, 2025
12d95c1
initial naive registry describe impl
mkatychev Jun 26, 2025
fe5b26c
registry describe passing test
mkatychev Jun 26, 2025
2907762
moved open-index to module root
mkatychev Jun 26, 2025
11a7a38
removed log calls intest
mkatychev Jun 26, 2025
b054db3
update registry header comment
mkatychev Jun 26, 2025
aa12a95
update registry init shim
mkatychev Jun 26, 2025
d5d6699
partial test pass
mkatychev Jun 27, 2025
bac9659
all tests pass
mkatychev Jun 27, 2025
baaf4f4
revert test mod closuer
mkatychev Jun 27, 2025
8542850
simplify variable setting
mkatychev Jun 28, 2025
af2a6ad
reverted nested nupm config
mkatychev Jul 7, 2025
081677b
remove flaten-nupm-env
mkatychev Jul 7, 2025
01b33b2
added registry_filename constants
mkatychev Jul 7, 2025
b2e85f2
removed fetch command
mkatychev Jul 10, 2025
4c55581
remoted editorconfig
mkatychev Jul 10, 2025
102d970
PR feedback on addtion of registry values
mkatychev Jul 16, 2025
78848fe
revert README
mkatychev Jul 16, 2025
919865b
updated formatting messages
mkatychev Jul 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/design/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Design of `nupm` :warning: Work In Progress :warning:
# Design of `nupm` :warning: Work In Progress :warning:

This file collects design ideas and directions. The intention is iterate on this document by PRs with discussion.

> **Note**
> **Note**
> in the following, until we settle down on precise names, we use the following placeholders:
> - `METADATA_FILE`: the file containing the metadata of a package,
> e.g. `project.nuon`, `metadata.json` or `nupm.nuon`
Expand All @@ -13,7 +13,7 @@ This file collects design ideas and directions. The intention is iterate on this
# Table of content
- [Project Structure](#project-structure-toc)
- [Separate virtual environments](#separate-virtual-environments-toc)
- [Installation, bootstraping](#installation-bootstraping-toc)
- [Installation, bootstrapping](#installation-bootstrapping-toc)
- [Dependency handling](#dependency-handling-toc)
- [Package repository](#package-repository-toc)
- [API / CLI Interface](#api--cli-interface-toc)
Expand All @@ -24,7 +24,7 @@ This file collects design ideas and directions. The intention is iterate on this

A `nupm` project is defined by `METADATA_FILE`.
This is where you define name of the project, version, dependencies, etc., and the type of the project.
> **Note**
> **Note**
> see [`METADATA.md`](references/METADATA.md) for a more in-depth description of
> the `METADATA_FILE`

Expand Down Expand Up @@ -73,17 +73,17 @@ The overlays could be used to achieve all three goals at the same time. When ins
* This file can be installed into a global location that's in your `NU_LIB_DIRS` (e.g., `NUPM_HOME/overlays`) -- now you have a global Python-like virtual environment
* Each overlay under `NUPM_HOME/overlays` will mimic the main NUPM_HOME structure, e.g., for an overlay `spam` there will be `NUPM_HOME/overlays/spam/bin`, `NUPM_HOME/overlays/spam/modules` (`NUPM_HOME/overlays/spam/overlays`? It might not be the best idea to have it recursive)

Each package would basically have its own overlay. This overlay file (it's just a module) could be used to also handle dependencies. If your project depends on `foo` and `bar` which both depend on `spam` but different versions, they could both import the different verions privately in their own overlay files and in your project's overlay file would be just `export use path/to/foo` and `export use path/to/bar`. This should prevent name clashing of `spam`. The only problem that needs to be figured out is how to tell `foo` to be aware of its overlay.
Each package would basically have its own overlay. This overlay file (it's just a module) could be used to also handle dependencies. If your project depends on `foo` and `bar` which both depend on `spam` but different versions, they could both import the different versions privately in their own overlay files and in your project's overlay file would be just `export use path/to/foo` and `export use path/to/bar`. This should prevent name clashing of `spam`. The only problem that needs to be figured out is how to tell `foo` to be aware of its overlay.

## Installation, bootstraping [[toc](#table-of-content)]
## Installation, bootstrapping [[toc](#table-of-content)]

Requires these actions from the user (this should be kept as minimal as possible):
* Add `NUPM_HOME/bin` to PATH (install location for binary projects)
* Add `NUPM_HOME/modules` to NU_LIB_DIRS
* Add `NUPM_HOME/overlays` to NU_LIB_DIRS
* Make the `nupm` command available somehow (e.g., `use` inside `config.nu`)

> :warning: **WIP**
> :warning: **WIP**
> The disadvantage of this is that the default install location is not an overlay. We could make `nupm` itself an overlay that adds itself as a command, so that you can activate/deactivate it. We might need a few attempts to get to the right solution.

There are several approaches:
Expand All @@ -98,7 +98,7 @@ There are several approaches:
In compiled programming languages, there are two kinds of dependencies: static and dynamic. Static are included statically and compiled when compiling the project,
dynamic are pre-compiled libraries linked to the project.

> **Note**
> **Note**
> Nushell is [similar to compiled languages][Nushell compiled] rather than typical dynamic languages like Python, so these concepts are relevant for Nushell.

Static dependencies:
Expand All @@ -120,7 +120,7 @@ as long as it has `METADATA_FILE` telling `nupm` what to do.

Nushell's module design conflates CLI interface with API -- they are the same. Not all of the below are of the same priority.

> **Note**
> **Note**
> commands like `list`, `install`, `search`, `uninstall`, `update`, ..., i.e. should
> - print short descriptions by default
> - print long descriptions with `--long-description (-l)`
Expand Down Expand Up @@ -161,7 +161,7 @@ Nushell's module design conflates CLI interface with API -- they are the same. N
- publish package to a repository
- **NOT SUPPORTED FOR NOW**: the repository will be a *GitHub* repo with packages submitted by PRs to start with

The following are for Python-style global overlays, we might need to re-think this for local package overlays:
The following are for Python-style global overlays, we might need to re-think this for local package overlays:
- `nupm overlay new`
- create a new global overlay (Python's virtual environment style)
- `--local` flag could generate an overlay locally from the currently opened project
Expand All @@ -178,7 +178,7 @@ The following are for Python-style global overlays, we might need to re-think th
### Other CLI-related notes [[toc](#table-of-content)]

* We could later think about being able to extend `nupm`, like `cargo` has plugins.
* Mutable actions (like install) have by default Y/n prompt, but can be overriden with `--yes`
* Mutable actions (like install) have by default Y/n prompt, but can be overridden with `--yes`
* By default, new projects are cross-platform:
* Windows
* MacOS
Expand Down
2 changes: 1 addition & 1 deletion docs/design/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ _See the new `registry/` directory, the following example slightly differs from

It is possible to only publish to a registry stored on your file system because we don't have a web service or anything like that.

The intented workflow for publishing a package is:
The intended workflow for publishing a package is:
1. Check out the git repository with the registry
2. `cd` into the package you want to publish
3. Run `nupm publish chosen_registry` to preview the changes
Expand Down
4 changes: 2 additions & 2 deletions nupm/install.nu
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use utils/completions.nu complete-registries
use utils/dirs.nu [ nupm-home-prompt cache-dir module-dir script-dir tmp-dir ]
use utils/dirs.nu [ nupm-home-prompt cache-dir module-dir script-dir tmp-dir PACKAGE_FILENAME ]
use utils/log.nu throw-error
use utils/misc.nu [check-cols hash-fn url]
use utils/package.nu open-package-file
Expand Down Expand Up @@ -106,7 +106,7 @@ def install-path [

do {
cd $tmp_dir
^$nu.current-exe $build_file ($pkg_dir | path join 'nupm.nuon')
^$nu.current-exe $build_file ($pkg_dir | path join $PACKAGE_FILENAME)
}

rm -rf $tmp_dir
Expand Down
18 changes: 11 additions & 7 deletions nupm/mod.nu
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,19 @@ use std/log

use utils/dirs.nu [
DEFAULT_NUPM_HOME DEFAULT_NUPM_TEMP DEFAULT_NUPM_CACHE
DEFAULT_NUPM_REGISTRIES nupm-home-prompt
DEFAULT_NUPM_REGISTRIES DEFAULT_NUPM_INDEX_PATH nupm-home-prompt
]

use utils/registry.nu open-index

export module install.nu
export module publish.nu
export module registry.nu
export module search.nu
export module status.nu
export module test.nu


export-env {
# Ensure that $env.NUPM_HOME is always set when running nupm. Any missing
# $env.NUPM_HOME during nupm execution is a bug.
Expand All @@ -22,12 +26,12 @@ export-env {
# Ensure install cache is set
$env.NUPM_CACHE = ($env.NUPM_CACHE? | default $DEFAULT_NUPM_CACHE)

# TODO: Maybe this is not the best way to set registries, but should be
# good enough for now.
# TODO: Add `nupm registry` for showing info about registries
# TODO: Add `nupm registry add/remove` to add/remove registry from the env?
$env.NUPM_REGISTRIES = ($env.NUPM_REGISTRIES?
| default $DEFAULT_NUPM_REGISTRIES)
# check for the index path
$env.NUPM_INDEX_PATH = ($env.NUPM_INDEX_PATH? | default $DEFAULT_NUPM_INDEX_PATH)

# read from registry index but don't overwrite registires already present in $env.NUPM_REGISTRIES
$env.NUPM_REGISTRIES = ($env.NUPM_INDEX_PATH | open-index
| merge ($env.NUPM_REGISTRIES? | default $DEFAULT_NUPM_REGISTRIES))

use std/log []
}
Expand Down
191 changes: 191 additions & 0 deletions nupm/registry.nu
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# Registry management for nupm

use utils/dirs.nu [nupm-home-prompt REGISTRY_FILENAME]
use utils/log.nu [throw-error append-help UNIMPLEMENTED]
use utils/registry.nu registry-cache

# Manage nupm registires
@example "List all configured registries" { nupm registry }
export def main []: nothing -> table {
list
}

# List all configured registries
@example "List all registries with details" { nupm registry list }
export def list []: nothing -> table {
$env.NUPM_REGISTRIES | transpose name url | sort-by name
}

# Formatting convenience function that is passed in the presence of the `--save` flag, informing the user on
# how to make their changes permanent
export def success-msg [success_msg: string, write_path: path, saved_to_disk: bool]: nothing -> string {
if $saved_to_disk {
return $"($success_msg) and written to ($write_path)"
}

$success_msg | append-help "To commit the change to disk, use the `--save` flag."
}


def registry-names [] { list | get name }
# Show detailed information about a specific registry
# returning a list of package names, type, and version
@example "Show registry information" { nupm registry describe nupm }
export def describe [
registry: string@registry-names
]: nothing -> table {
use utils/dirs.nu cache-dir

if not ($registry in $env.NUPM_REGISTRIES) {
throw-error $"Registry '($registry)' not found"
}

let registry_url = $env.NUPM_REGISTRIES | get $registry
let cache = registry-cache $registry
let cached_registry = $cache.dir | path join $REGISTRY_FILENAME

# Always check cache first, only fall back to URL if cache doesn't exist
let registry_data = if ($cache.file | path exists) {
open $cache.file
} else if ($registry_url | path exists) {
# Local registry file
open $registry_url
} else {
# Remote registry - fetch and cache
let data = http get $registry_url
mkdir $cache.dir
$data | save $cache.file
$data
}

$registry_data | each {|entry|
let package_cache_path = $cache.dir | path join $"($entry.name).nuon"

# Always check cache first for package data too
let package_file_data = if ($package_cache_path | path exists) {
open $package_cache_path
} else if ($registry_url | path exists) {
# Local package file
let package_path = $registry_url | path dirname | path join $entry.path
open $package_path
} else {
# Remote package - fetch and cache
let base_url = $registry_url | url parse
let package_url = $base_url | update path ($base_url.path | path dirname | path join $entry.path) | url join
let data = http get $package_url
$data | save $package_cache_path
$data
}

# Package data is a table of versions for this package
$package_file_data | each {|pkg|
{
name: $pkg.name,
# TODO rename package metadata type field to source
# to avoid confusion with custom|script|module type enumberable
source: $pkg.type,
version: $pkg.version,
# description: ($pkg.description? | default "")
}
}
} | flatten
}

# Add a new registry
@example "Add a new registry" { nupm registry add my-registry https://example.com/registry.nuon }
export def --env add [
name: string, # Name of the registry
url: string, # URL or path to the registry
--save, # Whether to commit the change to the registry index
] {
if ($name in $env.NUPM_REGISTRIES) {
throw-error $"Registry '($name)' already exists. Use 'nupm registry update' to modify it."
}
$env.NUPM_REGISTRIES = $env.NUPM_REGISTRIES | insert $name $url

if $save {
$env.NUPM_REGISTRIES | save --force $env.NUPM_INDEX_PATH
}

print (success-msg $"Registry '($name)' added successfully" $env.NUPM_INDEX_PATH $save)

}

# Remove a registry
@example "Remove a registry" { nupm registry remove my-registry }
export def --env remove [
name: string # Name of the registry to remove
--save, # Whether to commit the change to the registry index
] {
$env.NUPM_REGISTRIES = $env.NUPM_REGISTRIES | reject $name

if $save {
$env.NUPM_REGISTRIES | save --force $env.NUPM_INDEX_PATH
}

print (success-msg $"Registry '($name)' removed successfully" $env.NUPM_INDEX_PATH $save)
}

# Update a given registry url
@example "Update registry URL" { nupm registry set-url my-registry https://new-url.com/registry.nuon }
export def --env set-url [
name: string, # Name of the registry to update
url: string, # Registry target URL
--save, # Whether to commit the change to the registry index
]: nothing -> nothing {
$env.NUPM_REGISTRIES = $env.NUPM_REGISTRIES | update $name $url

if $save {
$env.NUPM_REGISTRIES | save --force $env.NUPM_INDEX_PATH
}

print (success-msg $"Registry '($name)' URL updated successfully" $env.NUPM_INDEX_PATH $save)
}

# https://www.nushell.sh/book/configuration.html#macos-keeping-usr-bin-open-as-open
alias nu-rename = rename
# Rename a registry
@example "Rename a registry" { nupm registry rename my-registry our-registry }
export def --env rename [
name: string, # Name of the registry to update
new_name: string, # New registry name
--save, # Whether to commit the change to the registry index
] {
$env.NUPM_REGISTRIES = $env.NUPM_REGISTRIES | nu-rename --column { $name: $new_name }

if $save {
$env.NUPM_REGISTRIES | save --force $env.NUPM_INDEX_PATH
}

print (success-msg $"Registry '($name)' renamed successfully" $env.NUPM_INDEX_PATH $save)
}

def init-index [] {
if not (nupm-home-prompt) {
throw-error "Cannot create NUPM_HOME directory."
}


if ($env.NUPM_INDEX_PATH | path exists) {
print $"Registry list already exists at ($env.NUPM_INDEX_PATH)"
return
}

$env.NUPM_REGISTRIES | save $env.NUPM_INDEX_PATH

print $"Registry index initialized at ($env.NUPM_INDEX_PATH)"
}


# Initialize a new nupm registry or a registry index if the `--index` flag is
# passed in
@example "Initialize registry index" { nupm registry init --index }
export def init [--index] {
if $index {
init-index
return
}
# TODO initialize registry index here
throw-error UNIMPLEMENTED "`nupm registry --index` is not implemented."
}

4 changes: 2 additions & 2 deletions nupm/test.nu
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use utils/dirs.nu [ tmp-dir find-root ]
use utils/dirs.nu [ tmp-dir find-root PACKAGE_FILENAME ]
use utils/log.nu throw-error

# Run tests for a nupm package
Expand Down Expand Up @@ -28,7 +28,7 @@ export def main [

if $pkg_root == null {
throw-error "package_file_not_found" (
$'Could not find "nupm.nuon" in ($dir) or any parent directory.'
$'Could not find "($PACKAGE_FILENAME)" in ($dir) or any parent directory.'
)
}

Expand Down
Loading
Loading