Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test to [[ where available #261

Merged
merged 109 commits into from
Jan 29, 2025
Merged

test to [[ where available #261

merged 109 commits into from
Jan 29, 2025

Conversation

balupton
Copy link
Member

@balupton balupton commented Oct 28, 2024

/close #265 #268

Using [[ ... ]] instead of test is a convention change that improves performance and resolves edge cases where test -n "$a" -a "$b" = "$c" fails when a='>'.

This PR also features these highlights:

  • clarified help text around return statuses, and help text will now mention companion commands
  • ask: fixed clearing issue /close Bug: ask prompt is not clearing correctly, causing ask prompt and answer to duplicate and cursor to move in-between data #265
  • reduce sudo escalations by introduction of auto-escalating is-* filesystem commands for filesystem interactions which use a new is-accessible command to determine if a sudo escalation is needed, benefiting Improvement: smooth installation on non-root systems #233 and closes Suggestion: instead of requiring sudo for is-present and is-missing failures, test if assumptions are correct #268, full list of commands:
    • is-accessible
    • is-broken-symlink
    • is-directory
    • is-empty-directory
    • is-empty-file
    • is-executable
    • is-file
    • is-missing
    • is-nonempty-file
    • is-not-directory
    • is-not-symlink
    • is-owner
    • is-present
    • is-readable
    • is-symlink
    • is-writable
    • echo-if-directory
    • echo-if-executable
    • echo-if-file
    • echo-if-present
    • fs-absolute
    • fs-own
    • fs-dequarantine
    • fs-parents
    • fs-realpath
    • fs-structure
    • gocryptfs-helper
    • secret
  • fs-rm: completely rewritten for a superior experience, with revised functionality to simplify and secure its human and programmatic usage (such as in setup-dns), also supports --trash
  • fs-structure: rewritten, and now used in fs-own, fs-rm, fs-trim, gocryptfs-helper, is-same, setup-util, eject-all
  • added new fs-trim to assist with fs-rm under certain circumstances
  • choose:
    • properly support cancelling to no selection when not required
    • support --truncate-body for when we allow the question's body/description/details to be truncated to make way for items, used by fs-rm and fs-trim, defaults to disabled
    • support --default-all for defaulting to selection of everything, defaults to disabled
    • support --confirm-cancel which now defaults to enabled
    • use styles.bash values for terminal title and resize modifications
    • fix typos in the legend
    • add support for the r key for reversion, along with the prior z key
  • dorothy:
    • fix dnf ecosystem changes breaking installation on dnf setups
    • install action: cleaner removal .profile modification if no bash, dash, ksh
    • lint and test actions now sort files by most recent first, and test action will call the new git-helper umt command and action on CI to ensure clone dates are correct
    • new commands action: lists all the available commands, recommended now in post-install instructions, uses new echo-if-present command, however doesn't yet use fs-structure
    • todos action: support checking core and beta commands, correct detection of usage, correct error status reporting, clearer indication when help text is missing, fix missing and incorrect local vars
    • use is-ci instead of an archaic technique
    • permissions action: no longer trim junk files, as they are already git-ignored
  • added a new sources/tests.bash for assisting the fs commands with their tests
  • added a new is-owner command for detecting if a user or group has ownership of a path
  • added a new is-root command for detecting if a user is a root user (maybe one day this should support the root group as well)
  • read-key: added a --continue option for debugging, but most importantly reduced the subsequent timeout from 0.0001 to 0.01 to prevent situations where a key was read only partially and caused incorrect keys to be assumed, causing say ask, choose, confirm to interpret an escape key that wasn't actually pressed
  • setup-util:
    • load styles.bash instead of calling echo-style for performance
    • use the new fs-rm flow for removing leftover and conflicting paths
    • rm_helper_confirm to rm_helper_which to better explain its purpose
    • echo_fallback to log_fallback
    • improve styling of APT_REPO http vs https warning
    • use the new fs-structure for outputting when a download build glob fails
    • ensure all output goes not to stdout but to the terminal device file, such that $(... something that calls setup-util ...)`` does not get a bunch of install junk in the result, as was the case for tests.bashcalling ofsymlink-helper` which installed coreutils for realpath
      • this was mostly achieved by adding the missing outputpipe="$terminal_device_file" to when use_alt_screen_buffer was no
    • also ensure errors go to stderr where appropriate
  • sudo-helper: add alternative values for --sudo option, to enable optional escalation under various circumstances, and simplified logic
  • symlink-helper: support --target= as alias for --existing=, and improve --quiet and --verbose handling to ensure outputs match expectations, and with nothing going to stdout

Other changes to the core:

  • everywhere:
    • remove unnecessary --status=0 option with eval-tester
    • source code is now spell checked
    • use =~ ^(...|...)$ instead of multiple == ... || statements for easier human parsing
  • ci:
    • test on ubuntu arm runners
    • compat with newer dnf versions
    • upgrade lint tooling
  • init.sh: fix a missing -d
  • config-edit: --editer changed to correct spelling of --editor
  • command-working: add rmtrash to the exception
  • down:
    • added --progress option that is enabled by default, except on CI where it is disabled by default which prevents progress bars from appearing on CI
    • fix an issue where the temporary download directory might not exist
  • edit: don't output None of the configured editors were available if there are no configured editors, such as the case on a new machine (review this later, as we may still want a warning, but a better warning on more specific certain situations)
  • echo-style: fixed empty --= and -- usage, the implementation seemed broken from the start
  • eval-helper:
    • categorise sudo-helper automatically as a shapeshifter, as it is impossible to know how many lines it used while attempting passwords, so impossible to clear it accurately in non-shapeshifter mode
    • all messages now correctly go to the terminal device file, such that stdout captures are not corrupted with messages
    • if using echo-revolving-door don't output anything to stdout, as anything would not be correct as it would be truncated
  • flush-dns: use --shapeshifter on the sudo-helper calls for interacting with dns responders to ensure it is all rendered correctly
  • fs-dequarantine: use /usr/bin/xattr as homebrew can cause conflicts with xattr
  • fs-filename: fix help convention
  • fs-own:
    • accept --recursive with variant spelling
    • add --root for setting ownership as the platform's root user and group
    • add support for --verbose on macos
  • fs-temp: clarified --touch behaviour, and support --no-touch on directories
  • get-flag-value: implement --fallback-on-empty option used by fs-absolute
  • get-json-api-url.awk: fix typo in var name
  • get-profile: correct spelling of possesive-pronoun to possessive-pronoun
  • get-terminal-color-support: clarified help text on return statuses in different modes
  • get-terminal-theme: use is-integer sanity check for colorfgbg source
  • git-helper: add new umt action to apply commit modification times to cloned git repos, used by dorothy test on CI
  • github-download: fix a missing -n on $pathname exclusivity check
  • is-empty-value: added tests and made performant
  • is-admin: fix multiple user handling
  • is-even: add tests, and only support integers
  • is-generic: added empty string as generic, added more tests
  • is-missing: fix incorrect help text on exit status
  • is-needle: added more tests, added --all and --any option modifiers
  • is-odd: add tests, and only support integers
  • mount-helper: fix two missing -n
    • samba username and password check: if test -n "$option_username" -o "$option_password"; then would have caused an error state if only password is defined
    • if test -n "$temp" -a "$option_label"; then changed to just if [[ -n $temp ]]; then as in this case, it seems better to not exist, as we always want the first line of $temp regardless of $option_label (however, then it is more a sanity procedure, but a necessary one)
    • fixed detection of forced unmount
  • researchgate-rename: fix bad is-same call
  • setup-dns: use new fs-rm flow for removing unwanted configuration and installation files, and fix potential regression issue when removals were already removed
  • setup-git: fix a missing test / [[
    • if test "$GPG_SIGNING_AGENT" != 'op' && "$GPG_SIGNING_AGENT" != 'krypton'; then affected GPG_SIGNING_AGENT when not using 1password
  • setup-mac-brew: don't enforce sudo for the cleanup fs-rm, as fs-rm is now intelligent enough to handle escalation itself if needed
  • add setup-util-asciinema
  • setup-util-eza: use https for APT_REPO
  • add setup-util-firefox
  • add setup-util-qpdf used by pdf-decrypt
  • add setup-util-tor-browser
  • setup-util-trash: updated for a library that actually works, and uninstall the ones that don't
  • setup-util-xcode: use shapeshifter for the xcode license agreement, as it requires it as it modifies cursor
  • waiter: fix missing -n on --stdout, --stderr, and --tty option handling
  • config/dns.bash: removed outdated configuration options
  • config/styles.bash: fixed terminal resize and end styles not being found due to typo
  • docs/errors.md: detailed the errors that dorothy actually uses
  • sources/nvm.sh: fix for edge case where nvm fails to detect system node

deprecated core commands:

  • echo-segment replaced by echo-style, this was done a year or more ago, before we had the deprecated directory
  • fs-size replaced by fs-structure or get-size depending on your use case
  • get-array-* and is-array-* commands replaced by echo-trim-empty-lines yes that is correct and yes it is weird, but those commands were weird
  • is-array-(empty|full|partial) replaced by is-whitespace or is-not-whitespace
  • is-color-enabled replaced by get-terminal-color-support --quiet
  • is-empty-ls replaced by is-empty-directory
  • is-empty-string replaced by is-whitespace
  • is-nonempty-string replaced by is-not-whitespace
  • get-line-count replaced by echo-count-lines
  • rm-junk replaced by fs-trim --cache
  • rm-modules replaced by fs-trim --module

deprecated beta commands:

  • echo-if-empty replaced by echo-with-whitespace-fallback
  • echo-if-path replaced by echo-if-present
  • is-dir replaced by is-directory
  • is-either replaced by is-needle
  • is-equal replaced by is-needle
  • is-exec replaced by is-executable
  • is-neither replaced by ! is-needle
  • rm-svn replaced by find ...
  • rm-sync replaced by find ...

promoted from beta commands:

  • echo-if-path promoted to echo-if-present
  • echo-last-line
  • echo-values
  • is-empty-directory
  • is-even
  • is-file

demoted to beta commands:

  • echo-affirmative
  • find-directories

beta command updates:

  • add echo-if-directory
  • add echo-if-executable
  • add echo-truncate-lines, however you probably just want head -n <lines>
  • add echo-with-empty-fallback, probably needs to be renamed to echo-with-zero-length-fallback
  • add echo-with-whitespace-fallback, renamed from echo-if-empty
  • eject-all: use fs-structure
  • eval-on-empty-stdin: rewrote for modern conventions, and support grep fallback for when ifne is not present
  • eval-on-not-empty-stdin: rewrote for modern conventions
  • get-codec: fix tests
  • is-suffix: use new bash.bash helpers
  • pdf-decrypt: rewrote for modern conventions
  • researchgate-rename: use echo-regexp instead of python-regex, and fix is-same call
  • rm-vmware renamed to rm-vmware-fusion
  • setup-server: updated for new is-* fs commands

merges:

fixes:

- init.sh: fix a missing `-d`
- github-download: fix a missing `-n`
- mount-helper: fix a missing `-n`
- setup-git: fix a missing `test`/`[[`
- waiter: fix missing `-n`

todos:

- note this change of convention in the docs
- see if there is an alternative for the sudo calls
- review the PR to check if there were any errors
@balupton balupton marked this pull request as draft October 28, 2024 12:03
@balupton balupton self-assigned this Oct 28, 2024
- add `is-(directory|file|missing|not-directory|present|readable)(.bash)?`
- is-missing: fix incorrect help text on exit status
- mount-helper: fix missing `-n`, however seems better to just always fetch a single line here
- debug-network: fix WIP commit using `[[` in a sh script, instead of `[`
- `while true` to `while :`, and `true` to `:` where appropriate
- brew: compact a conditional
- (choose|config-helper|confirm|echo-lines|is-empty-value|setup-(git|linux|mac|system)): compact an or conditional
- is-suffix: update for `__substr` shim in `bash.bash` for bash v3 support
- `is-dir`: move to deprecated, as there is now `is-directory`
commands/get-terminal-theme Outdated Show resolved Hide resolved
todos:

- choose: needs truncating of choose body, if too long, possible with `fs-rm` and `fs-trim` with large directory content listing: `eza -lA | sed -e '5,$d' -e '4,$s/.*/Truncated./'` is suitable, however our own command would work
- eval-helper: check if pending/success/failure messages are still going to TTY
- is-not-symlink: update copy and pasted tests to match its functionality
- consider renames of `echo-exit-affirmative`, `echo-exit-status`, `is-root`

changes:

- choose:
    - optional will now cancel to nothing, only required cancels to defaults
    - can now press R to revert to defaults
- dorothy:
    - permissions action no longer trims junk files, as they are already git-ignored
- `fs-realpath`, `fs-structure`, `fs-trim`, `is-broken-symlink`, `is-directory`, `is-empty-directory`, `is-empty-file`, `is-file`, `is-missing`, `is-not-directory`, `is-not-symlink`, `is-present`, `is-readable`, `is-symlink`, `is-nonempty-file`:
    - support `--(sudo|user|group)` arguments
    - consistent internal structure
    - add tests where appropriate
- `fs-rm`, `fs-trim`, `fs-structure`:
    - rewrote `fs-rm` for new `fs-trim` command, with exact prompts on what to do, no longer overly complex and fragile
    - rewrote `fs-structure` to prefer `exa`, comment out verbose `dust`/`du` handling as `exa` is better and more understandable for this use case, made perms and time toggles
- fs-temp:
    - clarified touch functionality (will make directories, but not files, by default), make `--no-touch` not make directories
- add `fs-trim`:
    - consolidated functionality from: `fs-rm`, `rm-junk`, `rm-modules`
    - clear handling of symlinks
- add `is-broken-symlink`, `is-empty-directory`, `is-empty-file`, `is-not-symlink`, `is-file`, `is-not-symlink`, `is-symlink`:
- is-generic:
    - change return status behaviour on multiple arguments to be consistent with others
- promoted `is-even` from beta commands
- add `setup-util-qpdf` for `pdf-decrypt`, and updated `pdf-decrypt` for modern covnentions
- sudo-helper:
    - add `--[no-]sudo` flag, which if no and no user nor group, then no sudo is needed

fixes:

- choose: pressing escape/q to cancel now works again
- echo-values: fix help text being incorrect
- eval-helper: wrapping will now have wraps go to stdout
- get-terminal-theme: more accurate parsing of `COLORFGBG`
- is-admin: fix multiple users handling
- setup-dns: improve handling of bad interfaces fetch

renames and deprecations:

- `is-nonempty-string` to `is-not-whitespace`
- `is-empty-string` to `is-whitespace`
- `is-empty-ls` to `is-empty-directory`, turns out we had two commands doing the same thing
- `is-dir` to `is-directory`
- `is-either` to `is-needle`, `is-neither` to `! is-needle`
- `echo-if-empty` to `echo-with-whitespace-fallback`, added `echo-with-empty-fallback` for zero-length
- `echo-values` promoted from beta, used within `setup-dns`
- `fs-size` deprecated as it was the same as `fs-structure`; use `fs-structure` or `get-size` instead
- `rm-vmware` to `rm-vmware-fusion`
- `rm-junk` and `rm-modules` to `fs-trim --junk`
- deprecate `is-array-*` and `get-array-*` commands
- deprecate `is-exec`, `is-equal`, `rm-svn`, and `rm-sync` until there is a use case for them

linting:

- down: as `fs-temp` touch usage is now clarified, improve logic around mkdirp call
- fs-filename: use the new options help text format
- get-terminal-color-support: fix return status help text
- remove unnecessary `--status=0` in `eval-tester` calls
- improve wording of return statuses in help text
- improve wording around whether symlinks are considered or not in various commands
- added `Companion to ...` line to improve discovery of (related) commands and their functionality
@balupton balupton force-pushed the dev/square-brackets branch from 2538a59 to ccda6c8 Compare November 16, 2024 04:27
- choose: add `--truncate-body` WIP
- config-edit: `--editer=` to `--editor=`
- add `echo-truncate-lines` WIP
- eval-helper: shapeshifter will be enabled on `sudo-helper` by default
- flush-dns, setup-util-docker, setup-util-xcode: fix formatting when sudo needed
- fs-rm: rewrote for new `fs-trim` command
- get-profile: `possesive-pronoun` to `possesive-pronoun`
- git-helper: fix bad formatting on failures
- is-present: speedier sudo checks, by first checking without sudo
- sudo-helper: support `--optional` for speedier checks without sudo
- mount-helper: fixed detection of forced unmount
- add `setup-util-tor-browser`
- styles: fix terminal title and resize not terminating
@balupton balupton force-pushed the dev/square-brackets branch 3 times, most recently from 9f7e1e8 to 808f55c Compare November 27, 2024 10:18
@balupton balupton force-pushed the dev/square-brackets branch from 808f55c to b96a30f Compare November 27, 2024 10:25
@balupton balupton force-pushed the dev/square-brackets branch from 5cc1ae7 to 839a82c Compare November 27, 2024 12:11
@balupton balupton requested a review from Copilot November 27, 2024 12:11

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 559 out of 576 changed files in this pull request and generated no suggestions.

Files not reviewed (17)
  • .vscode/workspace.code-workspace: Language not supported
  • commands.beta/convert-helper: Language not supported
  • commands.beta/echo-affirmative: Language not supported
  • commands.beta/echo-escape-special: Language not supported
  • commands.beta/echo-exit-affirmative: Language not supported
  • commands.beta/echo-exit-status: Language not supported
  • commands.beta/echo-html-coder.recode: Language not supported
  • commands.beta/echo-html-coder.textutil: Language not supported
  • commands.beta/echo-html-coder.xmlstarlet: Language not supported
  • commands.beta/echo-if-directory: Language not supported
  • commands.beta/echo-if-executable: Language not supported
  • commands.beta/echo-if-path: Language not supported
  • commands.beta/echo-last-line: Language not supported
  • commands.beta/echo-mkdir: Language not supported
  • commands.beta/echo-non-affirmative: Language not supported
  • commands.beta/echo-nonflags: Language not supported
  • commands.beta/echo-nothing-or-fail: Language not supported
- add `echo-if-directory`, `echo-if-executable`, `echo-if-present`, `is-executable`, `is-writable`
- `echo-if-file`, `fs-dequarantine` now consistent with other such helpers
- fs-trim: fix bad filename code handling, use new `is-needle` flags
- is-needle: add `--ignore-case`, `--all`, `--any` flags, and enforce expected arg usage
- setup-dns: simplify `fs-rm` call as the extra stuff is no longer necessary
- setup-util-bash: simplify `is-needle` call
- echo-truncate-lines: move to beta, as no active users
- setup-server: use fs helpers wherever appropriate
- deprecate `echo-if-path` for `echo-if-present`
@balupton balupton force-pushed the dev/square-brackets branch 3 times, most recently from 2e4f695 to 72339a7 Compare November 29, 2024 10:07
@balupton balupton force-pushed the dev/square-brackets branch 2 times, most recently from dffa6fc to dcb21f4 Compare November 29, 2024 11:33
@balupton balupton force-pushed the dev/square-brackets branch from dcb21f4 to 13c335f Compare November 29, 2024 12:15
balupton and others added 18 commits January 23, 2025 11:49
…or commands, so that `dorothy todos` can detect usage correctly
…` is now intelligent enough to handle escalation itself if needed
```
> nvm-env -- npm
++ '[' -z /Users/balupton/.nvm ']'
++ '[' -s /Users/balupton/.nvm/nvm.sh ']'
++ . /Users/balupton/.nvm/nvm.sh
+++ NVM_SCRIPT_SOURCE=']'
+++ '[' -z '' ']'
+++ export NVM_CD_FLAGS=
+++ NVM_CD_FLAGS=
+++ nvm_is_zsh
+++ '[' -n '' ']'
+++ '[' -z /Users/balupton/.nvm ']'
+++ case $NVM_DIR in
+++ unset NVM_SCRIPT_SOURCE
+++ nvm_process_parameters
+++ local NVM_AUTO_MODE
+++ NVM_AUTO_MODE=use
+++ '[' 0 -ne 0 ']'
+++ nvm_auto use
+++ local NVM_MODE
+++ NVM_MODE=use
+++ case "${NVM_MODE}" in
+++ local VERSION
+++ local NVM_CURRENT
++++ nvm_ls_current
++++ local NVM_LS_CURRENT_NODE_PATH
+++++ command which node
++++ NVM_LS_CURRENT_NODE_PATH=
++++ nvm_echo none
++++ command printf '%s\n' none
+++ NVM_CURRENT=none
+++ '[' _none = _none ']'
++++ nvm_resolve_local_alias default
++++ nvm_echo
++++ command printf '%s\n' ''
+++ VERSION=N/A
+++ '[' -n N/A ']'
+++ '[' _N/A '!=' _N/A ']'
+++ return 0
+ npm
env: node: No such file or directory
```
@balupton balupton requested a review from Copilot January 28, 2025 17:31
@balupton balupton marked this pull request as ready for review January 28, 2025 17:31
@balupton balupton requested a review from molleweide January 28, 2025 17:35
Copy link
Collaborator

@molleweide molleweide left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testing.

conventions.md seemed to show as draft PR even though we had finished the PR.

@balupton
Copy link
Member Author

@balupton balupton merged commit cf68611 into master Jan 29, 2025
38 of 41 checks passed
@balupton balupton deleted the dev/square-brackets branch January 29, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment