Configuration files

[QwertyZW]

This is a feature request.

Having the script automatically parse some sort of configuration file (json,xml,yaml,bash) that dictates default values for its arguements would nice.

Why not just use default parameters?

Assume a case where the scripts are written for a multitude of runtimes (say for e.g; servers) and the default parameters are meant to be different for each runtime.

[kvz]

You could likely do sth like:

config.env.sh:

export LOG_LEVEL=4

program.sh:

source config.env.sh

or no?

[zbeekman]

TL;DR;
Perhaps not AS elegant as a config file, but since you can specify defaults in usage string, you can add some logic here---that instead of using a here-doc---just reads __usage in from a file. That way you can separate the list of arguments and their defaults from the script itself. You could even read in multiple files, or mix a here-doc for common options and defaults and then concatenate the __usage string variable with text from a file that has the specialized defaults/options.

We have a fork over at OpenCoarrays, where we specify arguments and their defaults in a separate file. This makes it very human readable, and I don't think it would be too hard to extend our logic for your use case. We are a number of commits behind upstream (b3bp) though, so other functionality may be missing... In general, you can just alter the script to read the arguments and their defaults from a file rather than writing them to the usage variable in the main script. Here is an example of what I am talking about:

1. L44 of prerequisites/use-case/set_magic_variables.sh: Default usage file to current top level script with -usage appended to name
2. Parse arguments specified in usage file
3. B3BP was decomposed into separate files for handling different tasks (function defs, cmd-line parsing, magic-variables, etc.). Each one is sourced from bootstrap.sh Command line arguments are passed from top level to children via "${@}", so that the arguments can be parsed. All of the decomposed B3BP scripts are in prerequisites/use-case/
4. Some scripts specify/override the default usage file that gets parsed
5. An example usage file looks like this:

  -d --debug                 Enable debug mode.
  -D --print-downloader      Print default download program.
  -e --verbose               Enable verbose mode, print script as it is executed.
  -h --help                  This page.
  -i --install-dir [arg]     Install OFP in specified path. Default="${OPENCOARRAYS_SRC_DIR}/prerequisites/installations/"
  -I --install-version [arg] OFP version to install. (To see default, use -V or --print-version)
  -j --num-threads [arg]     Number of threads to use when invoking make. Default="1"
  -n --no-color              Disable color output.
  -P --print-path            Print OFP installation path.
  -U --print-url             Print OFP URL.
  -V --print-version         Print OFP version.

[QwertyZW]

@kvz that's for environment variables, how you would use that approach for default parameter values? By the way thank you for the B3BP

@zbeekman Thank you for the detailed response! I think specifying arguements in a different file for the sake of modularity is brilliant but I'm not sure about doing that for the sake of having default arguements for different environments because now we need to worry about how we're going to version the script arguements.

It seems that the direction you're going towards with B3BP is breaking it down into pieces (eventually forming a framework) from what I understand. I've seen similar frameworks lying around that come with a lot of useful modules, really the only reason I'm using B3BP is that everything is one file, which is great for little scripts that do specific tasks that aren't meant to be reusable (or simply aren't worth the time spent to make them reusable)

[zbeekman]

In your ideal world, how would you want to specify the defaults for your use case?

…

On Thu, Jul 6, 2017 at 11:18 PM QwertyZW ***@***.***> wrote: @kvz <https://github.com/kvz> that's for environment variables, how you would use that approach for default parameter values? By the way thank you for the B3BP @zbeekman <https://github.com/zbeekman> Thank you for the detailed response! I think specifying arguements in a different file for the sake of modularity is brilliant but I'm not sure about doing that for the sake of having default arguements for different environments because now we need to worry about how we're going to version the script arguements. It seems that the direction you're going towards with B3BP is breaking it down into pieces (eventually forming a framework) from what I understand. I've seen similar frameworks lying around that come with a lot of useful modules, really the only reason I'm using B3BP is that everything is one file, which is great for little scripts that do specific tasks that aren't meant to be reusable (or simply aren't worth the time spent to make them reusable) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#90 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAREPJBeVlGl8R6YgyWSwdPVu5xYy3UWks5sLaNrgaJpZM4OHaYU> .

[QwertyZW]

@zbeekman maybe I was unclear when I said default arguments.

So let's say we have our script.sh

It takes 1 switch -s and 1 long option --long-opt=<value>

Those arguments are key to what the script does and this script will accept these arguments no matter where it goes, those arguments are also tied to the version of the script such that for example version n doesn't support --long-opt where as version m does.

Now what the default values of those arguements are are gonna be different on different environments. Every environment will take in its own value of --long-opt.

I always invoke script.sh with --long-opt=env1 on environment 1 and with --long-opt=env2 on environment 2. I'd like the next person who comes after me to not make the mistake of doing --long-opt1=env2 on environment 1. Not to say that at some point I'm not going to want to do --long-opt1=env3 on environment 1, because I might.

In my ideal world I would have script.sh.defaults on env1 and env2, both not @versioned

Such that every time someone does ./script.sh on env1 --long-opt is automatically assigned env1 if it isn't specified explicitly to be something else

[zbeekman]

@QwertyZW I think I'm following you so far... the intent of my last question, is: "In an ideal world, how would you want to specify/control the defaults on each environment? With local environment variables?With some extra script that gets sourced at the beginning of the file?"

For example, if you want to pick up the environment variable my_dir or source an additional file where my_dir is defined as the default argument for -t/--temp (and provide a default if it's missing from the script environment) you could apply the following patch:

diff --git a/main.sh b/main.sh
index 6aa8bcb..0672369 100755
--- a/main.sh
+++ b/main.sh
@@ -135,9 +135,9 @@ function help () {
 #   you can use bash variables to work around this (so use ${HOME} instead)
 
 # shellcheck disable=SC2015
-[[ "${__usage+x}" ]] || read -r -d '' __usage <<-'EOF' || true # exits non-zero when EOF encountered
+[[ "${__usage+x}" ]] || read -r -d '' __usage <<-EOF || true # exits non-zero when EOF encountered
   -f --file  [arg] Filename to process. Required.
-  -t --temp  [arg] Location of tempfile. Default="/tmp/bar"
+  -t --temp  [arg] Location of tempfile. Default="${my_dir:-/tmp/bar}"
   -v               Enable verbose mode, print script as it is executed
   -d --debug       Enables debug mode
   -h --help        This page
@@ -376,6 +376,7 @@ info "__base: ${__base}"
 info "OSTYPE: ${OSTYPE}"
 
 info "arg_f: ${arg_f}"
+info "arg_t: ${arg_t}"
 info "arg_d: ${arg_d}"
 info "arg_v: ${arg_v}"
 info "arg_h: ${arg_h}"

If you're sourcing another file, do it above where __usage is defined. Then if you export my_dir=baz/frobnicator or set that in the script you're sourcing, arg_t is set to baz/frobnicator. (and is displayed in the usage message). If you don't do that, it picks up the default I left in: /foo/bar.

If you have a way you would like to see this implemented in b3bp please let us know and we can discuss further.

EDIT: N.B.: the removal of quotes around EOF is critical to get this to work correctly. This allows variable expansion etc. in the here documented being loaded into __usage. If you keep the quotes or escape EOF then the here document is treated as a literal string with no expansion or parameter substitution.