more work on #299; get ready to release RC2.

adobeDan · adobeDan · commit e2e99558c151 · 2017-12-03T16:55:57.000-08:00
- Update the docs for #299. - Move the `invocation_defaults` section to the bottom of the sample config. - Update version to 2.2rc2 - Update release notes.
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
@@ -1,6 +1,6 @@
 # Release Notes for User Sync Tool Version 2.3
 
-These notes apply to v2.3rc2 of 2017-11-21.
+These notes apply to v2.3rc2 of 2017-12-03.
 
 ## New Features
 
@@ -10,6 +10,7 @@ There is a new command-line argument `--connector` for specifying whether to get
 
 [#292](https://github.com/adobe-apiplatform/user-sync.py/issues/292) You can now specify the log file name as well as the log file directory in your configuration file.  The name is specified by giving a Python format string which, when applied to a Python `datetime` value at the start of the run, produces the name of the log file.  The default value of this string is backwards-compatible with prior User Sync behavior.  See [the docs](https://adobe-apiplatform.github.io/user-sync.py/en/user-manual/configuring_user_sync_tool.html#configure-logging) for details.
 
+[#299](https://github.com/adobe-apiplatform/user-sync.py/issues/299) You can now use an `invocation_defaults` section to specify desired values for command-line arguments in the main configuration file.  This can make it a lot easier to repeat runs with a stable set of arguments, even when running interactively rather than from a script.  The sample main configuration file specifies the configuration parameters to use as well as the syntax for specifying values.  See [the docs](https://adobe-apiplatform.github.io/user-sync.py/en/user-manual/command_parameters.html) for full details.
 
 ## Bug Fixes
 
diff --git a/docs/en/user-manual/command_parameters.md b/docs/en/user-manual/command_parameters.md
@@ -29,19 +29,45 @@ specific behavior in various situations.
 |------------------------------|------------------|
 | `-h`<br />`--help` | Show a help message and exit.  |
 | `-v`<br />`--version` | Show program's version number and exit.  |
-| `-t`<br />`--test-mode` | Run API action calls in test mode (does not execute changes). Logs what would have been executed.  |
+| `-t`<br />`--test-mode`<br />`-T`<br />`--no-test-mode` | Specifying `-t` or `--test-mode` causes User Sync to run API action calls in _test mode_ (that is, the server does syntax/semantics checking on the calls but does not execute any requested changes). User Sync still logs all actions, making this very useful for previewing what a run would have requested. Starting with version 2.3, specifying `-T` or `--no-test-mode` can be specified to turn off test mode. |
 | `-c` _filename_<br />`--config-filename` _filename_ | The complete path to the main configuration file, absolute or relative to the working folder. Default filename is "user-sync-config.yml" |
 | `--users` `all`<br />`--users` `file` _input_path_<br />`--users` `group` _grp1,grp2_<br />`--users` `mapped` | Specify the users to be selected for sync. The default is `all` meaning all users found in the directory. Specifying `file` means to take input user specifications from the CSV file named by the argument. Specifying `group` interprets the argument as a comma-separated list of groups in the enterprise directory, and only users in those groups are selected. Specifying `mapped` is the same as specifying `group` with all groups listed in the group mapping in the configuration file. This is a very common case where just the users in mapped groups are to be synced.|
 | `--user-filter` _regex\_pattern_ | Limit the set of users that are examined for syncing to those matching a pattern specified with a regular expression. See the [Python regular expression documentation](https://docs.python.org/2/library/re.html) or [for Python 3](https://docs.python.org/3/library/re.html) for information on constructing regular expressions in Python. The user name must completely match the regular expression.|
-| `--update-user-info` | When supplied, synchronizes user information. If the information differs between the enterprise directory side and the Adobe side, the Adobe side is updated to match. This includes the firstname and lastname fields. |
-| `--process-groups` | When supplied, synchronizes group membership information. If the membership in mapped groups differs between the enterprise directory side and the Adobe side, the group membership is updated on the Adobe side to match. This includes removal of group membership for Adobe users not listed in the directory side (unless the `--adobe-only-user-action exclude` option is also selected).|
+| `--update-user-info`<br />`--no-update-user-info` | Specifying `--update-user-info` synchronizes user information. If the information differs between the enterprise directory side and the Adobe side, the Adobe side is updated to match. This includes the firstname and lastname fields.  Starting with User Sync 2.3, `--no-update-user-info` can be specified to prevent this synchronization. |
+| `--process-groups`<br />`--no-process-groups` | Specifying `--process-groups` causes synchronization of group membership information: if the membership in mapped groups differs between the enterprise directory side and the Adobe side, the group membership is updated on the Adobe side to match. This includes removal of group membership for Adobe users not listed in the directory side (unless the `--adobe-only-user-action exclude` option is also selected).  Starting with User Sync 2.3, `--no-process-groups` can be specified to prevent this synchronization. |
 | `--adobe-only-user-action preserve`<br />`--adobe-only-user-action remove-adobe-groups`<br />`--adobe-only-user-action  remove`<br />`--adobe-only-user-action delete`<br /><br/>`--adobe-only-user-action  write-file`&nbsp;filename<br/><br/>`--adobe-only-user-action  exclude` | When supplied, if user accounts are found on the Adobe side that are not in the directory, take the indicated action.  <br/><br/>`preserve`: no action concerning account deletion is taken. This is the default.  There may still be group membership changes if the `--process-groups` option was specified.<br/><br/>`remove-adobe-groups`: The account is removed from user groups and product configurations, freeing any licenses it held, but is left as an active account in the organization.<br><br/>`remove`: In addition to remove-adobe-groups, the account is also removed from the organization, but the user account, with its associated assets, is left in the domain and can be re-added to the organization if desired.<br/><br/>`delete`: In addition to the action for remove, the account is deleted if its domain is owned by the organization.<br/><br/>`write-file`: No action concerning account deletion is taken. The list of user accounts present on the Adobe side but not in the directory is written to the file indicated.  You can then pass this file to the `--adobe-only-user-list` argument in a subsequent run.  There may still be group membership changes if the `--process-groups` option was specified.<br/><br/>`exclude`: No update of any kind is applied to users found only on the Adobe side.  This is used when doing updates of specific users via a file (`--users file f`) where only users needing explicit updates are listed in the file and all other users should be left alone.<br/><br>Only permitted actions will be applied.  Accounts of type adobeID are owned by the user so the delete action will do the equivalent of remove.  The same is true of Adobe accounts owned by other organizations. |
 | `--adobe-only-user-list` _filename_ | Specifies a file from which a list of users will be read.  This list is used as the definitive list of "Adobe only" user accounts to be acted upon.  One of the `--adobe-only-user-action` directives must also be specified and its action will be applied to user accounts in the list.  The `--users` option is disallowed if this option is present: only account removal actions can be processed.  |
 | `--config-file-encoding` _encoding_name_ | Optional.  Specifies the character encoding for the contents of the configuration files themselves.  This includes the main configuration file, "user-sync-config.yml" as well as other configuration files it may reference.  Default is `utf8` for User Sync 2.2 and later and `ascii` for earlier versions.<br />Character encoding in the user source data (whether csv or ldap) is declared by the connector configurations, and that encoding can be different than the encoding used for the configuration files (e.g., you could have a latin-1 configuration file but a CSV source file that uses utf-8 encoding).<br />The available encodings are dependent on the Python version used; see the documentation [here for Python 2.7](https://docs.python.org/2.7/library/codecs.html#standard-encodings) or [here for Python 3.6](https://docs.python.org/3.6/library/codecs.html#standard-encodings) for more information.  |
 | `--strategy sync`<br />`--strategy push` | Available in release 2.2 and later. Optional.  Default operating mode is `--strategy sync`.   Controls whether User Sync reads user information from Adobe and compares to the directory information and then issues updates to Adobe, or simply pushes the directory input to Adobe without considering the existing user information on Adobe.  `sync` is the default and the subject of the description of most of this documentation.  `push` is useful when there is a large number of users on the Adobe side (>30,000) and known additions or changes to a small number of users are desired, and the list of those users is available in a csv file or a specific directory group.<br />If `--strategy push` is specified, `--adobe-only-user-action` cannot be specified as the determination of adobe-only users is not made.<br/>`--strategy push` will create new users, modify their group memberships for mapped groups only (if `--process-groups` is present),  update user information (if `--update-user-info` is present), and will not remove users from the organization or delete their accounts.  See [Handling Push Notifications](usage_scenarios.md#handling-push-notifications) for information on how to remove users via push notifications. |
 | `--connector ldap`<br />`--connector okta`<br />`--connector csv` _filename_ | Available in release 2.3 and later. Optional. Specifies the directory connector to be used (defaults to LDAP).  If you specify the use of a CSV input file with this argument, then you cannot also specify one with `--users`, but you can then specify other `--users` options (such as `mapped` or `group`) for use with the CSV file.  (The Okta connector does not support `--users all`, so you must specify a `--users` option of `mapped` or `group` if you use the Okta connector.)
 {: .bordertablestyle }
 
+As of version 2.3 of User Sync, the values of most command-line parameters can also be specified in the main configuration file, in an optional section called `invocation_defaults`.  Here is an example use of that section:
+
+```yaml
+invocation_defaults:
+  adobe_only_user_action:
+    - write-file
+    - adobe-only-users.csv
+  connector: [csv, users-file.csv]
+  process_groups: Yes
+  strategy: sync
+  test_mode: Yes
+  update_user_info: True
+  users: mapped
+```
+
+As you can see from the example:
+
+* Each command-line parameter can be specified using a configuration option whose name is the same, but with hyphens replaced by underscores (e.g., the command-line parameter `process-groups` is specified by the configuration option `process_groups`).  Only those command-line parameters which control the loading of configuration files (`--config-filename`, `--config-file-encoding`) cannot be specified as configuration options, because they take effect _before_ the configuration file is loaded.
+* Command-line parameters that take zero arguments because they specify Yes/No (boolean) options (`--test-mode`, `--process-groups`, `--update-user-info`) can be specified as a having a value of Yes/True or No/False (case-insensitive), since YAML syntax treats these all as booleans.  The example above contains configuration options that use both formats.
+* Command-line parameters that are being given a single string argument should have the desired string specified as their value, as shown for the `users` option above.
+* Command-line parameters that are being given multiple string arguments should have a list of the desired strings specified as their value.  YAML supports two options for specifying lists of values, one of which (single-line) is shown for the `connector` option above and the other of which (multi-line) is shown for the `adobe_only_user_action` above.  (A list containing a single string is treated the same as a single string argument.)
+
+The handling of parameter values is identical regardless of whether the value is specified in the configuration file or on the command line.  So, for example, specifying a filename on the command line interprets that filename relative to the User Sync working directory, and the same applies to filenames specified in the `invocation_defaults` section of the configuration file.
+
+Command-line parameters that are actually specified on the command line take precedence over any specified in the configuration file.  To be precise, if a value is specified for a parameter on the command line, any value specified in the configuration file will be ignored.
+
 ---
 
 [Previous Section](configuring_user_sync_tool.md)  \| [Next Section](usage_scenarios.md)
diff --git a/examples/config files - basic/1 user-sync-config.yml b/examples/config files - basic/1 user-sync-config.yml
@@ -8,65 +8,6 @@
 # this sample and then customizing it.  Feel free to remove extraneous commentary
 # when you do your customization; doing so can greatly increase legibility.
 
-# The invocation_defaults section controls the default values used
-# for command-line arguments.  (Of course, these cannot be used to
-# change defaults for how the configuration files are read.)  This
-# entire section is optional, as are each of its entries.  But once
-# you standardize on specific argument values for all your runs,
-# putting them here can be an easy way to reliably use them, even
-# when running interactively rather than from a script.
-#
-# For arguments that take multiple arguments, such as --connector
-# and --users, the two-argument case should be specified as a list,
-# either using yaml compact notation:
-#   connector: [csv, users-file.csv]
-# or using yaml multi-line notation:
-#   connector:
-#     - csv
-#     - users-file.csv
-# The one-argument case can either use a string:
-#   connector: ldap
-# or a compact list of one value:
-#   connector: [ldap]
-# or a multi-line list of one value:
-#   connector:
-#     - ldap
-#
-# For arguments that are booleans, yaml syntax allows specifying their
-# default values using either True/False or Yes/No (case insensitive).
-#
-# If you specify filenames as part of your default values, they
-# are treated as if they were specified on the command line; that is,
-# they are interpreted relative to the User Sync current directory
-# (rather than the directory containing the configuration file).
-invocation_defaults:
-  # For argument --adobe-only-user-action, the default is 'preserve'.
-  adobe_only_user_action: preserve
-  # For argument --adobe-only-user-list, the default is empty (no value).
-  adobe_only_user_list:
-  # For argument --connector, the default is 'ldap'.
-  connector: ldap
-  # For argument --process-groups, the default is False (don't process).
-  # If you set this default to True, you can supply the argument
-  # --no-process-groups to override the default.
-  process_groups: No
-  # For argument --strategy, the default is 'sync'.
-  strategy: sync
-  # For argument --test-mode (or -t), the default is False (live run).
-  # if you set this default to True, you can supply the argument
-  # --no-test-mode (or -T) to override the default.
-  test_mode: No
-  # For argument --update-user-info, the default is False (don't update).
-  # If you set this default to True, you can supply the argument
-  # --no-update-user-info to override the default.
-  update_user_info: No
-  # For argument --user-filter, the default is empty (no value).
-  # Because regular expression notation uses special characters,
-  # any default you set should almost certainly be single-quoted.
-  user_filter:
-  # For argument --users, the default is 'all'.
-  users: all
-
 # The adobe_users section controls connection to the Adobe UM API endpoints
 # and also which users on the Adobe side are eligible to be updated.
 adobe_users:
@@ -304,3 +245,62 @@ logging:
   # This determines the detail level of the information logged to standard output.
   # See the description of file_log_level for details of the allowed values.
   console_log_level: info
+
+# The invocation_defaults section controls the default values used
+# for command-line arguments.  (Of course, these cannot be used to
+# change defaults for how the configuration files are read.)  This
+# entire section is optional, as are each of its entries.  But once
+# you standardize on specific argument values for all your runs,
+# putting them here can be an easy way to reliably use them, even
+# when running interactively rather than from a script.
+#
+# For arguments that take multiple arguments, such as --connector
+# and --users, the two-argument case should be specified as a list,
+# either using yaml compact notation:
+#   connector: [csv, users-file.csv]
+# or using yaml multi-line notation:
+#   connector:
+#     - csv
+#     - users-file.csv
+# The one-argument case can either use a string:
+#   connector: ldap
+# or a compact list of one value:
+#   connector: [ldap]
+# or a multi-line list of one value:
+#   connector:
+#     - ldap
+#
+# For arguments that are booleans, yaml syntax allows specifying their
+# default values using either True/False or Yes/No (case insensitive).
+#
+# If you specify filenames as part of your default values, they
+# are treated as if they were specified on the command line; that is,
+# they are interpreted relative to the User Sync current directory
+# (rather than the directory containing the configuration file).
+invocation_defaults:
+  # For argument --adobe-only-user-action, the default is 'preserve'.
+  adobe_only_user_action: preserve
+  # For argument --adobe-only-user-list, the default is empty (no value).
+  adobe_only_user_list:
+  # For argument --connector, the default is 'ldap'.
+  connector: ldap
+  # For argument --process-groups, the default is False (don't process).
+  # If you set this default to True, you can supply the argument
+  # --no-process-groups to override the default.
+  process_groups: No
+  # For argument --strategy, the default is 'sync'.
+  strategy: sync
+  # For argument --test-mode (or -t), the default is False (live run).
+  # if you set this default to True, you can supply the argument
+  # --no-test-mode (or -T) to override the default.
+  test_mode: No
+  # For argument --update-user-info, the default is False (don't update).
+  # If you set this default to True, you can supply the argument
+  # --no-update-user-info to override the default.
+  update_user_info: No
+  # For argument --user-filter, the default is empty (no value).
+  # Because regular expression notation uses special characters,
+  # any default you set should almost certainly be single-quoted.
+  user_filter:
+  # For argument --users, the default is 'all'.
+  users: all
diff --git a/user_sync/version.py b/user_sync/version.py
@@ -18,4 +18,4 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-__version__ = '2.3rc1'
+__version__ = '2.3rc2'
