COmanage Registry Dataverse Provisioner for CADRE

Design

Preliminaries

The Registry version 4.x documentation on writing plugins is at https://spaces.at.internet2.edu/display/COmanage/Writing+Registry+Plugins.

Documentation specifically on writing provisioner plugins is at https://spaces.at.internet2.edu/display/COmanage/Provisioner+Plugins.

The Dataverse API Guide for version 6.2 is at https://guides.dataverse.org/en/6.2/api/.

The current version of the plugin targets Dataverse version 6.2.

Dataverse Server Configuration

Manipulating Dataverse people and explicit group objects requires use of the Dataverse API.

The Dataverse API has both a public and admin component. The admin component is blocked from external access by default, so the Dataverse server should be configured with :BlockedApiPolicy set to unblock-key and :BlockedApiKey set. See the Dataverse documentation at https://guides.dataverse.org/en/6.2/installation/config.html#blockedapipolicy. The provisioner refers to this unblock key as the "Admin API Token".

In addition, an API token needs to be generated and linked with a Dataverse user that has permissions to create and view explicit groups in all dataverses that the provisioner will need access to. To generate an API token, see the Dataverse documentation at https://guides.dataverse.org/en/latest/user/account.html#api-token. The provisioner refers to this API token as the "API token"

Mapping Between Dataverse and Registry Objects

People

Dataverse Authenticed User Attribute	CO Person Object	Example	Notes
identifier	Identifier of configured type	skoranda	Identifier type chosen during plugin configuration. Usually an Extended Type.
persistentUserId	Identifier of configured type	http://cilogon.org/serverT/users/27326098	Identifier type chosen during plugin configuration. OIDC sub logical choice.
id	Identifier of type Provisioning Target	1:11:v2	The Dataverse account id, saved by the plugin after prepending the ID of the provisioner instance. The :v2 suffix marks a link made under the account ownership rules (see below); a link without it was made by an earlier version and is re-checked on first use.
authenticationProviderId	None	https://archive-dev.ada.edu.au/	Part of plugin configuration. Static for all users. Provided by Dataverse administrator.
firstName	Name (given)	Scott	Name type chosen during plugin configuration (e.g. Official)
lastName	Name (family)	Koranda	Name type chosen during plugin configuration (e.g. Official)
displayName	None		Does not seem to be required by Dataverse API and then is automatically generated?
email	EmailAddress of configured Type	skoranda@illinois.edu	EmailAddress type chosen during plugin configuration (e.g. Official)

Groups

Dataverse Explicit Group	CO Group Object	Example	Notes
displayName	Name	ADA-ANU Poll Dataverse-doi:10.26193/CI4Z2S	The DOI can be exchanged using DOI API for specific Dataverse server citation URL, and then the Dataverse queried to find the owning Dataverse Collection in which to create the group.
	Identifier of configured type	10.26193/CI4Z2S	Duplicates DOI that is part of Name
description	Description	Authorized users for ADA-ANU Poll Dataverse-doi:10.26193/CI4Z2S
owning collection identifier	See notes on displayName	anupoll
aliasInOwner		authorized_doi:10.26193/CI4Z2S

How Registry Objects Work Together

For each Dataserver to be provisioned the following Registry objects form a "set" and should use corresponding names and descriptions when they are created and configured:

• An Extended Identifier type that will hold the Dataverse identifier value for a CO Person record. An example is adaproddataverse.
• An Automatic Identifier Assignment that automatically creates the Identifier of the Extended Identifier type using a rule and attaches it to the CO Person record. An example Automatic Identifier Assignment, rule, and value are
  • ADA PROD Dataverse
  • (g:1)(f)[2:(#)] (first character lowercase from given name prepended to lowercase family name and prepended if necessary with a numerical disambiguator that starts with 2)
  • skoranda7
• A Server of Type HTTP that holds the URL and API token necessary to invoke the Dataverse server API. An example is ADA PROD Dataverse with URL https://dataverse.ada.edu.au.
• A ProvisioningTarget using plugin DataverseProvisioner. An example description is ADA PROD Dataverse.

Each instance of the ProvisioningTarget must be configured with a second Server of Type HTTP that represents a DOI API server. Multiple ProvisioningTarget instances may be configured to use the same DOI Server.

Registry CO Group Naming Convention and Dataverse Explicit Group Considerations

In Dataverse a role is a set of permissions and a role may be assigned to an explict group. Each member of the explicit group then is given the set of permissions.

Each explicit group is ''owned'' by a dataverse within a Dataverse server and different dataverses may ''own'' explicit groups that have the same name. For example, the dataverses ANU Poll 2008: Environment and ANU Poll 2009: Economics may both own and use distinct explicit groups named ANU Staff for managing authorization to the dataverse and the datasets and collections within it. Further, different Dataverse servers in the community (represented in Registry by unique instantiations of the plugin) and their dataverses may also use distinct explicit groups with the same names.

Registry, however, at this time only supports a flat CO Group namespace and no two CO Groups may share the same name.

So a naming convention and logic are required to disambiguate and map a CO Group to a specific explicit group owned by a specific dataverse in a specific Dataverse server.

The naming convention for CO Groups is

<dataverse name>-doi:<DOI>

It is common to append a trailing suffix after the DOI to distinguish access modes or other variants, for example -Restricted, -General, or any other single token without additional dashes. The provisioner first looks up the whole Identifier value as a DOI, so a DOI that itself contains a dash is used as is. Only when that lookup definitely finds nothing does the provisioner strip the final -<suffix> and look up the rest, so both ANU Poll-doi:10.12345/XYZ and ANU Poll-doi:10.12345/XYZ-Restricted resolve to the same dataset. Any other lookup failure stops provisioning for that CO Group. The Dataverse explicit group alias is generated as authorized_<doi> and, when a suffix is present, extended to authorized_<doi>_<suffix>, where the suffix is lowercased and non-alphanumeric characters are converted to underscores. This keeps each access variant distinct while still mapping to the same DOI owners. Provisioned groups created before this change may use the shorter alias; if so, reprovision or adjust the Dataverse explicit group to match the new naming rule.

Also note that the convention is that the CO Group has an attached Identifier of the configured DOI type that holds the DOI (value only, no prefix string 'doi').

The logic for mapping the CO Group name to an owner dataverse within a Dataverse server is:

1. Find the Identifier attached the CO Group with the configured DOI type and extract the DOI.
2. Query the configured DOI server and from the returned metadata extract the Dataverse server.
3. Query the Dataverse server with the DOI to obtain the dataverse owner of the explicit group.

Note that for development and testing purposes, when the plugin is so configured the second step above is skipped and the plugin assumes that all CO Groups are mapped to the Dataverse server associated with the plugin.

Implementation Details

• Each instantiation of the plugin targets one and only one Dataverse server instance.

• The plugin assumes that the Dataverse server is configured with an unblock-key (see above discussion) and so configuration (see below) requires both an API token and an Admin API token (used as the unblock-key).

• The plugin only takes action on the following CO Person record events in Registry:

  • CoPersonAdded
  • CoPersonPetitionProvisioned
  • CoPersonPipelineProvisioned
  • CoPersonReprovisionRequested
  • CoPersonUpddated

  All of those events are treated identitically by the plugin.

  Note especially that the Dataverse API does not support deleting a person object and the plugin does not take any action when a CO Person record is expunged or the Status is set to anything other than Active.

• The plugin only takes action on the following CO Group events in Registry:

  • CoGroupAdded
  • CoGroupUpdated
  • CoGroupReprovisionRequested

  All other provisioner events are ignored by the plugin.

• The plugin will only attempt to create a user in Dataverse when all of the following are true:

  • The CO Person record has a Status that is Active.
  • The CO Person record has an attached Identifier of the configured Dataverse Identifier type.
  • The CO Person record has an attached Identifier of the configured Dataverse Persistent User Identifier type.
  • The CO Person record has a Name of the configured Dataverse Name type.
  • The CO Person record has an attached EmailAddress of the configured Dataverse Email type.
  • The CO Person is a member of at least one authorization group, that is a member of a CO Group with the configured Group Identifier type.
  • No Dataverse account that belongs to the CO Person already exists (see Account Ownership below).

• See Account Ownership below for how the plugin decides which Dataverse account belongs to a CO Person, and what happens when it cannot.

• The plugin does not support updates for CO Group names or descriptions. Once the explicit group is created in Dataverse it cannot be changed or deleted.

• The plugin will process CoGroupUpdated events, however, in order to try and create an explicit group after an Identifier with the correct type has been attached to a CO Group. So effectively explicit groups are only created in Dataverse due to a CoGroupUpdated or CoGroupReprovisionRequested operation and not for CoGroupAdded operations, unless the CO Group and Identifier are created in one single transaction (which is usually not the case when creating using the REST API v1).

• When evaluating provisioning status for a CO Group the plugin now confirms that the Dataverse explicit group’s display name matches the Registry CO Group name. If the alias exists but the display name differs, status reports the mismatch instead of marking the group as provisioned.

Account Ownership

A Registry-assigned Dataverse username (for example RNadal) can match an existing Dataverse account that belongs to a different person. The plugin therefore never treats an account as a CO Person's because of the username alone. An account belongs to a CO Person only when:

1. its authenticationProviderId and persistentUserId equal the configured authentication provider and the CO Person's persistent user Identifier, or
2. failing that, its email equals the CO Person's EmailAddress of the configured type exactly (ignoring case), and that EmailAddress is marked verified in Registry.

The candidate accounts are the account at the CO Person's Registry username and any account whose email matches. When one belongs to the CO Person the plugin links it, stores the link as an Identifier of type Provisioning Target (<target>:<id>:v2), and, if the account's username differs, changes the CO Person's Dataverse Identifier to match. When none does and the username is free, the plugin creates the account and links it.

Email matching is safe only if a Dataverse account cannot carry an email its owner did not prove: builtin account sign-up and account email editing should be disabled on the Dataverse server, and every enabled login provider should assert verified emails.

Group membership changes are made only for the linked account. When a CO Person is linked, the plugin also grants the account membership in the explicit group of every DOI CO Group the CO Person belongs to. Identifier changes made by the plugin do not trigger provisioning, and viewing provisioning status never changes anything.

Conflicts

The plugin fails closed: it creates no account, grants no membership, and records a conflict in the Registry log (and in the CO Person's provisioning status) when

• the CO Person's username belongs to another person's Dataverse account,
• account creation fails because the email or login identity is already used,
• the CO Person's email matches an account but is not verified in Registry,
• the account is already linked to another CO Person,
• the account's username is the Registry Identifier of another CO Person, or
• a link made by an earlier version of the plugin fails the rules above.

Viewing the CO Person's provisioning status re-checks these conditions without changing anything and names the conflict. A failed account creation cannot be re-checked that way, so status then points to the Registry log.

A CO Group membership removal for a CO Person without a trusted link is not sent to Dataverse, because the earlier grant may sit on another person's account. The log and the provisioning status note this; the audit below finds such memberships.

To resolve a username conflict, change the CO Person's Identifier of the configured Dataverse Identifier type to an unused value and request reprovisioning of the CO Person. The plugin then links the CO Person's own account, or creates one, and grants their CO Group memberships. Access that was granted to the wrong account earlier is not removed automatically.

Audit

The audit lists Dataverse group access that may sit on the wrong account. It reads Registry and Dataverse and changes nothing. From the Registry app directory run

Console/cake DataverseProvisioner.DataverseAudit <target id>

where <target id> is the ID of the CO Dataverse Provisioner Target. For each CO Group with a DOI Identifier it prints

• PERSON lines for CO Group members whose Dataverse link is in conflict, naming the account at their username and whether that account is in the Dataverse explicit group, and
• UNEXPECTED lines for Dataverse explicit group members not backed by a CO Group member with a trusted link.

Run the audit after deploying this version and periodically afterward, and remove wrong memberships in Dataverse by hand.

Configuration

All plugin configuration must be done as a Registry CO administrator.

Create Extended Identifier Types

Repeat the steps below once for each Dataverse server.

1. Browse to Configuration > Extended Types.
2. In the drop-down for For Attribute choose Identifier (CO Person, Group), which is the default.
3. Click FILTER.
4. Click Add Extended Type.
5. Verify that the Attribute field in the form has value Identifier (CO Person, Group).
6. In the Name field enter a unique value, for example dataverseid. For a second Dataverse server the value might be dataverseid2, and for the third dataverseid3, and so on.
7. In the Display Name field enter a descriptive name. This is usually the name of a Dataverse server such as ADA PROD ANU Dataverse or ADA DEV Dataverse.
8. For Status choose Active from the drop-down menu.
9. Click ADD.

Complete the steps below once to create a DOI Identifier type to use with Registry CO Groups.

1. Browse to Configuration > Extended Types.
2. In the drop-down for For Attribute choose Identifier (CO Person, Group), which is the default.
3. Click FILTER.
4. Click Add Extended Type.
5. Verify that the Attribute field in the form has value Identifier (CO Person, Group).
6. In the Name field enter doi.
7. In the Display Name field enter DOI.
8. For Status choose Active from the drop-down menu.
9. Click ADD.

Create Automatic Identifier Assignments

Repeat the steps below once for each Dataverse server.

1. Browse to Configuration > Identifier Assignments.
2. Click Add Identifier Assignment.
3. For Description enter the name of a Dataverse server, for example ADA PROD ANU Dataverse.
4. For Status choose Active from the drop-down menu.
5. For Context choose CO Person from the drop-down menu.
6. For Type choose the type from the drop-down menu that corresponds to the value entered for Description, for example ADA PROD ANU Dataverse.
7. Do not tick the box for Login.
8. Do not enter a value for Order. The value will be auto-assigned.
9. For Algorithm select Sequential from the drop-down menu.
10. Ignore the Plugin field.
11. For Format enter (g:1)(f)[2:(#)]. This will cause the first character from the users Give Name in lowercase to be prepended to the lowercase Family Name to create the Identifier value. If a value already exists then a digit will be used as a disambiguator, starting with 2 and increasing as necessary until the next unique value is found and assigned.
12. Leave Minimum Length empty.
13. Do not tick Enable Transliteration.
14. For Permitted Characters choose AlphaNumberic Only from the drop-down menu.
15. Leave Minimum Collision Number empty.
16. Click SAVE.

Create the Registry Server Object for A Dataverse Server

Repeat the steps below once for each Dataverse server.

1. Browse to Servers > Add a New Server.
2. Provide a Description for the Server. This is usually a meaningful name, for example Production Dataverse (Australian Data Archive).
3. Choose Active for the Server Status.
4. Choose HTTP for the Server Type.
5. Click ADD.
6. For Server URL enter the URL for the Dataverse server, for example https://dataverse.ada.edu.au.
7. Leave the field Username empty.
8. For Password or Token enter a valid API token for the Dataverse server. Do not use an admin API token.
9. For HTTP Authentication Type choose None.
10. Tick the box for Require certificate verification.
11. Tick the box for Require name verification.
12. Click SAVE.

Create the Registry Server Object for a DOI Server

1. Browse to Servers > Add a New Server.
2. Provide a Description for the Server. This is usually a meaningful name, for example doi.org API Server.
3. Choose Active for the Server Status.
4. Choose HTTP for the Server Type.
5. Click ADD.
6. For Server URL enter the URL for the DOI API server, for example https://doi.org.
7. Leave the field Username empty.
8. Leave the field Password or Token empty.
9. For HTTP Authentication Type choose None.
10. Tick the box for Require certificate verification.
11. Tick the box for Require name verification.
12. Click SAVE.

Multiple DataverseProvisioner instantiations may use the same DOI API server.

Create the Provisioning Target

Repeat the steps below once for each Dataverse server.

1. Browse to Configuration > Provisioning Targets > Add Provisioning Target.
2. Provide a Description for the provisioning target. This is usually a meaningful name, for example Production Dataverse.
3. Choose DataverseProvisioner from the Plugin drop-down menu.
4. Choose Automatic Mode from the Status drop-down menu.
5. Do not choose a Provisioning Group.
6. Do not choose Skip If Associated With Org Identity Source.
7. Leave the field Order empty. It will later be auto-assigned.
8. Click ADD.
9. For Dataverse Server choose a Server object from the drop-down menu that represents a Dataverse server.
10. For Admin API Token enter a valid admin API token suitable for unblocking the /api/admin endpoint.
11. For DOI Server API choose a Server object from the drop-down menu that represents a DOI API server to use when mapping DOI values to Dataverse server.
12. For Dataverse Identifier Type select from the drop-down menu the Identifier of the type configured in the step above "Create Extended Identifier Types".
13. For Dataverse Persistent User Identifier Type select OIDC sub from the drop-down menu.
14. For Dataverse Name Type select Official from the drop-down menu.
15. For Dataverse Email Type select Official from the drop-down menu.
16. For Group Identifier Type select DOI from the drop-down menu.
17. For Dataverse Authentication Provider ID enter the value configured by the Dataverse administrator in the Dataverse server.
18. If operating in testing mode and a DOI server will not be able to map from the DOI to a Dataverse server instance then tick the box for Skip Dataverse Server Mapping. This will cause each CO Group representing an explicit group owned by a dataverse to be created in each Dataverse server represented by a provisioning target (instance of the plugin).
19. Click SAVE.

Testing

The account ownership rules in Lib/DataverseOwnership.php have unit tests that run with PHPUnit outside Registry:

composer install
vendor/bin/phpunit

Everything else needs a Registry deployment with a Dataverse server, such as CADRE TEST.