Private container registry authentication

[rivershah]

Certain pipelines need to access containers in private gitlab registries. Pushing images as an intermediate step to google built in registries is not a desired step due to security & maintenance policies. Is it possible to authenticate dsub / backend provider to gitlab and enable pulling private images? Thank you

[mbookman]

This is a good question @rivershah!

For the google-v2 and google-cls-v2, this would appear to be doable based on the following documentation:

https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#action

imageUri | stringRequired
....The image URI can be either a complete host and image specification (e.g., quay.io/biocontainers/samtools), a library and image name (e.g., google/cloud-sdk) or a bare image name ('bash') to pull from the default library. No schema is required in any of these cases.

If the specified image is not public, the service account specified for the Virtual Machine must have access to pull the images from GCR, or appropriate credentials must be specified in the google.cloud.lifesciences.v2beta.Action.credentials field.

Looking at the credentials field:

credentials | object (Secret)

If the specified image is hosted on a private registry other than Google Container Registry, the credentials required to pull the image must be specified here as an encrypted secret.

The secret must decrypt to a JSON-encoded dictionary containing both username and password keys.

For the (still experimental) google-batch provider, I see:

https://cloud.google.com/batch/docs/reference/rest/v1/projects.locations.jobs#Container

imageUri | string
The URI to pull the container image from.

username | stringOptional
username for logging in to a docker registry. If username matches projects//secrets//versions/* then Batch will read the username from the Secret Manager.

password | stringOptional
password for logging in to a docker registry. If password matches projects//secrets//versions/* then Batch will read the password from the Secret Manager

So it looks like there's something there to work with.

[rivershah]

Fantastic. May I please request that we include this feature request. I have looked at the authentication documentation for gitlab, and this seems straightforward if dsub can expose the relevant command line args.

I have included authentication token patterns to two of the most popular CI/CD platforms as a reference.

https://docs.gitlab.com/ee/user/packages/container_registry/authenticate_with_container_registry.html
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

edit: On further review, I should be able to contribute this feature. Will submit pull request when verified working.