[SPIKE] DT-909: Can we switch to using workload identity to auth as SAs across TDR GHAs? #1838
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fboulnois
reviewed
Oct 17, 2024
Comment on lines
9
to
11
| permissions: | ||
| contents: 'read' | ||
| id-token: 'write' |
There was a problem hiding this comment.
You can put these permissions at the job level:
jobs:
test-runner-staging:
runs-on: ubuntu-latest
permissions:
contents: 'read'
id-token: 'write'
snf2ye
force-pushed
the
sh/dcj-755-staging-testrunnersa
branch
from
ead90ca to
4e15e49
Compare
snf2ye
force-pushed
the
sh/dcj-755-staging-testrunnersa
branch
from
70e0d9b to
0db617f
Compare
|
This reverts commit 48e5cd6.
snf2ye
changed the title
snf2ye
changed the title
snf2ye
force-pushed
the
sh/dcj-755-staging-testrunnersa
branch
from
e04b7e1 to
965b50e
Compare
more things
snf2ye
force-pushed
the
sh/dcj-755-staging-testrunnersa
branch
from
965b50e to
200768f
Compare
snf2ye
commented
Dec 17, 2024
Comment on lines
-435
to
+440
| if (defaultCredentials instanceof ServiceAccountCredentials) { | ||
| return ((ServiceAccountCredentials) defaultCredentials).getClientEmail(); | ||
| if (defaultCredentials instanceof ServiceAccountCredentials serviceAccountCredentials) { | ||
| return serviceAccountCredentials.getClientEmail(); | ||
| } | ||
| if (defaultCredentials instanceof ExternalAccountCredentials externalAccountCredentials) { | ||
| return externalAccountCredentials.getServiceAccountEmail(); |
There was a problem hiding this comment.
We would have to handle three cases:
- service account for service account running in environments
- ExternalAccountCredentials for workload identity
- Impersonated credentials for local run if we auth while impersonating the TDR SA
snf2ye
changed the title
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira ticket: https://broadworkbench.atlassian.net/browse/DT-909
Addresses
Ideally, we would use workload identity to authenticate as service accounts across our Github actions rather than maintaining and rotating service account keys. This PR explores in what cases this is possible. I found that it is generally possible to move away from maintaining keys for external services, but without a lot of further work, maintaining keys for the TDR SA continues to be the best approach.
We currently have 9 Github action (GHA) workflows that we maintain in TDR. The approach looks different depending on the GHA.
Int and connected test run - We could move away from using the RBS SA, but still need to maintain the TDR SA.
Cherry pick image action - This PR demonstrates how to use workload identity to auth as TDR GCR SA - this will be implemented in a separate PR.
Dev Image Update action - we again just need the TDR SA, so it makes sense to use the same methodology as used in the test run action. I believe it would be easy to switch to workload identity here if desired, but I don't think it makes sense to introduce another auth mechanism if we can't totally switch to workload identity for this service account.
Staging Smoke tests - This action auths both as the TDR SA and a different "Test runner" SA. I worked to update client tests to work with just one SA, but its not set up to work with only one. My recommendation is to retire clienttests and create a new smoke test suite out of our existing integration/connected/unit junit tests. This would then just use the TDR SA. Work to be done in this ticket: https://broadworkbench.atlassian.net/browse/DT-895.
The remaining actions do not reference a service account
Obstacles encountered for TDR SA
I found it hard to pinpoint the root issue for why tests were not passing with workload identity. I have a few theories, but it was hard to test because you can't exactly replicate authenticating as a workload identity for a local run. I would argue that the different authentication methods per running environment would a blocker for moving forward with this work (local impersonation of SA, workload identity in GHA, and key in running service). But, it seems likely that I am missing something here - Could you auth as workload identity in all 3 cases? I have run out of time to continuing exploring this, but this seems like a good starting place: replicate GHA environment locally and pinpoint issue.
One theory - It seems possible that we are using an old Google API for google billing, particularly for testing if a user has access to a google billing project. However, it doesn't appear that google billing is fully supported by workload identity: https://cloud.google.com/iam/docs/federated-identity-supported-services#cloud-billing
Summary of changes
Related Changes
Cherry pick action
Successful test run: https://github.com/DataBiosphere/jade-data-repo/actions/runs/11441504960/job/31829699245