Skip to content

Commit 4023800

Browse files
author
Martin Jackson
committed
Add support for parsing secrets into intermediate structure and creating
k8s secret objects Ensure only push_secrets runs from vault_utils Update makefile - remove extra targets and make fix none Revert version bump as we only add fields Conditionalize check change output Start module to load parsed secrets into vault New machinery for vault secrets loading Make the linters pass again Inject policies Add some more code to test readiness to load Correct typo Add vault_hub Add vaultMount Rename new modules to v2 Update inject_field method Correct field typo Temporarily print command Add more logic Hopefully fix secret loading issue with counter Count per secret Pick stuff out of secret that we need Fix lint issue Refactor tests to use fixture constants Correctly spell exclusion for ansible-lint Provide a target to exercise legacy code path Add error exists for missing args and update docs Reverse test for override Also process base64 for generated secrets Be more explicit about what we load Test framework for loading parsed_secret data Fix linting errors Finish test suite Last linter stuff Change schema; code and tests to follow Add target_namespaces phase 1 more passing, but some still fail Passing again All pass Check the correct variable in golang-external-secrets chart Update YAML parsing to do decodes right Add tests and tighten up code for retrieving block yaml quotes Add test for kubernetes secret object and block yaml
1 parent 415f681 commit 4023800

File tree

78 files changed

+3439
-66
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+3439
-66
lines changed

‎.ansible-lint

+3
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,7 @@ skip_list:
1414
exclude_paths:
1515
- ./ansible/playbooks/vault/vault.yaml
1616
- ./ansible/playbooks/iib-ci/iib-ci.yaml
17+
- ./ansible/playbooks/k8s_secrets/k8s_secrets.yml
18+
- ./ansible/playbooks/process_secrets/process_secrets.yml
19+
- ./ansible/playbooks/process_secrets/display_secrets_info.yml
1720
- ./ansible/roles/vault_utils/tests/test.yml

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ __pycache__/
55
*.swo
66
values-secret.yaml
77
.*.expected.yaml
8+
.vscode
89
pattern-vault.init
910
pattern-vault.init.bak
1011
super-linter.log

‎Makefile

+29-1
Original file line numberDiff line numberDiff line change
@@ -77,9 +77,37 @@ uninstall: ## runs helm uninstall
7777
@oc delete csv -n openshift-operators $(CSV)
7878

7979
.PHONY: load-secrets
80-
load-secrets: ## loads the secrets into the vault
80+
load-secrets: ## loads the secrets into the backend determined by values-global setting
81+
common/scripts/process-secrets.sh $(NAME)
82+
83+
.PHONY: legacy-load-secrets
84+
legacy-load-secrets: ## loads the secrets into vault (only)
8185
common/scripts/vault-utils.sh push_secrets $(NAME)
8286

87+
.PHONY: secrets-backend-vault
88+
secrets-backend-vault: ## Edits values files to use default Vault+ESO secrets config
89+
common/scripts/set-secret-backend.sh vault
90+
common/scripts/manage-secret-app.sh vault present
91+
common/scripts/manage-secret-app.sh golang-external-secrets present
92+
common/scripts/manage-secret-namespace.sh validated-patterns-secrets absent
93+
@git diff --exit-code || echo "Secrets backend set to vault, please review changes, commit, and push to activate in the pattern"
94+
95+
.PHONY: secrets-backend-kubernetes
96+
secrets-backend-kubernetes: ## Edits values file to use Kubernetes+ESO secrets config
97+
common/scripts/set-secret-backend.sh kubernetes
98+
common/scripts/manage-secret-namespace.sh validated-patterns-secrets present
99+
common/scripts/manage-secret-app.sh vault absent
100+
common/scripts/manage-secret-app.sh golang-external-secrets present
101+
@git diff --exit-code || echo "Secrets backend set to kubernetes, please review changes, commit, and push to activate in the pattern"
102+
103+
.PHONY: secrets-backend-none
104+
secrets-backend-none: ## Edits values files to remove secrets manager + ESO
105+
common/scripts/set-secret-backend.sh none
106+
common/scripts/manage-secret-app.sh vault absent
107+
common/scripts/manage-secret-app.sh golang-external-secrets absent
108+
common/scripts/manage-secret-namespace.sh validated-patterns-secrets absent
109+
@git diff --exit-code || echo "Secrets backend set to none, please review changes, commit, and push to activate in the pattern"
110+
83111
.PHONY: load-iib
84112
load-iib: ## CI target to install Index Image Bundles
85113
@set -e; if [ x$(INDEX_IMAGES) != x ]; then \

‎ansible/playbooks/k8s_secrets/k8s_secrets.yml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
- name: Secrets parsing and direct loading
3+
hosts: localhost
4+
connection: local
5+
gather_facts: false
6+
roles:
7+
- find_vp_secrets
8+
- cluster_pre_check
9+
- k8s_secret_utils

‎ansible/playbooks/process_secrets/display_secrets_info.yml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
- name: Parse and display secrets
3+
hosts: localhost
4+
connection: local
5+
gather_facts: false
6+
vars:
7+
secrets_backing_store: "vault"
8+
tasks:
9+
# Set the VALUES_SECRET environment variable to the file to parse
10+
- name: Find and decrypt secrets if needed
11+
ansible.builtin.include_role:
12+
name: find_vp_secrets
13+
14+
# find_vp_secrets will return a plaintext data structure called values_secrets_data
15+
# This will allow us to determine schema version and which backend to use
16+
- name: Determine how to load secrets
17+
ansible.builtin.set_fact:
18+
secrets_yaml: '{{ values_secrets_data | from_yaml }}'
19+
20+
- name: Parse secrets data
21+
no_log: '{{ override_no_log | default(true) }}'
22+
parse_secrets_info:
23+
values_secrets_plaintext: "{{ values_secrets_data }}"
24+
secrets_backing_store: "{{ secrets_backing_store }}"
25+
register: secrets_results
26+
27+
- name: Display secrets data
28+
ansible.builtin.debug:
29+
var: secrets_results

‎ansible/playbooks/process_secrets/process_secrets.yml

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
- name: Parse and load secrets
3+
hosts: localhost
4+
connection: local
5+
gather_facts: false
6+
vars:
7+
secrets_role: 'vault_utils'
8+
pattern_name: 'common'
9+
pattern_dir: '.'
10+
secrets_backing_store: 'vault'
11+
tasks_from: 'push_parsed_secrets'
12+
tasks:
13+
- name: "Run secret-loading pre-requisites"
14+
ansible.builtin.include_role:
15+
name: '{{ item }}'
16+
loop:
17+
- cluster_pre_check
18+
- find_vp_secrets
19+
20+
# find_vp_secrets will return a plaintext data structure called values_secrets_data
21+
# This will allow us to determine schema version and which backend to use
22+
- name: Determine how to load secrets
23+
ansible.builtin.set_fact:
24+
secrets_yaml: '{{ values_secrets_data | from_yaml }}'
25+
26+
- name: Parse secrets data
27+
no_log: '{{ override_no_log | default(true) }}'
28+
parse_secrets_info:
29+
values_secrets_plaintext: "{{ values_secrets_data }}"
30+
secrets_backing_store: "{{ secrets_backing_store }}"
31+
register: secrets_results
32+
33+
# Use the k8s secrets loader when explicitly requested
34+
- name: Determine role to use to load secrets
35+
ansible.builtin.set_fact:
36+
secrets_role: 'k8s_secret_utils'
37+
tasks_from: 'inject_k8s_secrets'
38+
when:
39+
- secrets_backing_store == "kubernetes" or secrets_backing_store == "none"
40+
- secrets_yaml['version'] | default('2.0') >= '2.0'
41+
42+
# secrets_role will have been changed from the default if needed
43+
- name: Load secrets using designated role and tasks
44+
ansible.builtin.include_role:
45+
name: '{{ secrets_role }}'
46+
tasks_from: '{{ tasks_from }}'
47+
vars:
48+
kubernetes_secret_objects: "{{ secrets_results['kubernetes_secret_objects'] }}"
49+
vault_policies: "{{ secrets_results['vault_policies'] }}"
50+
parsed_secrets: "{{ secrets_results['parsed_secrets'] }}"

‎ansible/playbooks/vault/vault.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,6 @@
44
connection: local
55
gather_facts: false
66
roles:
7+
- find_vp_secrets
8+
- cluster_pre_check
79
- vault_utils

‎ansible/plugins/module_utils/load_secrets_common.py

+20
Original file line numberDiff line numberDiff line change
@@ -102,3 +102,23 @@ def get_ini_value(inifile, inisection, inikey):
102102
config = configparser.ConfigParser()
103103
config.read(inifile)
104104
return config.get(inisection, inikey, fallback=None)
105+
106+
107+
def stringify_dict(input_dict):
108+
"""
109+
Return a dict whose keys and values are all co-erced to strings, for creating labels and annotations in the
110+
python Kubernetes module
111+
112+
Parameters:
113+
input_dict(dict): A dictionary of keys and values
114+
115+
Returns:
116+
117+
obj: The same dict in the same order but with the keys coerced to str
118+
"""
119+
output_dict = {}
120+
121+
for key, value in input_dict.items():
122+
output_dict[str(key)] = str(value)
123+
124+
return output_dict

0 commit comments

Comments
 (0)