Skip to content

Commit 5738352

Browse files
jimrightjimright
authored
Improve creation and handling of CDP groups in cdp-deploy (#101)
Signed-off-by: Jim Enright <[email protected]>
1 parent 47d968e commit 5738352

26 files changed

+313
-142
lines changed

‎modules/terraform-cdp-deploy/README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ No resources.
4343
| Name | Description | Type | Default | Required |
4444
|------|-------------|------|---------|:--------:|
4545
| <a name="input_backup_storage_location"></a> [backup\_storage\_location](#input\_backup\_storage\_location) | Backup storage location. The location has to be in uri format for the cloud provider - i.e. s3a:// for AWS, abfs:// for Azure, gs:// | `string` | n/a | yes |
46+
| <a name="input_cdp_groups"></a> [cdp\_groups](#input\_cdp\_groups) | List of CDP Groups to be added to the IDBroker mappings of the environment. If create\_group is set to true then the group will be created. | <pre>set(object({<br> name = string<br> create_group = bool<br> sync_membership_on_user_login = optional(bool)<br> add_id_broker_mappings = bool<br> })<br> )</pre> | n/a | yes |
4647
| <a name="input_data_storage_location"></a> [data\_storage\_location](#input\_data\_storage\_location) | Data storage location. The location has to be in uri format for the cloud provider - i.e. s3a:// for AWS, abfs:// for Azure, gs:// | `string` | n/a | yes |
4748
| <a name="input_deployment_template"></a> [deployment\_template](#input\_deployment\_template) | Deployment Pattern to use for Cloud resources and CDP | `string` | n/a | yes |
4849
| <a name="input_env_prefix"></a> [env\_prefix](#input\_env\_prefix) | Shorthand name for the environment. Used in CDP resource descriptions. This will be used to construct the value of where any of the CDP resource variables (e.g. environment\_name, cdp\_iam\_admin\_group\_name) are not defined. | `string` | n/a | yes |
@@ -85,8 +86,6 @@ No resources.
8586
| <a name="input_azure_vnet_name"></a> [azure\_vnet\_name](#input\_azure\_vnet\_name) | Azure Virtual Network ID. Required for CDP deployment on Azure. | `string` | `null` | no |
8687
| <a name="input_azure_xaccount_app_pword"></a> [azure\_xaccount\_app\_pword](#input\_azure\_xaccount\_app\_pword) | Password for the Azure AD Cross Account Application. Required for CDP deployment on Azure. | `string` | `null` | no |
8788
| <a name="input_azure_xaccount_app_uuid"></a> [azure\_xaccount\_app\_uuid](#input\_azure\_xaccount\_app\_uuid) | UUID for the Azure AD Cross Account Application. Required for CDP deployment on Azure. | `string` | `null` | no |
88-
| <a name="input_cdp_admin_group_name"></a> [cdp\_admin\_group\_name](#input\_cdp\_admin\_group\_name) | Name of the CDP IAM Admin Group associated with the environment. Defaults to '<env\_prefix>-cdp-admin-group' if not specified. | `string` | `null` | no |
89-
| <a name="input_cdp_user_group_name"></a> [cdp\_user\_group\_name](#input\_cdp\_user\_group\_name) | Name of the CDP IAM User Group associated with the environment. Defaults to '<env\_prefix>-cdp-user-group' if not specified. | `string` | `null` | no |
9089
| <a name="input_cdp_xacccount_credential_name"></a> [cdp\_xacccount\_credential\_name](#input\_cdp\_xacccount\_credential\_name) | Name of the CDP Cross Account Credential. Defaults to '<env\_prefix>-xaccount-cred' if not specified. If create\_cdp\_credential is set to false then this should should be a valid pre-existing credential. | `string` | `null` | no |
9190
| <a name="input_create_cdp_credential"></a> [create\_cdp\_credential](#input\_create\_cdp\_credential) | Flag to specify if the CDP Cross Account Credential should be created. If set to false then cdp\_xacccount\_credential\_name should be a valid pre-existing credential. | `bool` | `true` | no |
9291
| <a name="input_datalake_async_creation"></a> [datalake\_async\_creation](#input\_datalake\_async\_creation) | Flag to specify if Terraform should wait for CDP datalake resource creation/deletion | `bool` | `false` | no |
@@ -148,4 +147,5 @@ No resources.
148147
|------|-------------|
149148
| <a name="output_cdp_environment_crn"></a> [cdp\_environment\_crn](#output\_cdp\_environment\_crn) | CDP Environment CRN |
150149
| <a name="output_cdp_environment_name"></a> [cdp\_environment\_name](#output\_cdp\_environment\_name) | CDP Environment Name |
150+
| <a name="output_cdp_groups"></a> [cdp\_groups](#output\_cdp\_groups) | Details about CDP Groups |
151151
<!-- END_TF_DOCS -->

‎modules/terraform-cdp-deploy/defaults.tf

-6
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,6 @@ locals {
3030
cdp_xacccount_credential_name = coalesce(var.cdp_xacccount_credential_name,
3131
"${var.env_prefix}-xaccount-cred")
3232

33-
cdp_admin_group_name = coalesce(var.cdp_admin_group_name,
34-
"${var.env_prefix}-${local.cloud_shorthand[var.infra_type]}-cdp-admin-group")
35-
36-
cdp_user_group_name = coalesce(var.cdp_user_group_name,
37-
"${var.env_prefix}-${local.cloud_shorthand[var.infra_type]}-cdp-user-group")
38-
3933
datalake_scale = coalesce(
4034
var.datalake_scale,
4135
(var.deployment_template == "public" ?

‎modules/terraform-cdp-deploy/examples/ex01-aws-basic/main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ module "cdp_deploy" {
9292
region = var.aws_region
9393
keypair_name = var.aws_key_pair
9494
deployment_template = var.deployment_template
95+
cdp_groups = var.cdp_groups
9596

9697
environment_async_creation = var.environment_async_creation
9798
datalake_async_creation = var.datalake_async_creation

‎modules/terraform-cdp-deploy/examples/ex01-aws-basic/variables.tf

+29
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,35 @@ variable "datalake_async_creation" {
6868

6969
default = false
7070
}
71+
72+
variable "cdp_groups" {
73+
type = set(object({
74+
name = string
75+
create_group = bool
76+
sync_membership_on_user_login = optional(bool)
77+
add_id_broker_mappings = bool
78+
})
79+
)
80+
81+
description = "List of CDP Groups to be added to the IDBroker mappings of the environment. If create_group is set to true then the group will be created."
82+
83+
validation {
84+
condition = (var.cdp_groups == null ? true : alltrue([
85+
for grp in var.cdp_groups :
86+
length(grp.name) >= 1 && length(grp.name) <= 64
87+
]))
88+
error_message = "The length of all CDP group names must be 64 characters or less."
89+
}
90+
validation {
91+
condition = (var.cdp_groups == null ? true : alltrue([
92+
for grp in var.cdp_groups :
93+
can(regex("^[a-zA-Z0-9\\-\\_\\.]{1,90}$", grp.name))
94+
]))
95+
error_message = "CDP group names can consist only of letters, numbers, dots (.), hyphens (-) and underscores (_)."
96+
}
97+
98+
default = null
99+
}
71100
# ------- Network Resources -------
72101
variable "ingress_extra_cidrs_and_ports" {
73102
type = object({

‎modules/terraform-cdp-deploy/examples/ex02-azure-basic/main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ module "cdp_deploy" {
8383
region = var.azure_region
8484
public_key_text = var.public_key_text
8585
deployment_template = var.deployment_template
86+
cdp_groups = var.cdp_groups
8687

8788
environment_async_creation = var.environment_async_creation
8889
datalake_async_creation = var.datalake_async_creation

‎modules/terraform-cdp-deploy/examples/ex02-azure-basic/variables.tf

+29
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,35 @@ variable "datalake_async_creation" {
6767

6868
default = false
6969
}
70+
71+
variable "cdp_groups" {
72+
type = set(object({
73+
name = string
74+
create_group = bool
75+
sync_membership_on_user_login = optional(bool)
76+
add_id_broker_mappings = bool
77+
})
78+
)
79+
80+
description = "List of CDP Groups to be added to the IDBroker mappings of the environment. If create_group is set to true then the group will be created."
81+
82+
validation {
83+
condition = (var.cdp_groups == null ? true : alltrue([
84+
for grp in var.cdp_groups :
85+
length(grp.name) >= 1 && length(grp.name) <= 64
86+
]))
87+
error_message = "The length of all CDP group names must be 64 characters or less."
88+
}
89+
validation {
90+
condition = (var.cdp_groups == null ? true : alltrue([
91+
for grp in var.cdp_groups :
92+
can(regex("^[a-zA-Z0-9\\-\\_\\.]{1,90}$", grp.name))
93+
]))
94+
error_message = "CDP group names can consist only of letters, numbers, dots (.), hyphens (-) and underscores (_)."
95+
}
96+
97+
default = null
98+
}
7099
# ------- Network Resources -------
71100
variable "ingress_extra_cidrs_and_ports" {
72101
type = object({

‎modules/terraform-cdp-deploy/examples/ex03-gcp-basic/main.tf

+1
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ module "cdp_deploy" {
6969
region = var.gcp_region
7070
public_key_text = var.public_key_text
7171
deployment_template = var.deployment_template
72+
cdp_groups = var.cdp_groups
7273

7374
environment_async_creation = var.environment_async_creation
7475
datalake_async_creation = var.datalake_async_creation

‎modules/terraform-cdp-deploy/examples/ex03-gcp-basic/variables.tf

+29
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,35 @@ variable "datalake_async_creation" {
7474

7575
default = false
7676
}
77+
78+
variable "cdp_groups" {
79+
type = set(object({
80+
name = string
81+
create_group = bool
82+
sync_membership_on_user_login = optional(bool)
83+
add_id_broker_mappings = bool
84+
})
85+
)
86+
87+
description = "List of CDP Groups to be added to the IDBroker mappings of the environment. If create_group is set to true then the group will be created."
88+
89+
validation {
90+
condition = (var.cdp_groups == null ? true : alltrue([
91+
for grp in var.cdp_groups :
92+
length(grp.name) >= 1 && length(grp.name) <= 64
93+
]))
94+
error_message = "The length of all CDP group names must be 64 characters or less."
95+
}
96+
validation {
97+
condition = (var.cdp_groups == null ? true : alltrue([
98+
for grp in var.cdp_groups :
99+
can(regex("^[a-zA-Z0-9\\-\\_\\.]{1,90}$", grp.name))
100+
]))
101+
error_message = "CDP group names can consist only of letters, numbers, dots (.), hyphens (-) and underscores (_)."
102+
}
103+
104+
default = null
105+
}
77106
# ------- Network Resources -------
78107
variable "ingress_extra_cidrs_and_ports" {
79108
type = object({

‎modules/terraform-cdp-deploy/main.tf

+3-6
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,7 @@ module "cdp_on_aws" {
2525
datalake_name = local.datalake_name
2626
create_cdp_credential = var.create_cdp_credential
2727
cdp_xacccount_credential_name = local.cdp_xacccount_credential_name
28-
cdp_admin_group_name = local.cdp_admin_group_name
29-
cdp_user_group_name = local.cdp_user_group_name
28+
cdp_groups = var.cdp_groups
3029

3130
security_group_default_id = var.aws_security_group_default_id
3231
security_group_knox_id = var.aws_security_group_knox_id
@@ -101,8 +100,7 @@ module "cdp_on_azure" {
101100
datalake_name = local.datalake_name
102101
create_cdp_credential = var.create_cdp_credential
103102
cdp_xacccount_credential_name = local.cdp_xacccount_credential_name
104-
cdp_admin_group_name = local.cdp_admin_group_name
105-
cdp_user_group_name = local.cdp_user_group_name
103+
cdp_groups = var.cdp_groups
106104

107105
security_group_default_uri = var.azure_security_group_default_uri
108106
security_group_knox_uri = var.azure_security_group_knox_uri
@@ -194,8 +192,7 @@ module "cdp_on_gcp" {
194192
datalake_name = local.datalake_name
195193
create_cdp_credential = var.create_cdp_credential
196194
cdp_xacccount_credential_name = local.cdp_xacccount_credential_name
197-
cdp_admin_group_name = local.cdp_admin_group_name
198-
cdp_user_group_name = local.cdp_user_group_name
195+
cdp_groups = var.cdp_groups
199196

200197
firewall_default_id = var.gcp_firewall_default_id
201198
firewall_knox_id = var.gcp_firewall_knox_id

‎modules/terraform-cdp-deploy/modules/aws/data.tf

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Copyright 2025 Cloudera, Inc. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
data "cdp_iam_group" "cdp_groups" {
16+
17+
for_each = {
18+
for k, v in coalesce(var.cdp_groups, []) : k.name => v
19+
}
20+
21+
group_name = each.value.name
22+
23+
depends_on = [cdp_iam_group.cdp_groups]
24+
}
25+

‎modules/terraform-cdp-deploy/modules/aws/defaults.tf

+10
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,14 @@ locals {
1919
var.cdp_xacccount_credential_name :
2020
cdp_environments_aws_credential.cdp_cred[0].credential_name
2121
)
22+
23+
# Construct IDBroker mappings
24+
cdp_group_id_broker_mappings = [
25+
for grp, grp_details in coalesce(var.cdp_groups, []) :
26+
{
27+
accessor_crn = data.cdp_iam_group.cdp_groups[grp_details.name].crn
28+
role = var.datalake_admin_role_arn
29+
}
30+
if try(grp_details.add_id_broker_mappings, false)
31+
]
2232
}

‎modules/terraform-cdp-deploy/modules/aws/main.tf

+13-25
Original file line numberDiff line numberDiff line change
@@ -82,27 +82,21 @@ resource "cdp_environments_aws_environment" "cdp_env" {
8282
]
8383
}
8484

85-
# ------- CDP Admin Group -------
85+
# ------- CDP Group -------
8686
# Create group
87-
resource "cdp_iam_group" "cdp_admin_group" {
88-
group_name = var.cdp_admin_group_name
89-
sync_membership_on_user_login = false
90-
}
91-
92-
# TODO: Assign roles and resource roles to the group
93-
94-
# TODO: Assign users to the group
87+
resource "cdp_iam_group" "cdp_groups" {
88+
for_each = {
89+
for k, v in coalesce(var.cdp_groups, []) : k.name => v
90+
if v.create_group
91+
}
9592

96-
# ------- CDP User Group -------
97-
# Create group
98-
resource "cdp_iam_group" "cdp_user_group" {
99-
group_name = var.cdp_user_group_name
100-
sync_membership_on_user_login = false
93+
group_name = each.value.name
94+
sync_membership_on_user_login = each.value.sync_membership_on_user_login
10195
}
10296

103-
# TODO: Assign roles and resource roles to the group
97+
# TODO: (When supported) Assign roles and resource roles to the group
10498

105-
# TODO: Assign users to the group
99+
# TODO: (When supported) Assign users to the group
106100

107101
# ------- IdBroker Mappings -------
108102
resource "cdp_environments_id_broker_mappings" "cdp_idbroker" {
@@ -113,17 +107,11 @@ resource "cdp_environments_id_broker_mappings" "cdp_idbroker" {
113107
data_access_role = var.datalake_admin_role_arn
114108
ranger_cloud_access_authorizer_role = var.enable_raz ? var.raz_role_arn : null
115109

116-
mappings = [{
117-
accessor_crn = cdp_iam_group.cdp_admin_group.crn
118-
role = var.datalake_admin_role_arn
119-
},
120-
{
121-
accessor_crn = cdp_iam_group.cdp_user_group.crn
122-
role = var.datalake_admin_role_arn
123-
}
124-
]
110+
mappings = local.cdp_group_id_broker_mappings
111+
set_empty_mappings = length(local.cdp_group_id_broker_mappings) == 0 ? true : null
125112

126113
depends_on = [
114+
cdp_iam_group.cdp_groups,
127115
cdp_environments_aws_environment.cdp_env
128116
]
129117
}

‎modules/terraform-cdp-deploy/modules/aws/outputs.tf

+7
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,10 @@ output "cdp_environment_crn" {
2323

2424
description = "CDP Environment CRN"
2525
}
26+
27+
output "cdp_groups" {
28+
29+
value = data.cdp_iam_group.cdp_groups
30+
31+
description = "CDP Group information"
32+
}

‎modules/terraform-cdp-deploy/modules/aws/variables.tf

+9-9
Original file line numberDiff line numberDiff line change
@@ -56,16 +56,16 @@ variable "cdp_xacccount_credential_name" {
5656

5757
}
5858

59-
variable "cdp_admin_group_name" {
60-
type = string
61-
description = "Name of the CDP IAM Admin Group associated with the environment."
62-
63-
}
64-
65-
variable "cdp_user_group_name" {
66-
type = string
67-
description = "Name of the CDP IAM User Group associated with the environment."
59+
variable "cdp_groups" {
60+
type = set(object({
61+
name = string
62+
create_group = bool
63+
sync_membership_on_user_login = optional(bool)
64+
add_id_broker_mappings = bool
65+
})
66+
)
6867

68+
description = "List of CDP Groups to be added to the id broker of the environment. If create_group is set to true then the group will be created."
6969
}
7070

7171
variable "enable_ccm_tunnel" {

‎modules/terraform-cdp-deploy/modules/azure/data.tf

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Copyright 2025 Cloudera, Inc. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
data "cdp_iam_group" "cdp_groups" {
16+
17+
for_each = {
18+
for k, v in coalesce(var.cdp_groups, []) : k.name => v
19+
}
20+
21+
group_name = each.value.name
22+
23+
depends_on = [cdp_iam_group.cdp_groups]
24+
}
25+

‎modules/terraform-cdp-deploy/modules/azure/defaults.tf

+10
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,14 @@ locals {
1919
var.cdp_xacccount_credential_name :
2020
cdp_environments_azure_credential.cdp_cred[0].credential_name
2121
)
22+
23+
# Construct IDBroker mappings
24+
cdp_group_id_broker_mappings = [
25+
for grp, grp_details in coalesce(var.cdp_groups, []) :
26+
{
27+
accessor_crn = data.cdp_iam_group.cdp_groups[grp_details.name].crn
28+
role = var.datalakeadmin_identity_id
29+
}
30+
if try(grp_details.add_id_broker_mappings, false)
31+
]
2232
}

0 commit comments

Comments
 (0)