Skip to content
3 changes: 3 additions & 0 deletions operations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ This is the README for Ops-files. To learn more about `cf-deployment`, go to the
| [`enable-cc-worker-metrics.yml`](enable-cc-worker-metrics.yml) | Enable metrics for cc-workers. | This will setup the metrics endpoint, configure the prom_scraper job and uses mTLS. If you want to use another scraper than prom_scraper you can additionally set `cc.prom_scraper.disabled` to true. | **NO** |
| [`enable-scheduler-metrics.yml`](enable-scheduler-metrics.yml) | Enable metrics for schedulers. | This will setup the metrics endpoint, configure the prom_scraper job and uses mTLS. If you want to use another scraper than prom_scraper you can additionally set `cc.prom_scraper.disabled` to true. | **NO** |
| [`enable-cpu-throttling.yml`](enable-cpu-throttling.yml) | Configure Garden containers with CPU entitlement. | This ops file requires `set-cpu-weight.yml`. | **YES** |
| [`enable-identity-aware-routing.yml`](enable-identity-aware-routing.yml) | Enables identity-aware routing with mTLS domains. | Creates a new self-signed CA (`apps_identity_ca`) for identity routing on `*.apps.identity` domain. Generates router TLS certificate and configures gorouter and Diego to trust the CA. | **NO** |
| [`enable-identity-aware-routing-cflinuxfs5.yml`](enable-identity-aware-routing-cflinuxfs5.yml) | Enables identity-aware routing for cflinuxfs5 app containers. | Requires `experimental/add-cflinuxfs5.yml` and `enable-identity-aware-routing.yml` to be applied first. | **NO** |
| [`enable-nfs-ldap.yml`](enable-nfs-ldap.yml) | Enables LDAP authentication for NFS volume services | Requires `enable-nfs-volume-service.yml`. Introduces [new variables](example-vars-files/vars-enable-nfs-ldap.yml) | **NO** |
| [`enable-nfs-volume-service.yml`](enable-nfs-volume-service.yml) | Enables volume support and deploys an NFS broker and volume driver | As of cf-deployment v2, you must use the `nfsbrokerpush` errand to cf push the nfs broker after `bosh deploy` completes. | **YES** |
| [`enable-privileged-container-support.yml`](enable-privileged-container-support.yml) | Enables Diego privileged container support. | | **NO** |
Expand Down Expand Up @@ -76,6 +78,7 @@ This is the README for Ops-files. To learn more about `cf-deployment`, go to the
| [`use-latest-stemcell.yml`](use-latest-stemcell.yml) | Use the latest stemcell available on your BOSH director instead of the one in `cf-deployment.yml`. **Caution**: This ops-file should not be used in conjunction with `use-compiled-releases.yml`, since the latter relies on a specific stemcell version being used. | | **NO** |
| [`use-latest-windows2019-stemcell.yml`](use-latest-windows2019-stemcell.yml) | Use the latest `windows2019` stemcell available on your BOSH director instead of the one in `windows2019-cell.yml` | Requires `windows2019-cell.yml` | **NO** |
| [`use-metric-store.yml`](use-metric-store.yml) | Adds a single-node metric store. | | **NO** |
| [`use-operator-provided-identity-routing-domain.yml`](use-operator-provided-identity-routing-domain.yml) | Allows operators to provide a custom domain for identity-aware routing. | Requires `enable-identity-aware-routing.yml`. Introduces [new variables](example-vars-files/vars-use-operator-provided-identity-routing-domain.yml) for the custom identity routing domain. | **NO** |
| [`use-operator-provided-router-tls-certificates.yml`](use-operator-provided-router-tls-certificates.yml) | Allows an operator to provide their own certificates for the gorouter by providing variables [`router_ssl_pem`](example-vars-files/vars-use-operator-provided-router-tls-certificates.yml) | This is required if using AWS Network Load Balancers. | **YES** |
| [`use-postgres.yml`](use-postgres.yml) | Replaces the MySQL instance group with a postgres instance group. **Warning**: this will lead to total data loss if applied to an existing deployment with MySQL or removed from an existing deployment with postgres. | If combined with [`use-compiled-releases.yml`](use-compiled-releases.yml), you must apply this ops file **after** `use-compiled-releases.yml`. | **YES** |
| [`use-trusted-ca-cert-for-apps.yml`](use-trusted-ca-cert-for-apps.yml) | Injects the CA specified with `trusted_cert_for_apps` into the Diego `rep` job's trust store and cf-deployment's default root filesystem | Applications that *explicitly* look in the canonical location (`/etc/cf-system-certificates`) will trust certificates signed by the given CA, regardless of filesystem. Applications that use the default root filesystem will trust certificates signed by the given CA *implicitly*. <br/> Please see the [documentation](https://docs.cloudfoundry.org/running/trusted-system-certificates.html) for information about configuring additional trusted CA certificates. | **NO** |
Expand Down
15 changes: 15 additions & 0 deletions operations/enable-identity-aware-routing-cflinuxfs5.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
# Trust apps_identity_ca in cflinuxfs5 app containers.
# Kept separate from enable-identity-aware-routing.yml because cflinuxfs5-rootfs-setup
# is only present when add-cflinuxfs5.yml has been applied; merging into the main
# ops-file would break deployments that don't use cflinuxfs5.
#
# Prerequisites (apply in this order):
# 1. operations/experimental/add-cflinuxfs5.yml
# 2. operations/enable-identity-aware-routing.yml
# 3. this file
# Note: no '?' on the path — a hard BOSH error is the intended signal if
# add-cflinuxfs5.yml was not applied first.
- type: replace
path: /instance_groups/name=diego-cell/jobs/name=cflinuxfs5-rootfs-setup/properties/cflinuxfs5-rootfs/trusted_certs/-
value: ((apps_identity_ca.certificate))
68 changes: 68 additions & 0 deletions operations/enable-identity-aware-routing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
# BOSH DNS alias: *.apps.identity -> router instances
# (BOSH DNS uses underscore prefix for wildcards; _.apps.identity matches *.apps.identity)
- type: replace
path: /addons/name=bosh-dns-aliases/jobs/name=bosh-dns-aliases/properties/aliases/-
value:
domain: _.apps.identity
targets:
- deployment: cf
domain: bosh
instance_group: router
network: default
query: '*'

# Configure mTLS domain on gorouter.
# Gorouter verifies client certs against diego_instance_identity_ca (existing CF PKI).
# sanitize_set: strips any incoming XFCC header (prevents spoofing) and sets a fresh one from the verified client cert.
# xfcc_format: envoy format is required by Diego/Envoy sidecars consuming the header.
- type: replace
path: /instance_groups/name=router/jobs/name=gorouter/properties/router/domains?
value:
- name: "*.apps.identity"
ca_certs: ((diego_instance_identity_ca.certificate))
forwarded_client_cert: sanitize_set
xfcc_format: envoy

# Add *.apps.identity TLS cert to gorouter (served via SNI)
- type: replace
path: /instance_groups/name=router/jobs/name=gorouter/properties/router/tls_pem/-
value:
cert_chain: ((apps_identity_router_tls.certificate))
private_key: ((apps_identity_router_tls.private_key))

# Trust apps_identity_ca in cflinuxfs4 app containers so outbound TLS to
# *.apps.identity succeeds.
- type: replace
path: /instance_groups/name=diego-cell/jobs/name=cflinuxfs4-rootfs-setup/properties/cflinuxfs4-rootfs/trusted_certs/-
value: ((apps_identity_ca.certificate))

# Trust apps_identity_ca in the Diego rep container trust store (Envoy sidecars).
- type: replace
path: /instance_groups/name=diego-cell/jobs/name=rep/properties/containers/trusted_ca_certificates/-
value: ((apps_identity_ca.certificate))

# Generate apps_identity_ca: self-signed CA that issues the router SNI cert and is
# trusted by app containers.
- type: replace
path: /variables/-
value:
name: apps_identity_ca
type: certificate
options:
common_name: apps-identity-ca
is_ca: true

# Generate apps_identity_router_tls: server cert for *.apps.identity served via SNI.
- type: replace
path: /variables/-
value:
name: apps_identity_router_tls
type: certificate
options:
ca: apps_identity_ca
common_name: "*.apps.identity"
alternative_names:
- "*.apps.identity"
extended_key_usage:
- server_auth
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
identity_routing_domain: apps.example.com
28 changes: 28 additions & 0 deletions operations/use-operator-provided-identity-routing-domain.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
# Override the hardcoded apps.identity domain set by enable-identity-aware-routing.yml.
# Apply on top of that ops-file:
#
# bosh deploy cf-deployment.yml \
# -o operations/enable-identity-aware-routing.yml \
# -o operations/use-operator-provided-identity-routing-domain.yml \
# -v identity_routing_domain=apps.example.com

# Override BOSH DNS alias domain.
- type: replace
path: /addons/name=bosh-dns-aliases/jobs/name=bosh-dns-aliases/properties/aliases/domain=_.apps.identity/domain
value: "_.((identity_routing_domain))"

# Override gorouter mTLS domain name.
- type: replace
path: /instance_groups/name=router/jobs/name=gorouter/properties/router/domains/name=*.apps.identity/name
value: "*.((identity_routing_domain))"

# Override TLS cert common_name.
- type: replace
path: /variables/name=apps_identity_router_tls/options/common_name
value: "*.((identity_routing_domain))"

# Override TLS cert SAN (first and only entry).
- type: replace
path: /variables/name=apps_identity_router_tls/options/alternative_names/0
value: "*.((identity_routing_domain))"
3 changes: 3 additions & 0 deletions units/tests/semantic_test/semantic_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,9 @@ func TestSemantic(t *testing.T) {
if ca == "((diego_instance_identity_ca.certificate))" {
continue
}
if ca == "((apps_identity_ca.certificate))" {
continue
}

t.Errorf("CAs should be referenced from their certificate variables: %s in %s", ca, strings.Replace(path, operationsSubDirectory, "operations", 1))
}
Expand Down
12 changes: 12 additions & 0 deletions units/tests/standard_test/operations.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ enable-cc-v2-rate-limiting.yml:
enable-cc-worker-metrics.yml: {}
enable-scheduler-metrics.yml: {}
enable-cpu-throttling.yml: {}
enable-identity-aware-routing.yml: {}
enable-identity-aware-routing-cflinuxfs5.yml:
ops:
- experimental/add-cflinuxfs5.yml
- enable-identity-aware-routing.yml
- enable-identity-aware-routing-cflinuxfs5.yml
enable-nfs-ldap.yml:
ops:
- enable-nfs-volume-service.yml
Expand Down Expand Up @@ -135,6 +141,12 @@ use-online-windows2019fs.yml:
ops:
- windows2019-cell.yml
- use-online-windows2019fs.yml
use-operator-provided-identity-routing-domain.yml:
ops:
- enable-identity-aware-routing.yml
- use-operator-provided-identity-routing-domain.yml
varsfiles:
- example-vars-files/vars-use-operator-provided-identity-routing-domain.yml
use-operator-provided-router-tls-certificates.yml:
varsfiles:
- example-vars-files/vars-use-operator-provided-router-tls-certificates.yml
Expand Down