Skip to content

Add data.credentials support to Buildpack and CNB lifecycles - #5547

Merged
johha merged 2 commits into
mainfrom
worktree-lifecycle-credentials-consistency
Oct 20, 2026
Merged

johha merged 2 commits into
mainfrom
worktree-lifecycle-credentials-consistency

Conversation

@johha

@johha johha commented Oct 14, 2026

Copy link
Copy Markdown
Member

Adds support for data.credentials to the buildpack lifecycle and unifies how credentials are provided across lifecycles. Previously only the CNB lifecycle had a data.credentials field; buildpack apps could only authenticate to private buildpacks by embedding credentials in the buildpack URL. This change makes credential handling consistent between the two lifecycles.

Specifically:

  • Buildpack lifecycle data can now store credentials (encrypted at rest, in new encrypted_credentials_json / _salt columns on buildpack_lifecycle_data).
  • The manifest gains a unified top-level credentials key that works for both buildpack and CNB apps. The existing cnb-credentials key is kept as a backward-compatible alias. Specifying both is a validation error.
  • For the buildpack lifecycle, credentials are injected into the custom buildpack URL (both the url and key entry fields) so the staging git clone - which is driven by key via -buildpackOrder - authenticates. CNB continues to receive credentials only via CNB_REGISTRY_CREDS; its buildpack URL stays clean.
  • XOR validation: credentials embedded in a buildpack URL cannot be combined with data.credentials. When the conflict is with already-stored credentials (not supplied in the same request), the error explains how to clear them.
  • An explicit credentials: null clears stored credentials; omitting the key leaves them untouched (consistent with the additive manifest contract).
  • Credentials are redacted everywhere they surface: GET /v3/apps/:guid lifecycle data, the generated app manifest, the space manifest diff (both the credentials keys and userinfo embedded in buildpack URLs, singular andplural), and audit events.

Example - buildpack app with a private git buildpack:

# manifest.yml
applications:
- name: my-app
  lifecycle: buildpack
  buildpacks:
  - https://github.com/acme/private-buildpack   # no credentials in the URL
  credentials:
    github.com:
      username: acme-bot
      password: <token>

CC resolves the credentials at staging and injects them into the buildpack entry so the clone authenticates (pseudocode):

# BuildpackEntryGenerator#buildpack_entries (custom buildpack branch)
url = inject_credentials(buildpack_info.buildpack_url, credentials)
# inject_credentials: look up credentials[URI.parse(url).host], and if an
# entry with a username exists, set url userinfo -> https://user:pass@host/...
{ name: 'custom', key: url, url: url, skip_detect: true }
# Diego builds -buildpackOrder from `key`, so the authenticated URL is used.

Example - clearing stored credentials:

applications:
- name: my-app
  credentials: null   # explicit null clears; omitting the key keeps them
  • An explanation of the use cases your change solves

Operators and developers can authenticate to private buildpacks (buildpack lifecycle) and private buildpack registries/images (CNB lifecycle) using a single, consistent manifest/API field instead of embedding secrets in buildpack URLs. This smooths and unifies the API ahead of further lifecycle work, and avoids credentials leaking into URLs that previously had to carry them inline.

  • I have reviewed the contributing guide

  • I have viewed, signed, and submitted the Contributor License Agreement

  • I have made this pull request to the main branch

  • I have run all the unit tests using bundle exec rake

  • I have run CF Acceptance Tests

@johha
johha force-pushed the worktree-lifecycle-credentials-consistency branch 4 times, most recently from 6f9fd9f to 4568e76 Compare October 14, 2026 12:44
Makes credential handling consistent across lifecycles by introducing
data.credentials for both buildpack and CNB lifecycle types, stored
encrypted in the buildpack_lifecycle_data table.

- New migration adds encrypted_credentials_json / _salt columns to
  buildpack_lifecycle_data (previously CNB-only via cnb_lifecycle_data)
- BuildpackLifecycleDataModel gains credentials accessors backed by
  set_field_as_encrypted; to_hash redacts the value
- AppBaseLifecycle#update_lifecycle_data_model persists requested lifecycle
  fields (including credentials, honoring an explicit null to clear); staging
  reads credentials via LifecycleBase#credentials_for_staging
- Buildpack lifecycle injects credentials into the custom buildpack URL for
  both the url and key fields, so the staging git clone (driven by key via
  -buildpackOrder) authenticates; CNB does not inject into the URL and relies
  solely on CNB_REGISTRY_CREDS
- Validators enforce XOR between URL-embedded credentials and data.credentials,
  with a clearer message when the conflicting credentials are already stored
- Manifest: unified top-level credentials key for both lifecycles, with
  cnb-credentials kept as a backward-compatible alias; specifying both is a
  validation error; an explicit null clears stored credentials
- Redaction: audit events, the app manifest output, and the manifest diff all
  mask credentials and buildpack-URL userinfo (singular and plural keys)
- url_has_userinfo? detects password-only userinfo (:token@host)
- Docs: document data.credentials for both lifecycles and the manifest
  credentials / cnb-credentials keys
@johha
johha force-pushed the worktree-lifecycle-credentials-consistency branch from 4568e76 to 83f7460 Compare October 14, 2026 13:01
@johha
johha marked this pull request as ready for review October 14, 2026 13:35
- inject_credentials: normalize credentials with deep_stringify_keys so
  request-sourced (symbol-keyed) credentials are matched, not just stored
  (string-keyed) ones; a POST /v3/builds with inline lifecycle.data.credentials
  was otherwise silently skipped
- inject_credentials: use CGI.escape for userinfo, matching the codebase
  convention (config.rb#escape_userinfo)
- migration: size encrypted_credentials_json at 16_000 (was unsized ->
  varchar(255) on MySQL, too small for an encrypted multi-host credentials map),
  and grow the existing CNB encrypted_registry_credentials_json to match
@johha
johha requested a review from philippthun October 20, 2026 10:27
@johha johha changed the title feat: add data.credentials support to Buildpack and CNB lifecycles Add data.credentials support to Buildpack and CNB lifecycles Oct 20, 2026
@johha
johha merged commit f29bf1a into main Oct 20, 2026
17 checks passed
@johha
johha deleted the worktree-lifecycle-credentials-consistency branch October 20, 2026 16:38
ari-wg-gitbot added a commit to cloudfoundry/capi-release that referenced this pull request Oct 20, 2026
Changes in cloud_controller_ng:

- Add data.credentials support to Buildpack and CNB lifecycles
    PR: cloudfoundry/cloud_controller_ng#5547
    Author: Johannes Haass <johannes.haass@sap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants