Implement SecretsMasker for Variables and Connections in the Go SDK

[jason810496]

Dependency

Wait until both #67317 and #67318 merge. #67317 lands the coordinator-side GetVariable / GetConnection TODO sites; #67318 lands the pkg/worker/runner.go task log handler that the masker plugs into. Until both are on main, the consumer-side hook this feature needs does not exist.

Background

Python's airflow.models.variable.Variable.get and airflow.models.connection.Connection.get register sensitive values with a SecretsMasker so any subsequent task log line that contains the secret is automatically redacted. The Go SDK currently does not — secrets retrieved via GetVariable / GetConnection flow straight into task logs unredacted.

The gap is documented as four coordinated TODOs across the SDK and the runtime. None of them is useful in isolation: log redaction needs both a producer (the lookup site registering the secret) and a consumer (the log handler running the redaction). Until all four land, the SDK ships without log-side masking parity.

HTTP-backed client (already on main):

// go-sdk/sdk/client.go - GetVariable
// TODO: register secret-named variables with a SecretsMasker so the
// returned value is automatically redacted from subsequent task logs,
// matching Python's airflow.models.variable.Variable.get behaviour.
// Pairs with the "TODO: mask secrets here" hook in
// pkg/worker/runner.go's task log handler — both halves are needed
// before secret masking actually works end-to-end.

// go-sdk/sdk/client.go - GetConnection
// TODO: register Connection.Password and sensitive-keyed entries of
// Connection.Extra with a SecretsMasker so they are auto-redacted from
// subsequent task logs, matching Python's
// airflow.models.connection.Connection.get behaviour. Pairs with the
// "TODO: mask secrets here" hook in pkg/worker/runner.go's task log
// handler and the matching TODO on GetVariable above.

Coordinator-mode client (lands with #67317):

// go-sdk/pkg/execution/client.go - GetVariable and GetConnection
// (same TODO text, same SecretsMasker hook)

Log-handler hook (lands with #67318): the // TODO: mask secrets here comment in go-sdk/pkg/worker/runner.go is where the consumer side of the redaction belongs.

What needs to happen

1. Design and add a SecretsMasker type to the Go SDK with a register-and-redact API roughly equivalent to airflow.utils.log.secrets_masker.SecretsMasker (case-insensitive secret-name detection, replace-with-*** semantics, per-task scoping).
2. Register secret-named variables in both sdk.client.GetVariable and execution.CoordinatorClient.GetVariable.
3. Register Connection.Password and sensitive-keyed entries of Connection.Extra in both sdk.client.GetConnection and execution.CoordinatorClient.GetConnection.
4. Wire the masker into the task log handler at the // TODO: mask secrets here site in pkg/worker/runner.go so emitted log lines are redacted before they leave the runtime.
5. Remove the four coordinated TODOs above.

Acceptance criteria

• Looking up a variable named e.g. my_password via either sdk.client or CoordinatorClient causes any subsequent task log line containing the variable's value to be redacted to *** (or the project's chosen sentinel).
• Connection.Password and sensitive-keyed Connection.Extra entries produce the same redaction behaviour from both clients.
• Tests cover: (a) registration from each client, (b) redaction in the log handler, (c) negative case where a non-sensitive variable name is not registered.
• All four TODOs cross-referencing this work are removed from the codebase.

Context

• Originating review: self-review thread on Go-SDK: Add concurrent-safe coordinator comms, log handler, and client #67317 (coordinator comms + client PR).
• HTTP-side TODOs (already on main): go-sdk/sdk/client.go GetVariable and GetConnection.
• Coordinator-side TODOs (land with Go-SDK: Add concurrent-safe coordinator comms, log handler, and client #67317): go-sdk/pkg/execution/client.go GetVariable and GetConnection.
• Log-handler hook (lands with Go-SDK: Implement coordinator-mode runtime entry point and task runner #67318): go-sdk/pkg/worker/runner.go.
• Python parity: airflow.utils.log.secrets_masker.SecretsMasker, airflow.models.variable.Variable.get, airflow.models.connection.Connection.get.
• Related project board: AIP-72 (addendum): Go-SDK (view)

---

Drafted-by: Claude Code (Opus 4.7) (no human review before posting)

[uranusjr]

I think instead of re-implementing this in each language, we should move secrets and secret masking to the supervisor, so it’s implemented only once in Python. This is probably not strictly required yet, so we can move this beyond 3.3. Keeping this here for now so we remember to make a decision.