Skip to content

docs: document schema registry connectivity over private endpoints #104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mintlify wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: document schema registry connectivity over private endpoints #104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions integrations/clickpipes/aws-privatelink.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ data source types:
- MySQL
- MongoDB

<Note>
For Kafka, the [schema registry](/integrations/clickpipes/kafka/schema-registries) can be reached over the same reverse private endpoint as the brokers — it does not need to be publicly accessible — provided its hostname resolves to the endpoint's private IP addresses.
</Note>

## Supported AWS PrivateLink endpoint types {#aws-privatelink-endpoint-types}

ClickPipes reverse private endpoint can be configured with one of the following AWS PrivateLink approaches:
Expand Down
11 changes: 11 additions & 0 deletions integrations/clickpipes/kafka/schema-registries.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,17 @@ To integrate with a schema registry during ClickPipes configuration, you must us
2. Provide a complete path to the schema ID (e.g. `https://registry.example.com/schemas/ids/1000`)
3. Provide the root schema registry URL (e.g. `https://registry.example.com`)

## Network connectivity {#network-connectivity}

ClickPipes connects to the schema registry over HTTPS at the URL you provide. The schema registry does not need to be publicly accessible.

If your Kafka brokers are reached through a [reverse private endpoint](/integrations/clickpipes/aws-privatelink) (AWS PrivateLink or GCP Private Service Connect), the schema registry can use the same private connectivity. ClickPipes resolves the registry hostname through the reverse private endpoint's private DNS, so a registry hosted privately alongside your brokers is reachable as long as its hostname resolves to the reverse private endpoint's private IP addresses (via the endpoint's private DNS support or a custom private DNS mapping).

Keep the following in mind:

- The schema registry URL must use `https://`.
- If the registry hostname resolves to a private address, it must be reachable through a reverse private endpoint selected for the ClickPipe; otherwise the connectivity check during setup will fail.

## How it works {#how-schema-registries-work}

ClickPipes dynamically retrieves and applies the schema from the configured schema registry.
Expand Down