Skip to content

Commit 9bd2272

Browse files
authored
Merge pull request #6945 from segmentio/dbt-git-extensions-ga
Extensions dbt and Git Sync GA [DOC-983]
2 parents 407ec51 + 4de9e1c commit 9bd2272

File tree

3 files changed

+76
-24
lines changed

3 files changed

+76
-24
lines changed

src/segment-app/extensions/dbt.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,38 +6,37 @@ Segment's dbt extension lets you use [Reverse ETL](/docs/connections/reverse-etl
66

77
With Segment's dbt extension, you can:
88

9-
- Securely connect Segment to the GitHub repository that stores your dbt models.
9+
- Securely connect Segment to a Git repository that stores your dbt models.
1010
- Use centralized dbt models to set up Reverse ETL.
1111
- Trigger Reverse ETL syncs from dbt jobs.
1212

1313
This page explains how to set up a dbt Model and then use the model with Reverse ETL.
1414

15-
> info ""
16-
> Extensions, including dbt sync, is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
17-
1815
## Before you begin
1916

2017
Keep the following in mind as you set up the dbt extension:
2118

2219
- The extension supports [dbt Core v1.7](https://docs.getdbt.com/docs/dbt-versions/core-upgrade/upgrading-to-v1.7){:target="_blank"}.
2320
- You can use [Snowflake](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/snowflake-setup/), [Databricks](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/databricks-setup/), [Redshift](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/redshift-setup/), [Postgres](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/postgres-setup/), and [BigQuery](/docs/connections/reverse-etl/reverse-etl-source-setup-guides/bigquery-setup/) as Reverse ETL sources.
24-
- dbt models aren't synchronized from the dbt cloud. The model sync connects to a Git repository that loads models into Segment for use with Reverse ETL.
21+
- dbt models aren't synchronized from the dbt cloud. The model sync connects to a Git repository that loads models into Segment for use with Reverse ETL.
22+
- You can connect to GitHub using a GitHub App, token, or SSH.
23+
- For [GitLab](https://docs.gitlab.com/ee/user/ssh.html){:target="_blank"} and [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/){:target="_blank"}, use SSH to connect.
2524

2625
## Set up Git dbt Models and dbt Cloud
2726

2827
To set up the dbt extension, you'll need:
2928

3029
- an existing dbt account with a Git repository
31-
- (for job syncs:) dbt cloud with jobs already created
30+
- for job syncs, dbt cloud with jobs already created
3231

3332
### Git repository and dbt Models setup
3433

3534
Follow these steps to connect the Git repository that stores your dbt Models:
3635

3736
1. In your Segment workspace, navigate to **Settings > Extensions**.
3837
2. Click **Set up Git sync**.
39-
3. On the **Configure service credentials** page, select a service and protocol, add your SSH private key or GitHub token, then click **Next**.
40-
4. In the **Connect source** window, select a Reverse ETL warehouse source from the dropdown, then click **Save**.
38+
3. On the **Configure service credentials** page, select a service and protocol, add your GitHub App, SSH private key or GitHub token, then click **Next**.
39+
4. In the **Connect source** window, select an existing Reverse ETL warehouse source from the dropdown, then click **Save**.
4140

4241
After you've saved your setup, you can configure your Git repository's settings to your needs by changing the repository, branch, dbt version, default schema, and project path.
4342

src/segment-app/extensions/git.md

Lines changed: 64 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,77 @@
22
title: Git Sync Extension
33
---
44

5-
Segment's Git extension lets you manage versioning by syncing changes you make in Sources and Destinations from your Segment workspace to a Git repository.
5+
Segment's Git extension lets you manage versioning by syncing changes you make in your Segment workspace to a Git repository.
66

7-
> info ""
8-
> Extensions, including Git sync, is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
7+
Git Sync supports one-way synchronization from Segment to Git. This sync captures the current state of your workspace through a full sync and includes all new records and changes for supported resources.
98

10-
## Set up Git sync
9+
Segment doesn't support syncing changes from Git back to Segment.
1110

12-
Follow these steps to set up Git sync:
11+
## Set up Git Sync
12+
13+
Follow these steps to set up Git Sync:
1314

1415
1. In your Segment workspace, navigate to **Settings > Extensions**.
1516
2. Click **Set up Git sync**.
16-
3. On the **Configure service credentials** page, select a service and protocol, add your SSH private key or GitHub token, then click **Next**.
17+
3. On the **Configure service credentials** page, select a service and protocol, add your GitHub App, SSH private key, or GitHub token, then click **Next**.
18+
- To connect to GitLab or Bitbucket, use your SSH private key.
19+
20+
## Working with Git Sync
21+
22+
The Git sync extension syncs the following resources from Segment to your Git repository:
23+
24+
- [Sources](/docs/connections/sources/) and [Destinations](/docs/connections/destinations/)
25+
- [Warehouses](/docs/connections/storage/warehouses/)
26+
- [Destination Filters and Mappings](/docs/connections/destinations/destination-filters/)
27+
- [Tracking Plans](/docs/protocols/tracking-plan/create/)
28+
- [Functions](/docs/connections/functions/)
29+
- [Transformations](/docs/protocols/transform/)
30+
- [Reverse ETL](/docs/connections/reverse-etl/)
31+
- [Users](/docs/segment-app/iam/concepts/#team-members) and [User groups](/docs/segment-app/iam/concepts/#user-groups)
32+
- [Labels](/docs/segment-app/iam/labels/#where-can-i-create-labels)
33+
34+
Reach out to [Segment support](https://app.segment.com/workspaces?contact=1){:target="blank"} to request support for additional Git Sync resources.
35+
36+
After you set up the Git sync extension for the first time, Segment performs an initial sync that sends the current state of your Segment workspace to the Git repository you connected. Segment automatically tracks all following workspace updates.
37+
38+
You can manually trigger syncs at any time by clicking **Full Sync** on the Git Sync page. To disable Git Sync from the Git Sync page, switch the **Enabled** toggle to off.
39+
40+
## Git Sync architecture and data model
41+
42+
Because a Segment workspace can represent a distinct environment (testing, staging, production), each workspace is mapped directly to a single Git repository. This direct mapping ensures a clear and organized relationship between workspace resources and a Git repository.
43+
44+
Segment uses its [Terraform provider](https://registry.terraform.io/providers/segmentio/segment/1.0.3){:target="_blank"} to manage key functions like tracking changes and retrieving information about those changes in Segment. Segment stores changes in HashiCorp Configuration Language (HCL), the format used by Terraform. To learn more about HCL and how it compares to JSON or YAML, visit [HashiCorp's HCL repository on GitHub](https://github.com/hashicorp/hcl){:target="_blank"}.
45+
46+
Using HCL makes it easier to document Segment's data model, especially for users managing versioning and Git Sync with Terraform. It also helps manage Segment configurations directly from Git. For more details on the Git Sync data model, read [Segment's Terraform provider documentation](https://registry.terraform.io/providers/segmentio/segment/latest/docs){:target="_blank"}.
47+
48+
## Managing your Segment workspace with Terraform and Git Sync
49+
50+
Segment supports one-way synchronization from Segment to Git, but you can set up two-way synchronization using the Segment Terraform provider.
51+
52+
Terraform offers an open-source way to manage Segment resources through a Git repository as an alternative to a fully managed two-way sync. However, this method requires third-party tools like [Atlantis](https://www.runatlantis.io/){:target="_blank"} for CI integration, which Segment doesn’t officially support.
53+
54+
To manage Segment resources using Git and Terraform, follow these steps:
55+
56+
1. Copy the generated Terraform configuration for the resources you want to manage into a separate Git repository dedicated to Terraform.
57+
2. Include the following provider configuration blocks:
58+
59+
```hcl
60+
# providers.tf
1761
18-
## Working with Git syncs
62+
terraform {
63+
required_providers {
64+
segment = {
65+
source = "segmentio/segment"
66+
version = "1.0.4"
67+
}
68+
}
69+
}
1970
20-
The Git sync extension syncs [Connections](/docs/connections/) ([Sources](/docs/connections/sources/) and [Destinations](/docs/connections/destinations/)) from Segment to your Git repository.
71+
provider "segment" {
72+
# Provide the token directly or load it from an environment variable
73+
}
74+
```
75+
3. Apply configuration changes by running Terraform locally or using a tool like Atlantis to run it directly from your Git provider.
2176
22-
After you set up the Git sync extension for the first time, Segment performs an initial sync that sends source and destination information in your Segment workspace to the Git repository you connected.
2377
24-
You can run syncs at any time by clicking **Full resync** on the Git sync page. To disable Git sync from the Git sync page, switch the **Enabled** toggle to off.
78+
For more information on using Terraform, visit [Terraform's documentation](https://developer.hashicorp.com/terraform/docs){:target="_blank"}.

src/segment-app/extensions/index.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,11 @@
22
title: Extensions
33
---
44

5-
Extensions integrate third-party tools into your existing Segment workspace to help you automate tasks.
5+
Extensions let you integrate third-party tools into your existing Segment workspace, helping you automate tasks, manage data flows, and maintain version control.
66

7-
> info ""
8-
> Extensions is currently in public beta and is governed by Segment's [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}. During Public Beta, Extensions is available for Team and Developer plans only. [Reach out to Segment](mailto:[email protected]) if you're on a Business Tier plan and would like to participate in the Public Beta.
7+
Segment offers the following extensions:
98

10-
During public beta, Segment offers the following extensions:
9+
- [dbt models and dbt Cloud](/docs/segment-app/extensions/dbt): Sync your dbt Labs models with Segment to streamline model management, versioning, and CI checks. This extension lets you securely connect Segment to a Git repository, making it easier to integrate and manage dbt models across different environments like testing, staging, and production.
10+
- [Git Sync](/docs/segment-app/extensions/git): Manage versioning and track changes by syncing your Segment workspace a Git repository. The Git Sync extension helps maintain a clear and organized relationship between your workspace and its corresponding Git repository, ensuring that your resources are consistently managed and versioned across your environments.
1111

12-
- [dbt models and dbt Cloud](/docs/segment-app/extensions/dbt), which lets you sync models with dbt Labs.
13-
- [Git sync](/docs/segment-app/extensions/git), which you can use to manage versioning and track changes in your Segment workspace.
12+
Segment built Extensions to help you get the most out of your Segment workspace, allowing you to keep your projects organized, efficient, and aligned with best practices for data management and version control.

0 commit comments

Comments
 (0)