Skip to content

Commit 65cd26d

Browse files
pwsegpwseg
authored
Merge pull request #6929 from segmentio/pwseg-patch-2
Snowflake Setup Updates
2 parents 3885e02 + 7961436 commit 65cd26d

File tree

1 file changed

+73
-86
lines changed

1 file changed

+73
-86
lines changed

‎src/unify/data-graph/setup-guides/snowflake-setup.md

Lines changed: 73 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,68 @@
11
---
22
title: Snowflake Setup
33
plan: unify
4-
beta: true
5-
hidden: true
64
redirect_from:
75
- '/unify/linked-profiles/setup-guides/snowflake-setup'
86
---
7+
> warning ""
8+
> Data Graph, Reverse ETL, Profiles Sync require different warehouse permissions.
99
10-
> info "Linked Audiences is in public beta"
11-
> Linked Audiences (with Data Graph, Linked Events) is in public beta, and Segment is actively working on this feature. Some functionality may change before it becomes generally available.
12-
13-
On this page, you'll learn how to connect your Snowflake data warehouse to Segment.
14-
15-
Log in to Snowflake with admin privileges to provide Segment Data Graph with the necessary permissions below.
16-
17-
18-
## Required connection settings within Segment
19-
20-
Segment requires the following settings to connect to your Snowflake warehouse.
21-
22-
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to Data Graph" width="5888"/>
23-
24-
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
25-
- **Database**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
26-
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
27-
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
28-
- **Authentication**: There are 2 supported authentication methods:
29-
1. **Key Pair**: This is the recommended method of authentication. You would need to first create the user and assign it a key pair following the instructions in the [Snowflake docs](https://docs.snowflake.com/en/user-guide/key-pair-auth). Then, follow the Segment docs above to set up Snowflake permissions and set the `segment_connections_username` variable in the SQL script to the user you just created.
30-
2. **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
10+
On this page, you'll learn how to connect your Snowflake data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/data-graph/).
3111

32-
## Set up Snowflake credentials
12+
## Snowflake credentials
3313

34-
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas.
14+
Segment assumes that you already have a warehouse that includes the datasets you'd like to use for the Data Graph. Log in to Snowflake with admin privileges to provide the Data Graph with the necessary permissions below.
3515

36-
### Step 1: Create Segment user and internal database
16+
## Step 1: Create a user and internal database for Segment to store checkpoint tables
3717

38-
The first step is to create a new Segment role and grant it the appropriate permissions. Run the SQL code block below in your SQL worksheet in Snowflake. It executes the following commands:
18+
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas. Run the SQL code block below in your SQL worksheet in Snowflake to execute the following tasks:
3919

40-
- Create a new role and user for Segment Data Graph. This new role will have access to only the datasets you want to access from the Segment Data Graph.
20+
- Create a new role and user for the Segment Data Graph. This new role will only have access to the datasets you provide access to for the Data Graph.
4121
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below.
42-
- Create a new database for Segment Data Graph. **Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed**. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
22+
- **Segment requires write access to this database in order to create a schema for internal bookkeeping and to store checkpoint tables for the queries that are executed. Therefore, Segment recommends creating a new database for this purpose.** This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
4323

4424
> info ""
45-
> The variables specified at the top of the code block with the `SET` command are placeholders and should be updated.
25+
> Segment recommends creating a new database for the Data Graph.
26+
> If you choose to use an existing database that has also been used for [Segment Reverse ETL](/docs/connections/reverse-etl/), you must follow the [additional instructions](#update-user-access-for-segment-reverse-etl-schema)to update user access for the Segment Reverse ETL schema.
4627
47-
```
28+
29+
```SQL
4830
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
49-
-- Edit the following variables
50-
SET segment_connection_username='SEGMENT_LINKED_USER';
51-
SET segment_connection_password='my-safe-password';
52-
SET segment_connection_warehouse='SEGMENT_LINKED_WH';
53-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
5431

55-
-- The DB used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
32+
-- Update the following variables
33+
SET segment_connection_username = 'SEGMENT_LINKED_USER';
34+
SET segment_connection_password = 'my-safe-password';
35+
SET segment_connection_warehouse = 'SEGMENT_LINKED_WH';
36+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
37+
38+
-- The DB used for Segment's internal bookkeeping.
39+
-- Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
5640
SET segment_connection_db = 'SEGMENT_LINKED_PROFILES_DB';
5741

5842
-- ********** [OPTIONAL] UNCOMMENT THE CODE BELOW IF YOU NEED TO CREATE A NEW WAREHOUSE **********
43+
5944
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_connection_warehouse)
6045
-- WITH WAREHOUSE_SIZE = 'XSMALL'
6146
-- WAREHOUSE_TYPE = 'STANDARD'
6247
-- AUTO_SUSPEND = 600 -- 5 minutes
6348
-- AUTO_RESUME = TRUE;
6449

65-
6650
-- ********** RUN THE COMMANDS BELOW TO FINISH SETTING UP THE WAREHOUSE PERMISSIONS **********
6751

6852
-- Use admin role for setting grants
6953
USE ROLE ACCOUNTADMIN;
7054

71-
-- Create a role for Segment Data Graph
55+
-- Create a role for the Data Graph
7256
CREATE ROLE IF NOT EXISTS identifier($segment_connection_role)
7357
COMMENT = 'Used for Segment Data Graph';
7458

75-
-- Create a user for Segment Data Graph
59+
-- Create a user for the Data Graph
7660
CREATE USER IF NOT EXISTS identifier($segment_connection_username)
7761
MUST_CHANGE_PASSWORD = FALSE
7862
DEFAULT_ROLE = $segment_connection_role
79-
PASSWORD=$segment_connection_password
80-
COMMENT='Segment Data Graph User'
81-
TIMEZONE='UTC';
63+
PASSWORD = $segment_connection_password
64+
COMMENT = 'Segment Data Graph User'
65+
TIMEZONE = 'UTC';
8266

8367
-- Grant permission to the role to use the warehouse
8468
GRANT USAGE ON WAREHOUSE identifier($segment_connection_warehouse) TO ROLE identifier($segment_connection_role);
@@ -94,18 +78,17 @@ GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE iden
9478

9579
```
9680

97-
### Step 2: Grant read-only access to other databases
98-
99-
Next, give the Segment role **read-only** access to all the other databases you want to use for Data Graph including the **Profiles Sync database**
81+
## Step 2: Grant read-only access to additional databases for the Data Graph
10082

101-
Run the SQL query below for **each** database you want to use for Data Graph. **You may have to re-run this multiple times for each database you want to give access to**.
83+
Next, give the Segment role **read-only** access to additional databases you want to use for Data Graph including the Profiles Sync database. Repeat the following SQL query for **each** database you want to use for the Data Graph.
10284

103-
```
85+
``` SQL
10486

105-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
87+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
10688

107-
-- Change this for each DB you want to access and re-run the SQL below.
108-
SET linked_read_only_database='MARKETING_DB';
89+
-- ********** REPEAT THE SQL QUERY BELOW FOR EACH DATABASE YOU WANT TO USE FOR THE DATA GRAPH **********
90+
-- Change this for each DB you want to grant the Data Graph read-only access to
91+
SET linked_read_only_database = 'MARKETING_DB';
10992

11093
GRANT USAGE ON DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
11194
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
@@ -120,16 +103,15 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_on
120103

121104
```
122105

123-
### (Optional) Step 3: Restrict Snowflake schema access
106+
## (Optional) Step 3: Restrict read-only access to schemas
124107

125-
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank”}, run the following commands:
108+
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank"}, then run the following commands:
126109

127-
```
110+
```SQL
128111
-- [Optional] Further restrict access to only specific schemas and tables
129-
SET db='MY_DB';
130-
SET schema='MY_DB.MY_SCHEMA_NAME';
131-
SET segment_connection_role='SEGMENT_LINKED_ROLE';
132-
112+
SET db = 'MY_DB';
113+
SET schema = 'MY_DB.MY_SCHEMA_NAME';
114+
SET segment_connection_role = 'SEGMENT_LINKED_ROLE';
133115

134116
-- View specific schemas in database
135117
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_connection_role);
@@ -143,45 +125,50 @@ GRANT SELECT ON FUTURE EXTERNAL TABLES IN SCHEMA identifier($linked_read_only_da
143125
GRANT SELECT ON ALL MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
144126
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
145127

146-
147128
```
148129

149-
### (If applicable) Step 4: Update user acccess for Segment Reverse ETL schema
130+
## Step 4: Confirm permissions
150131

151-
> warning ""
152-
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL.
132+
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table.
153133

154-
Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient privileges on an existing `_segment_reverse_etl` schema.
155-
156-
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema.
134+
``` SQL
135+
set segment_connection_role = 'SEGMENT_LINKED_ROLE';
136+
set linked_read_only_database = 'YOUR_DB';
137+
set table_name = 'YOUR_DB.SCHEMA.TABLE';
157138

158-
Add the Snowflake table permissions by running the following commands:
139+
USE ROLE identifier($segment_connection_role);
140+
USE DATABASE identifier($linked_read_only_database) ;
141+
SHOW SCHEMAS;
142+
SELECT * FROM identifier($table_name) LIMIT 10;
159143

160144
```
145+
## Step 5: Connect your warehouse to the Data Graph
146+
147+
To connect your warehouse to the Data Graph:
148+
149+
1. Navigate to **Unify > Data Graph**. This should be a Unify space with Profiles Sync already set up.
150+
2. Click **Connect warehouse**.
151+
3. Select Snowflake as your warehouse type.
152+
4. Enter your warehouse credentials. Segment requires the following settings to connect to your Snowflake warehouse:
153+
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account
154+
- **Database**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below
155+
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below
156+
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below
157+
- **Authentication**: There are 2 supported authentication methods:
158+
- **Key Pair**: This is the recommended method of authentication. You would need to first create the user and assign it a key pair following the instructions in the [Snowflake docs](https://docs.snowflake.com/en/user-guide/key-pair-auth){:target="_blank"}. Then, follow the Segment docs above to set up Snowflake permissions and set the `segment_connections_username` variable in the SQL script to the user you just created
159+
- **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
160+
161+
5. Test your connection, then click Save.
162+
163+
## Update user acccess for Segment Reverse ETL schema
164+
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema. Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient privileges on an existing `_segment_reverse_etl` schema.
165+
166+
``` SQL
161167
-- If you want to use an existing database that already has Segment Reverse ETL schemas, you’ll need to run some additional steps below to grant the role access to the existing schemas.
162168

163169
SET retl_schema = concat($segment_connection_db,'.__segment_reverse_etl');
164-
165170
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
166-
167171
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
168-
169172
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
170173

171-
```
172-
173-
### Step 5: Confirm permissions
174-
175-
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table.
176-
177-
```
178-
set segment_connection_role='SEGMENT_LINKED_ROLE';
179-
set linked_read_only_database='YOUR_DB';
180-
set table_name = 'YOUR_DB.SCHEMA.TABLE';
181-
182-
USE ROLE identifier($segment_connection_role);
183-
USE DATABASE identifier($linked_read_only_database) ;
184-
SHOW SCHEMAS;
185-
SELECT * FROM identifier($table_name) LIMIT 10;
186-
187-
```
174+
```

0 commit comments

Comments
 (0)