Skip to content

Commit 14c63ba

Browse files
committed
Update Snowflake setup doc
1 parent ccaf648 commit 14c63ba

File tree

2 files changed

+97
-69
lines changed

2 files changed

+97
-69
lines changed

src/unify/images/snowflake-setup.png

155 KB
Loading

src/unify/linked-profiles/setup-guides/snowflake-setup.md

Lines changed: 97 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -10,43 +10,51 @@ beta: true
1010

1111
On this page, you'll learn how to connect your Snowflake data warehouse to Segment.
1212

13-
Be sure to log in with a user that has read and write permissions so that Segment can write to your database.
13+
Within Snowflake, log in with a user that has admin priveleges to provide Segment Linked Profiles with the necessary permissions below.
1414

1515
> info ""
1616
> Both Linked Events and Linked Profiles support Snowflake.
1717
1818

19-
## Getting started
19+
## Required connection settings within Segment
2020

21-
Use the following steps to set up Snowflake for Linked Profiles.
21+
Segment requires the following settings to connect to your Snowflake warehouse.
2222

23-
### Set up Snowflake Credentials and create internal Segment DB
23+
![Connect Snowflake to the Data Graph](/docs/unify/images/snowflake-setup.png)
2424

25-
Run the SQL below to provide Segment Linked Profiles with the necessary permissions and roles to access the databases, tables, and schemas. These steps involve:
26-
- Creating a new role and user for Segment Linked Profiles.
27-
- Granting read-only access to specific databases and schemas that you want to use for Linked Profiles.
28-
- Granting write access to an internal database that Segment requires for bookkeeping purposes.
29-
- [Optional] Creating a new warehouse if it does not exist yet. You can skip this step if a warehouse already exists.
30-
- [Optional] As a best practice, Segment recommends that restrict access to specific databases and schemas.
31-
- Running the script below to configure the Warehouse permissions.
25+
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
26+
- **Database Name**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
27+
- **Warehouse**: The warehouse in your Snowflake account that you wish to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
28+
- **Username**: The Snowflake user that will be used by Segment to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
29+
- **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
3230

33-
```ts
34-
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
35-
-- Edit the following variables
36-
SET segment_linked_username='SEGMENT_LINKED_USER';
37-
SET segment_linked_password='my-safe-password';
38-
SET segment_linked_default_warehouse='SEGMENT_LINKED_WH';
39-
SET segment_linked_default_role='SEGMENT_LINKED_ROLE';
31+
## Set up Snowflake credentials
32+
33+
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas for Segment Linked Profiles.
34+
35+
Segment only requires write access to one database where it creates a schema for internal bookkeeping. Segment recommends creating an empty database for this purpose using the script below. All other databases and schemas require read-only access granted to the Segment user.
4036

41-
-- Use the same DB that has Profiles Sync configured. This DB is also used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app.
42-
SET segment_internal_database = 'SEGMENT_INTERNAL_DB';
4337

44-
-- Use for each DB you want Linked Profiles to access
45-
SET segment_linked_database='MARKETING_DB';
38+
Use the following steps to set up your Snowflake credentials:
4639

40+
- Create a new role and user for Segment Linked Profiles.
41+
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below.
42+
- Create a new database that Segment will use for internal bookkeeping. You'll need to grant the Segment user **write** access to this database that is used to store checkpoint tables for the queries that are executed. This is the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
43+
44+
45+
```ts
46+
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
47+
-- Edit the following variables
48+
SET segment_connection_username='SEGMENT_LINKED_USER';
49+
SET segment_connection_password='my-safe-password';
50+
SET segment_connection_warehouse='SEGMENT_LINKED_WH';
51+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
52+
53+
-- The DB used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
54+
SET segment_connection_db = 'SEGMENT_LINKED_PROFILES_DB';
4755

4856
-- ********** [OPTIONAL] UNCOMMENT THE CODE BELOW IF YOU NEED TO CREATE A NEW WAREHOUSE **********
49-
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_linked_default_warehouse)
57+
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_connection_warehouse)
5058
-- WITH WAREHOUSE_SIZE = 'XSMALL'
5159
-- WAREHOUSE_TYPE = 'STANDARD'
5260
-- AUTO_SUSPEND = 600 -- 5 minutes
@@ -59,93 +67,113 @@ SET segment_linked_database='MARKETING_DB';
5967
USE ROLE ACCOUNTADMIN;
6068

6169
-- Create a role for Segment Linked Profiles
62-
CREATE ROLE IF NOT EXISTS identifier($segment_linked_default_role)
70+
CREATE ROLE IF NOT EXISTS identifier($segment_connection_role)
6371
COMMENT = 'Used for Segment Linked Profiles';
6472

6573
-- Create a user for Segment Linked Profiles
66-
CREATE USER IF NOT EXISTS identifier($segment_linked_username)
74+
CREATE USER IF NOT EXISTS identifier($segment_connection_username)
6775
MUST_CHANGE_PASSWORD = FALSE
68-
DEFAULT_ROLE = $segment_linked_default_role
69-
PASSWORD=$segment_linked_password
76+
DEFAULT_ROLE = $segment_connection_role
77+
PASSWORD=$segment_connection_password
7078
COMMENT='Segment Linked Profiles User';
7179

7280
-- Grant permission to the role to use the warehouse
73-
GRANT USAGE ON WAREHOUSE identifier($segment_linked_default_warehouse) TO ROLE identifier($segment_linked_default_role);
81+
GRANT USAGE ON WAREHOUSE identifier($segment_connection_warehouse) TO ROLE identifier($segment_connection_role);
7482

7583
-- Grant role to the user
76-
GRANT ROLE identifier($segment_linked_default_role) TO USER identifier($segment_linked_username);
77-
78-
-- Grant access to Segment internal DB used for bookkeeping. This is the same DB that contains the Profiles Sync schema.
79-
GRANT USAGE ON DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
80-
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
81-
GRANT CREATE SCHEMA ON DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
82-
GRANT SELECT ON ALL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
83-
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
84-
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
85-
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
86-
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
87-
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
88-
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
89-
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
90-
91-
-- Grant read-only access to all DBs
92-
GRANT USAGE ON DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
93-
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
94-
GRANT SELECT ON ALL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
95-
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
96-
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
97-
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
98-
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
99-
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
100-
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
101-
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
84+
GRANT ROLE identifier($segment_connection_role) TO USER identifier($segment_connection_username);
85+
86+
-- Create and Grant access to Segment internal DB used for bookkeeping. This is the only DB that Segment requires write access to. This is also the DB you will use in the "Database Name" config while setting up the connection in the Segment app.
87+
CREATE DATABASE IF NOT EXISTS identifier($segment_connection_db);
88+
GRANT USAGE ON DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
89+
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
90+
GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
91+
10292
```
10393

94+
Next, give the Segment user **read-only** access to all the other databases you want to use for Linked Profiles.
10495

105-
(Optional) [Snowflake Schema Access](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges): If you want to restrict access to specific schemas or tables, then run the following command:
96+
Run the SQL query below for **each** database you want to use for Linked Profiles:
97+
98+
```ts
99+
100+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
101+
102+
-- Change this for each DB you want to access and re-run the SQL below.
103+
SET linked_read_only_database='MARKETING_DB';
104+
105+
GRANT USAGE ON DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
106+
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
107+
GRANT SELECT ON ALL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
108+
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
109+
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
110+
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
111+
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
112+
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
113+
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
114+
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
115+
116+
```
117+
118+
### (Optional) Snowflake schema access
119+
120+
[Snowflake schema access](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges): If you want to restrict access to specific schemas or tables, then run the following commands:
106121

107122
```ts
108123
-- [Optional] Further restrict access to only specific schemas and tables
109124
SET db='MY_DB';
110125
SET schema='MY_DB.MY_SCHEMA_NAME';
126+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
127+
111128

112129
-- View specific schemas in database
113-
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_linked_default_role);
130+
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_connection_role);
114131

115-
GRANT USAGE ON SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
132+
GRANT USAGE ON SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
133+
134+
GRANT SELECT ON ALL TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
135+
136+
GRANT SELECT ON FUTURE TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
116137

117-
GRANT SELECT ON ALL TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
118138

119-
GRANT SELECT ON FUTURE TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
120139
```
121140

122-
### (If applicable) Add table permissions if Reverse ETL has ever run in your database
141+
### (If applicable) Add Snowflake table permissions
142+
Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient priveleges on an existing `_segment_reverse_etl` schema.
143+
144+
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema.
145+
146+
> warning ""
147+
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL.
123148
124-
If Reverse ETL has ever run in the database you are configuring as the Segment Internal DB, a Segment-managed schema is created and a new user is added. Add the Snowflake table permissions by running the following command.
149+
150+
Add the Snowflake table permissions by running the following commands:
125151

126152
```ts
127153
-- If you want to use an existing database that already has Segment Reverse ETL schemas, youll need to run some additional steps below to grant the role access to the existing schemas.
128154

129155
SET retl_schema = concat($segment_internal_database,'.__segment_reverse_etl');
130156

131-
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
157+
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
158+
159+
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
132160

133-
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
161+
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
134162

135-
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
136163
```
137164

138165
### Confirm permissions
139166

140-
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_LINKED_USER` and run the following command to verify the role you created has the correct permissions. This command should succeed and you should be able to view the respective table.
167+
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. This command should succeed and you should be able to view the respective table.
141168

142169
```ts
143-
set segment_linked_default_role='SEGMENT_LINKED_ROLE';
144-
set segment_linked_database='YOUR_DB';
170+
set segment_connection_role='SEGMENT_LINKED_ROLE';
171+
set linked_read_only_database='YOUR_DB';
145172
set table_name = 'YOUR_DB.SCHEMA.TABLE';
146173

147-
USE ROLE identifier($segment_linked_default_role);
148-
USE DATABASE identifier($segment_linked_database) ;
174+
USE ROLE identifier($segment_connection_role);
175+
USE DATABASE identifier($linked_read_only_database) ;
149176
SHOW SCHEMAS;
150177
SELECT * FROM identifier($table) LIMIT 10;
178+
151179
```

0 commit comments

Comments
 (0)