Skip to content

Commit 1721aa1

Browse files
authored
Merge pull request #6011 from segmentio/Snowflake-setup-updates
Update Snowflake setup doc
2 parents 1a9bbe0 + d7f6ee6 commit 1721aa1

File tree

3 files changed

+103
-75
lines changed

3 files changed

+103
-75
lines changed

src/unify/images/snowflake-setup.png

95.9 KB
Loading

src/unify/linked-profiles/data-graph.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ The parameters are:
8383

8484
| Parameters | Definition |
8585
| ----------- | --------------------------------------------------------------------- |
86-
| `profile_folder` | This is the folder or schema location for the profile tables. |
86+
| `profile_folder` | This is the fully qualified path of the folder or schema location for the profile tables. |
8787
| `materialization` | Identifies the type of materialization (`none`). |
8888

8989
Example:
@@ -92,7 +92,7 @@ Example:
9292
# Define a profile entity
9393

9494
profile {
95-
profile_folder = "segment"
95+
profile_folder = "PRODUCTION.segment"
9696
materialization = "none"
9797

9898
}
@@ -278,7 +278,7 @@ data_graph {
278278

279279
#define a profile entity
280280
profile {
281-
profile_folder = "segment"
281+
profile_folder = "PRODUCTION.segment"
282282
materialization = "none"
283283

284284
#relate accounts to profiles with an external ID

src/unify/linked-profiles/setup-guides/snowflake-setup.md

Lines changed: 100 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -10,43 +10,49 @@ beta: true
1010

1111
On this page, you'll learn how to connect your Snowflake data warehouse to Segment.
1212

13-
Be sure to log in with a user that has read and write permissions so that Segment can write to your database.
13+
Log in to Snowflake with a user that has admin privileges to provide Segment Linked Profiles with the necessary permissions below.
1414

1515
> info ""
1616
> Both Linked Events and Linked Profiles support Snowflake.
17+
1718

19+
## Required connection settings within Segment
1820

19-
## Getting started
21+
Segment requires the following settings to connect to your Snowflake warehouse.
2022

21-
Use the following steps to set up Snowflake for Linked Profiles.
23+
![Connect Snowflake to the Data Graph](/docs/unify/images/snowflake-setup.png)
2224

23-
### Set up Snowflake Credentials and create internal Segment DB
25+
- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
26+
- **Database Name**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
27+
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
28+
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
29+
- **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
2430

25-
Run the SQL below to provide Segment Linked Profiles with the necessary permissions and roles to access the databases, tables, and schemas. These steps involve:
26-
- Creating a new role and user for Segment Linked Profiles.
27-
- Granting read-only access to specific databases and schemas that you want to use for Linked Profiles.
28-
- Granting write access to an internal database that Segment requires for bookkeeping purposes.
29-
- [Optional] Creating a new warehouse if it does not exist yet. You can skip this step if a warehouse already exists.
30-
- [Optional] As a best practice, Segment recommends that restrict access to specific databases and schemas.
31-
- Running the script below to configure the Warehouse permissions.
31+
## Set up Snowflake credentials
3232

33-
```ts
34-
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
35-
-- Edit the following variables
36-
SET segment_linked_username='SEGMENT_LINKED_USER';
37-
SET segment_linked_password='my-safe-password';
38-
SET segment_linked_default_warehouse='SEGMENT_LINKED_WH';
39-
SET segment_linked_default_role='SEGMENT_LINKED_ROLE';
33+
Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas for Segment Linked Profiles.
34+
35+
### Create Segment user and internal database
36+
37+
Use the following steps to set up your Snowflake credentials:
4038

41-
-- Use the same DB that has Profiles Sync configured. This DB is also used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app.
42-
SET segment_internal_database = 'SEGMENT_INTERNAL_DB';
39+
- Create a new role and user for Segment Linked Profiles.
40+
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below.
41+
- Create a new database for Segment Linked Profiles. Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
4342

44-
-- Use for each DB you want Linked Profiles to access
45-
SET segment_linked_database='MARKETING_DB';
43+
```
44+
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
45+
-- Edit the following variables
46+
SET segment_connection_username='SEGMENT_LINKED_USER';
47+
SET segment_connection_password='my-safe-password';
48+
SET segment_connection_warehouse='SEGMENT_LINKED_WH';
49+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
4650
51+
-- The DB used for Segment's internal bookkeeping. Note: Use this DB in the connection settings on the Segment app. This is the only DB that Segment requires write access to.
52+
SET segment_connection_db = 'SEGMENT_LINKED_PROFILES_DB';
4753
4854
-- ********** [OPTIONAL] UNCOMMENT THE CODE BELOW IF YOU NEED TO CREATE A NEW WAREHOUSE **********
49-
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_linked_default_warehouse)
55+
-- CREATE WAREHOUSE IF NOT EXISTS identifier($segment_connection_warehouse)
5056
-- WITH WAREHOUSE_SIZE = 'XSMALL'
5157
-- WAREHOUSE_TYPE = 'STANDARD'
5258
-- AUTO_SUSPEND = 600 -- 5 minutes
@@ -59,93 +65,115 @@ SET segment_linked_database='MARKETING_DB';
5965
USE ROLE ACCOUNTADMIN;
6066
6167
-- Create a role for Segment Linked Profiles
62-
CREATE ROLE IF NOT EXISTS identifier($segment_linked_default_role)
68+
CREATE ROLE IF NOT EXISTS identifier($segment_connection_role)
6369
COMMENT = 'Used for Segment Linked Profiles';
6470
6571
-- Create a user for Segment Linked Profiles
66-
CREATE USER IF NOT EXISTS identifier($segment_linked_username)
72+
CREATE USER IF NOT EXISTS identifier($segment_connection_username)
6773
MUST_CHANGE_PASSWORD = FALSE
68-
DEFAULT_ROLE = $segment_linked_default_role
69-
PASSWORD=$segment_linked_password
74+
DEFAULT_ROLE = $segment_connection_role
75+
PASSWORD=$segment_connection_password
7076
COMMENT='Segment Linked Profiles User';
7177
7278
-- Grant permission to the role to use the warehouse
73-
GRANT USAGE ON WAREHOUSE identifier($segment_linked_default_warehouse) TO ROLE identifier($segment_linked_default_role);
79+
GRANT USAGE ON WAREHOUSE identifier($segment_connection_warehouse) TO ROLE identifier($segment_connection_role);
7480
7581
-- Grant role to the user
76-
GRANT ROLE identifier($segment_linked_default_role) TO USER identifier($segment_linked_username);
77-
78-
-- Grant access to Segment internal DB used for bookkeeping. This is the same DB that contains the Profiles Sync schema.
79-
GRANT USAGE ON DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
80-
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
81-
GRANT CREATE SCHEMA ON DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
82-
GRANT SELECT ON ALL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
83-
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
84-
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
85-
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
86-
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
87-
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
88-
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
89-
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($segment_internal_database) TO ROLE identifier($segment_linked_default_role);
90-
91-
-- Grant read-only access to all DBs
92-
GRANT USAGE ON DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
93-
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
94-
GRANT SELECT ON ALL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
95-
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
96-
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
97-
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
98-
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
99-
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
100-
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
101-
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($segment_linked_database) TO ROLE identifier($segment_linked_default_role);
82+
GRANT ROLE identifier($segment_connection_role) TO USER identifier($segment_connection_username);
83+
84+
-- Create and Grant access to a Segment internal DB used for bookkeeping. This is the only DB that Segment requires write access to. This is also the DB you will use in the "Database Name" config while setting up the connection in the Segment app.
85+
CREATE DATABASE IF NOT EXISTS identifier($segment_connection_db);
86+
GRANT USAGE ON DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
87+
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
88+
GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE identifier($segment_connection_role);
89+
10290
```
10391

92+
### Grant access to other databases
93+
94+
Next, give the Segment user **read-only** access to all the other databases you want to use for Linked Profiles.
95+
96+
Run the SQL query below for **each** database you want to use for Linked Profiles:
97+
98+
```
99+
100+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
101+
102+
-- Change this for each DB you want to access and re-run the SQL below.
103+
SET linked_read_only_database='MARKETING_DB';
104104
105-
(Optional) [Snowflake Schema Access](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges): If you want to restrict access to specific schemas or tables, then run the following command:
105+
GRANT USAGE ON DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
106+
GRANT USAGE ON ALL SCHEMAS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
107+
GRANT SELECT ON ALL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
108+
GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
109+
GRANT SELECT ON ALL VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
110+
GRANT SELECT ON FUTURE VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
111+
GRANT SELECT ON ALL EXTERNAL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
112+
GRANT SELECT ON FUTURE EXTERNAL TABLES IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
113+
GRANT SELECT ON ALL MATERIALIZED VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
114+
GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_only_database) TO ROLE identifier($segment_connection_role);
106115
107-
```ts
116+
```
117+
118+
### (Optional) Restrict Snowflake schema access
119+
120+
If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank”}, run the following commands:
121+
122+
```
108123
-- [Optional] Further restrict access to only specific schemas and tables
109124
SET db='MY_DB';
110125
SET schema='MY_DB.MY_SCHEMA_NAME';
126+
SET segment_connection_role='SEGMENT_LINKED_ROLE';
127+
111128
112129
-- View specific schemas in database
113-
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_linked_default_role);
130+
GRANT USAGE ON DATABASE identifier($db) TO ROLE identifier($segment_connection_role);
131+
132+
GRANT USAGE ON SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
114133
115-
GRANT USAGE ON SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
134+
GRANT SELECT ON ALL TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
135+
136+
GRANT SELECT ON FUTURE TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_connection_role);
116137
117-
GRANT SELECT ON ALL TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
118138
119-
GRANT SELECT ON FUTURE TABLES IN SCHEMA identifier($schema) TO ROLE identifier($segment_linked_default_role);
120139
```
121140

122-
### (If applicable) Add table permissions if Reverse ETL has ever run in your database
141+
### (If applicable) Update user acccess for Segment Reverse ETL schema
142+
143+
> warning ""
144+
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL.
123145
124-
If Reverse ETL has ever run in the database you are configuring as the Segment Internal DB, a Segment-managed schema is created and a new user is added. Add the Snowflake table permissions by running the following command.
146+
Run the following SQL if you run into an error on the Segment app indicating that the user doesn't have sufficient privileges on an existing `_segment_reverse_etl` schema.
147+
148+
If Segment Reverse ETL has ever run in the database you are configuring as the Segment connection database, a Segment-managed schema is already created and you need to provide the new Segment user access to the existing schema.
125149

126-
```ts
150+
Add the Snowflake table permissions by running the following commands:
151+
152+
```
127153
-- If you want to use an existing database that already has Segment Reverse ETL schemas, you’ll need to run some additional steps below to grant the role access to the existing schemas.
128154
129155
SET retl_schema = concat($segment_internal_database,'.__segment_reverse_etl');
130156
131-
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
157+
GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
132158
133-
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
159+
GRANT CREATE TABLE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
160+
161+
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);
134162
135-
GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schema) TO ROLE identifier($segment_linked_default_role);
136163
```
137164

138165
### Confirm permissions
139166

140-
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_LINKED_USER` and run the following command to verify the role you created has the correct permissions. This command should succeed and you should be able to view the respective table.
167+
To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table.
141168

142-
```ts
143-
set segment_linked_default_role='SEGMENT_LINKED_ROLE';
144-
set segment_linked_database='YOUR_DB';
169+
```
170+
set segment_connection_role='SEGMENT_LINKED_ROLE';
171+
set linked_read_only_database='YOUR_DB';
145172
set table_name = 'YOUR_DB.SCHEMA.TABLE';
146173
147-
USE ROLE identifier($segment_linked_default_role);
148-
USE DATABASE identifier($segment_linked_database) ;
174+
USE ROLE identifier($segment_connection_role);
175+
USE DATABASE identifier($linked_read_only_database) ;
149176
SHOW SCHEMAS;
150177
SELECT * FROM identifier($table) LIMIT 10;
178+
151179
```

0 commit comments

Comments
 (0)