Skip to content

Commit 25a9a85

Browse files
committed
docs: create table update syntax - webhook table(part 3)
1 parent 7222520 commit 25a9a85

14 files changed

+274
-217
lines changed

doc/user/content/sql/create-table.md

Lines changed: 74 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ Materialize, you can create:
1919
be written to by the user; they are populated through data ingestion from a
2020
source.
2121

22+
- Webhook-populated tables. Webhook-populated tables cannot be written to by the
23+
user; they are populated through data posted to associated **public** webhook
24+
endpoint, automatically created with the table creation.
25+
2226
Tables can be joined with other tables, materialized views, and views. Tables in
2327
Materialize are similar to tables in standard relational databases: they consist
2428
of rows and columns where the columns are fixed when the table is created.
@@ -45,14 +49,14 @@ CREATE [TEMP|TEMPORARY] TABLE <table_name> (
4549
;
4650
```
4751

48-
{{% yaml-table data="syntax_options/create_table_options_user_populated" %}}
52+
{{% yaml-table data="syntax_options/create_table/create_table_options_user_populated" %}}
4953

5054
{{</ tab >}}
5155

52-
{{< tab "Source-populated tables (DB source)" >}}
56+
{{< tab "Source-populated tables (DB connector)" >}}
5357

54-
To create a table from a [source](/sql/create-source/), where the source maps to
55-
an external database system:
58+
To create a table from a [source](/sql/create-source/) connected (via native
59+
connector) to an external database system:
5660

5761
{{< note >}}
5862

@@ -73,46 +77,14 @@ CREATE TABLE <table_name> FROM SOURCE <source_name> (REFERENCE <ref_object>)
7377
;
7478
```
7579

76-
{{% yaml-table data="syntax_options/create_table_options_source_populated_db" %}}
80+
{{% yaml-table data="syntax_options/create_table/create_table_options_source_populated_db" %}}
7781

7882
<a name="supported-db-source-types" ></a>
7983

8084
{{< tabs >}}
8185
{{< tab "Supported MySQL types">}}
8286

83-
Materialize natively supports the following MySQL types:
84-
85-
<ul style="column-count: 3">
86-
<li><code>bigint</code></li>
87-
<li><code>binary</code></li>
88-
<li><code>bit</code></li>
89-
<li><code>blob</code></li>
90-
<li><code>boolean</code></li>
91-
<li><code>char</code></li>
92-
<li><code>date</code></li>
93-
<li><code>datetime</code></li>
94-
<li><code>decimal</code></li>
95-
<li><code>double</code></li>
96-
<li><code>float</code></li>
97-
<li><code>int</code></li>
98-
<li><code>json</code></li>
99-
<li><code>longblob</code></li>
100-
<li><code>longtext</code></li>
101-
<li><code>mediumblob</code></li>
102-
<li><code>mediumint</code></li>
103-
<li><code>mediumtext</code></li>
104-
<li><code>numeric</code></li>
105-
<li><code>real</code></li>
106-
<li><code>smallint</code></li>
107-
<li><code>text</code></li>
108-
<li><code>time</code></li>
109-
<li><code>timestamp</code></li>
110-
<li><code>tinyblob</code></li>
111-
<li><code>tinyint</code></li>
112-
<li><code>tinytext</code></li>
113-
<li><code>varbinary</code></li>
114-
<li><code>varchar</code></li>
115-
</ul>
87+
{{< include-md file="shared-content/mysql-supported-types.md" >}}
11688

11789
Replicating tables that contain **unsupported data types** is
11890
possible via the [`TEXT COLUMNS` option](#text-columns) for the
@@ -130,39 +102,8 @@ use the [`EXCLUDE COLUMNS`](#exclude-columns) option.
130102
{{</ tab >}}
131103

132104
{{< tab "Supported PostgreSQL types">}}
133-
Materialize natively supports the following PostgreSQL types (including the
134-
array type for each of the types):
135105

136-
<ul style="column-count: 3">
137-
<li><code>bool</code></li>
138-
<li><code>bpchar</code></li>
139-
<li><code>bytea</code></li>
140-
<li><code>char</code></li>
141-
<li><code>date</code></li>
142-
<li><code>daterange</code></li>
143-
<li><code>float4</code></li>
144-
<li><code>float8</code></li>
145-
<li><code>int2</code></li>
146-
<li><code>int2vector</code></li>
147-
<li><code>int4</code></li>
148-
<li><code>int4range</code></li>
149-
<li><code>int8</code></li>
150-
<li><code>int8range</code></li>
151-
<li><code>interval</code></li>
152-
<li><code>json</code></li>
153-
<li><code>jsonb</code></li>
154-
<li><code>numeric</code></li>
155-
<li><code>numrange</code></li>
156-
<li><code>oid</code></li>
157-
<li><code>text</code></li>
158-
<li><code>time</code></li>
159-
<li><code>timestamp</code></li>
160-
<li><code>timestamptz</code></li>
161-
<li><code>tsrange</code></li>
162-
<li><code>tstzrange</code></li>
163-
<li><code>uuid</code></li>
164-
<li><code>varchar</code></li>
165-
</ul>
106+
{{< include-md file="shared-content/postgres-supported-types.md" >}}
166107

167108
Replicating tables that contain **unsupported data types** is possible via the
168109
[`TEXT COLUMNS` option](#text-columns). When decoded as `text`, the specified
@@ -179,17 +120,41 @@ columns will not have the expected PostgreSQL type features. For example:
179120
[`enum`]: https://www.postgresql.org/docs/current/datatype-enum.html
180121
[`money`]: https://www.postgresql.org/docs/current/datatype-money.html
181122

123+
{{</ tab >}}
124+
125+
{{< tab "Supported SQL Server types">}}
126+
127+
{{< include-md file="shared-content/sql-server-supported-types.md" >}}
128+
129+
Replicating tables that contain **unsupported data types** is possible via the
130+
[`EXCLUDE COLUMNS`
131+
option](#exclude-columns) for the
132+
following types:
133+
134+
<ul style="column-count: 3">
135+
<li><code>text</code></li>
136+
<li><code>ntext</code></li>
137+
<li><code>image</code></li>
138+
<li><code>varchar(max)</code></li>
139+
<li><code>nvarchar(max)</code></li>
140+
<li><code>varbinary(max)</code></li>
141+
</ul>
142+
143+
**Timestamp rounding**
144+
145+
{{< include-md file="shared-content/sql-server-timestamp-rounding.md" >}}
146+
182147
{{</ tab >}}
183148
{{</ tabs >}}
184149

185150
See also [Materialize SQL data types](/sql/types/).
186151

187152
{{</ tab >}}
188153

189-
{{< tab "Source-populated tables (Kafka/Redpanda source)" >}}
154+
{{< tab "Source-populated tables (via Kafka/Redpanda)" >}}
190155

191-
To create a table from a source, where the source maps to an external
192-
Kafka/Redpanda system:
156+
To create a table from a source, where the source is connected to
157+
Kafka/Redpanda:
193158

194159
{{< note >}}
195160

@@ -210,7 +175,6 @@ CREATE TABLE <table_name> FROM SOURCE <source_name> [(REFERENCE <ref_object>)]
210175
-- INLINE <schema> | ID <schema_registry_id> | LATEST ]
211176
-- | PROTOBUF USING CONFLUENT SCHEMA REGISTRY CONNECTION <conn_name>
212177
-- | PROTOBUF MESSAGE <msg_name> USING SCHEMA <encoded_schema>
213-
-- | CSV WITH HEADER ( <col_name>[, ...]) [DELIMITED BY <char>]
214178
-- | CSV WITH <num> COLUMNS DELIMITED BY <char>
215179
-- | JSON | TEXT | BYTES
216180
]
@@ -227,10 +191,36 @@ CREATE TABLE <table_name> FROM SOURCE <source_name> [(REFERENCE <ref_object>)]
227191
;
228192
```
229193

230-
{{% yaml-table data="syntax_options/create_table_options_source_populated_kafka"
194+
{{% yaml-table data="syntax_options/create_table/create_table_options_source_populated_kafka"
231195
%}}
232196

233197

198+
{{</ tab >}}
199+
200+
{{< tab "Webhook-populated table" >}}
201+
202+
To create a table (and the associated webhook endpoint) that is populated with
203+
data POSTed to the associated webhook endpoint. The created table has, by
204+
default, 1 column `body`. You can specify `INCLUDE <header_option>` to include
205+
header columns.
206+
207+
```mzsql
208+
CREATE TABLE <table_name>
209+
FROM WEBHOOK
210+
BODY FORMAT <TEXT | JSON [ARRAY] | BYTES>
211+
[ INCLUDE <header_option> ]
212+
-- <header_option> can be:
213+
-- INCLUDE HEADER <header_name> AS <col_name> [BYTES] [, ... ]
214+
-- | INCLUDE HEADERS [ ([NOT] <header_name> [, [NOT] <header_name> [, ...] ]) ]
215+
[ CHECK (
216+
[ WITH (<BODY|HEADERS|SECRET <secret_name>> [AS <alias>] [BYTES] [, ... ]) ]
217+
<check_expression>
218+
) ]
219+
```
220+
221+
{{% yaml-table data="syntax_options/create_table/create_table_options_webhook_populated"
222+
%}}
223+
234224
{{</ tab >}}
235225

236226
{{</ tabs >}}
@@ -262,6 +252,15 @@ connections. They are always created in the special `mz_temp` schema.
262252
Temporary tables may depend upon other temporary database objects, but non-temporary
263253
tables may not depend on temporary objects.
264254

255+
### Required privileges
256+
257+
The privileges required to execute the command are:
258+
259+
- `CREATE` privileges on the containing schema.
260+
- `USAGE` privileges on all types used in the table definition.
261+
- `USAGE` privileges on the schemas that all types in the statement are
262+
contained in.
263+
265264
## Examples
266265

267266
### Creating a table
@@ -287,13 +286,6 @@ a true int4
287286
b false text
288287
```
289288

290-
## Privileges
291-
292-
The privileges required to execute this statement are:
293-
294-
- `CREATE` privileges on the containing schema.
295-
- `USAGE` privileges on all types used in the table definition.
296-
- `USAGE` privileges on the schemas that all types in the statement are contained in.
297289

298290
## Related pages
299291

doc/user/data/syntax_options/create_table_options_source_populated_kafka.yml renamed to doc/user/data/syntax_options/create_table/create_table_options_source_populated_kafka.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,6 @@ rows:
4848
| `JSON` | Decode the data as JSON. |
4949
| `TEXT` | Decode the data as TEXT. |
5050
| `BYTES` | Decode the data as BYTES. |
51-
| `CSV WITH HEADER ( <col_name>[, ...]) [DELIMITED BY <char>]` | Parse the data as CSV with a header row. Materialize uses this header to infer both the number of columns and their names. The header is **not** ingested as data. The optional `DELIMITED BY <char>` clause specifies the delimiter character. <br><br>The data is decoded as [`text`](/sql/types/text). You can convert the data to other to other types using explicit [casts](/sql/functions/cast/) when creating views.|
5251
| `CSV WITH <num> COLUMNS DELIMITED BY <char>` | Parse the data as CSV with a specified number of columns and a specified delimiter. The columns are named `column1`, `column2`...`columnN`. <br><br> The data is decoded as [`text`](/sql/types/text). You can convert the data to other to other types using explicit [casts](/sql/functions/cast/) when creating views.|
5352
5453
{{< include-md file="shared-content/kafka-format-envelope-compat-table.md"
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
columns:
2+
- column: "Parameter"
3+
- column: "Description"
4+
rows:
5+
- "Parameter": "`<table_name>`"
6+
"Description": |
7+
8+
The name of the table to create. Names for tables must follow the [naming
9+
guidelines](/sql/identifiers/#naming-restrictions).
10+
11+
- "Parameter": |
12+
**BODY FORMAT \<format\>**
13+
"Description": |
14+
15+
The mesage body format. The following `<format>`s are supported:
16+
17+
| Format | Type | Description |
18+
|-------------|--------|-------------|
19+
| `BYTES` | `bytea` | Does **no parsing** of the request, and stores the body of a request as it was received. |
20+
| `JSON` | `jsonb` | Parses the body of a request as JSON. Also accepts events batched as newline-delimited JSON (`NDJSON`). If the body is not valid JSON, a response of `400 Bad Request` will be returned. |
21+
| `JSON ARRAY`| `jsonb` | Parses the body of a request as a list of JSON objects, automatically expanding the list of objects to individual rows. Also accepts a single JSON object. If the body is not valid JSON, a response of `400 Bad Request` will be returned. |
22+
| `TEXT` | `text` | Parses the body of a request as `UTF-8` text. If the body is not valid `UTF-8`, a response of `400 Bad Request` will be returned. |
23+
24+
- "Parameter": |
25+
**INCLUDE <header_option\>**
26+
"Description": |
27+
28+
Optional. If specified, include the header information in the table.
29+
30+
| header_option | Description |
31+
|---------------|-------------|
32+
| `INCLUDE HEADER <header_name> AS <col_name> [BYTES] [,...]` | Include the specified header in a column named `<col_name>`. |
33+
| `INCLUDE HEADERS [([NOT] <header_name> [, [NOT] <header_name> ... ])]`| Include a column named `headers` of type `map[text => text]`: <ul><li>`INCLUDE HEADERS` to include all headers of the request.</li><li>`INCLUDE HEADERS (<header_name>, ...)` to only include the specified headers.</li><li>`INCLUDE HEADERS (NOT <header_name>, ...)` to include all headers except those specified with the `NOT` options.</li></ul>|
34+
35+
- "Parameter": |
36+
**CHECK( [ WITH (...)] <check_expression>)**
37+
"Description": |
38+
39+
Optional. If specified, validate each request received using the boolean
40+
`<check_expression>`. To reference the body, headers, or secret in the
41+
check expression, use the `WITH` clause.
42+
43+
| WITH options | Description
44+
| ------------------|-----------------
45+
| `BODY [AS <alias>] [BYTES]` | Makes available the `body` column -- typed as `text` -- to the check expression. The column can be renamed with the optional **AS** _alias_ statement. The data type can be changed to `bytea` with the optional **BYTES** keyword.
46+
| `HEADERS [AS <alias>] [BYTES]` | Makes available the `headers` column -- typed as `map[text => text]` -- to the check expression. The column can be renamed with the optional **AS** _alias_ statement. The data type can be changed to `map[text => bytea]` with the optional **BYTES** keyword.
47+
| `SECRET <secret_name> [AS <alias>] [BYTES]` | Securely makes available a [`SECRET`](/sql/create-secret) -- typed as `text` -- to the check expression. The column can be renamed with the optional **AS** _alias_ statement. The data type can be changed to `map[text => bytea]` with the optional **BYTES** keyword. When used with the [`constant_time_eq` function](/sql/functions/#constant_time_eq), if the secret is in a different namespace to the source, rename the column with the **AS** _alias_ statement since the function **does not support** fully qualified secret names.

doc/user/layouts/partials/head.html

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,8 @@
116116
{{/* Tabs */}}
117117
<script>
118118
document.addEventListener("DOMContentLoaded", function () {
119+
const currentHash = window.location.hash;
120+
119121
document.querySelectorAll(".code-tabs").forEach(function (tabGroup, groupIndex) {
120122
const navTabs = tabGroup.querySelector(".nav-tabs");
121123
const tabContent = tabGroup.querySelector(".tab-content");
@@ -126,8 +128,7 @@
126128
// Create tab headers from panes
127129
tabPanes.forEach(function (pane, tabIndex) {
128130
const title = pane.getAttribute("title") || `Tab ${tabIndex + 1}`;
129-
const id = `tab-${groupIndex}-${tabIndex}`;
130-
pane.setAttribute("id", id);
131+
const id = pane.getAttribute("id");
131132

132133
const tabItem = document.createElement("li");
133134
const link = document.createElement("a");
@@ -155,9 +156,16 @@
155156
});
156157
});
157158

158-
// Activate the first tab by default
159-
const firstLink = navTabs.querySelector("a");
160-
if (firstLink) firstLink.click();
159+
const matchingLink = currentHash
160+
? navTabs.querySelector(`a[href="${currentHash}"]`)
161+
: null;
162+
163+
if (matchingLink) {
164+
matchingLink.click();
165+
} else {
166+
const firstLink = navTabs.querySelector("a");
167+
if (firstLink) firstLink.click();
168+
}
161169
});
162170
});
163171

doc/user/layouts/shortcodes/tab.html

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
<!-- Adapted from https://discourse.gohugo.io/t/code-tabs-widget/975/7 -->
2-
<div class="tab-pane" title="{{ .Get 0 }}">
3-
{{ .Inner | .Page.RenderString }}
2+
{{ $title := .Get 0 }}
3+
{{ $id := printf "tab-%s" (anchorize $title) }}
4+
5+
<div class="tab-pane" title="{{ $title }}" id="{{ $id }}">
6+
{{ .Inner | .Page.RenderString }}
47
</div>

0 commit comments

Comments
 (0)