Skip to content

Commit 0dc77a5

Browse files
puzpuzpuzpuzpuzpuz
authored
Document mat view enhancements in 8.3.2 (#176)
1 parent cea979d commit 0dc77a5

File tree

9 files changed

+311
-15
lines changed

9 files changed

+311
-15
lines changed

‎documentation/guides/mat-views.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,11 @@ description:
77

88
:::info
99

10-
Materialized View support is now generally available (GA) and ready for production use.
10+
Materialized View support is now generally available (GA) and ready for
11+
production use.
1112

12-
If you are using versions earlier than `8.3.1`, we suggest you upgrade at your earliest convenience.
13+
If you are using versions earlier than `8.3.1`, we suggest you upgrade at your
14+
earliest convenience.
1315

1416
:::
1517

@@ -596,7 +598,8 @@ You can monitor refresh status using the `materialized_views()` system function:
596598
```questdb-sql title="Listing all materialized views"
597599
SELECT
598600
view_name,
599-
last_refresh_timestamp,
601+
last_refresh_start_timestamp,
602+
last_refresh_finish_timestamp,
600603
view_status,
601604
refresh_base_table_txn,
602605
base_table_txn
@@ -605,9 +608,9 @@ FROM materialized_views();
605608

606609
Here is an example output:
607610

608-
| view_name | last_refresh_timestamp | view_status | refresh_base_table_txn | base_table_txn |
609-
| ----------- | ---------------------- | ----------- | ---------------------- | -------------- |
610-
| trades_view | null | valid | 102 | 102 |
611+
| view_name | last_refresh_start_timestamp | last_refresh_finish_timestamp | view_status | refresh_base_table_txn | base_table_txn |
612+
| ----------- | ---------------------------- | ----------------------------- | ----------- | ---------------------- | -------------- |
613+
| trades_view | 2025-05-02T13:46:11.828212Z | 2025-05-02T13:46:21.828212Z | valid | 102 | 102 |
611614

612615
When `refresh_base_table_txn` matches `base_table_txn`, the materialized view is
613616
fully up-to-date.
@@ -672,7 +675,8 @@ To create a materialized view, your query:
672675
- Must use either `SAMPLE BY` or `GROUP BY` with a designated timestamp column
673676
key.
674677
- Must not contain `FROM-TO`, `FILL`, and `ALIGN TO FIRST OBSERVATION` clauses
675-
in `SAMPLE BY` queries
678+
in `SAMPLE BY` queries.
679+
- Must not use non-deterministic SQL functions like `now()` or `rnd_uuid4()`.
676680
- Must use join conditions that are compatible with incremental refreshing.
677681
- When the base table has [deduplication](/docs/concept/deduplication/) enabled,
678682
the non-aggregate columns selected by the materialized view query must be a

‎documentation/reference/function/meta.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -493,10 +493,13 @@ Returns a `table` including the following information:
493493
- `view_name` - materialized view name
494494
- `refresh_type` - refresh strategy type
495495
- `base_table_name` - base table name
496-
- `last_refresh_timestamp` - last time when the view was incrementally refreshed
496+
- `last_refresh_start_timestamp` - last time when an incremental refresh for the
497+
view was started
498+
- `last_refresh_finish_timestamp` - last time when an incremental refresh for
499+
the view finished
497500
- `view_sql` - query used to populate view data
498501
- `view_table_dir_name` - view directory name
499-
- `view_status` - view status: valid or invalid
502+
- `view_status` - view status: 'valid', 'refreshing', or 'invalid'
500503
- `invalidation_reason` - message explaining why the view was marked as invalid
501504
- `refresh_base_table_txn` - the last base table transaction used to refresh the
502505
materialized view
@@ -508,9 +511,9 @@ Returns a `table` including the following information:
508511
materialized_views();
509512
```
510513

511-
| view_name | refresh_type | base_table_name | last_refresh_timestamp | view_sql | view_table_dir_name | view_status | invalidation_reason | refresh_base_table_txn | base_table_txn |
512-
| --------- | ------------ | --------------- | --------------------------- | ------------- | ------------------- | ----------- | ------------------- | ---------------------- | -------------- |
513-
| trades_1h | incremental | trades | 2024-10-24T17:22:09.842574Z | query text... | trades_1h~10 | valid | | 42 | 42 |
514+
| view_name | refresh_type | base_table_name | last_refresh_start_timestamp | last_refresh_finish_timestamp | view_sql | view_table_dir_name | view_status | invalidation_reason | refresh_base_table_txn | base_table_txn |
515+
| --------- | ------------ | --------------- | ---------------------------- | ----------------------------- | ------------- | ------------------- | ----------- | ------------------- | ---------------------- | -------------- |
516+
| trades_1h | incremental | trades | 2024-10-24T17:22:20.536577Z | 2024-10-24T17:22:09.842574Z | query text... | trades_1h~10 | refreshing | | 42 | 42 |
514517

515518
## version/pg_catalog.version
516519

‎documentation/reference/sql/alter-mat-view-set-refresh-limit.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
title: ALTER MATERIALIZED VIEW SET REFRESH LIMIT
3+
sidebar_label: SET REFRESH LIMIT
4+
description:
5+
ALTER MATERIALIZED VIEW SET REFRESH LIMIT SQL keyword reference documentation.
6+
---
7+
8+
Sets the time limit for incremental refresh on a materialized view.
9+
10+
## Syntax
11+
12+
![Flow chart showing the syntax of ALTER MATERIALIZED VIEW SET REFRESH LIMIT command](/images/docs/diagrams/alterMatViewSetRefreshLimit.svg)
13+
14+
## Description
15+
16+
To protect older aggregated data from being overwritten by inserts with old
17+
timestamps, configure a refresh limit on a materialized view using the
18+
`ALTER MATERIALIZED VIEW SET REFRESH LIMIT` command. This means that base
19+
table's rows with timestamps older than the refresh limit will not be aggregated
20+
in the materialized view.
21+
22+
Let's suppose we've just configured refresh limit on a materialized view:
23+
24+
```sql
25+
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH LIMIT 1 WEEK;
26+
```
27+
28+
Next, the current time is `2025-05-02T12:00:00.000000Z` and we're inserting a
29+
few rows into the base table:
30+
31+
```sql
32+
INSERT INTO trades VALUES
33+
('2025-03-02T12:00:00.000000Z', 'BTC-USD', 39269.98, 0.042),
34+
('2025-04-02T12:00:00.000000Z', 'BTC-USD', 39170.01, 0.042),
35+
('2025-05-02T12:00:00.000000Z', 'BTC-USD', 38450.10, 0.042);
36+
```
37+
38+
The first two rows here are older than a week, so incremental refresh will only
39+
take place for the third row with the `2025-05-02T12:00:00.000000Z` timestamp.
40+
41+
:::note
42+
43+
The limit is only applied to incremental refresh, but not to the
44+
[`REFRESH MATERIALIZED VIEW FULL`](/docs/reference/sql/refresh-mat-view)
45+
command. This means that when you run a full refresh command, all rows from the
46+
base table are aggregated in the materialized view.
47+
48+
:::
49+
50+
The `REFRESH LIMIT` value consists of a number and a time unit, one of:
51+
52+
- `HOURS`
53+
- `DAYS`
54+
- `WEEKS`
55+
- `MONTHS`
56+
- `YEARS`
57+
58+
The limit units fall into two categories:
59+
60+
1. Fixed time periods:
61+
- `HOURS`
62+
- `DAYS`
63+
- `WEEKS`
64+
2. Calendar-based periods:
65+
- `MONTHS`
66+
- `YEARS`
67+
68+
Fixed-time periods are always exact durations: `1 WEEK` is always 7 days.
69+
70+
Calendar-based periods may vary in length: `1 MONTH` from January 15th goes to
71+
February 15th and could be between 28 and 31 days.
72+
73+
QuestDB accepts both singular and plural forms:
74+
75+
- `HOUR` or `HOURS`
76+
- `DAY` or `DAYS`
77+
- `WEEK` or `WEEKS`
78+
- `MONTH` or `MONTHS`
79+
- `YEAR` or `YEARS`
80+
81+
It also supports shorthand notation: `3h` for 3 hours, `2M` for 2 months.
82+
83+
## Examples
84+
85+
Set the refresh limit to 1 day:
86+
87+
```sql
88+
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH LIMIT 1 DAY;
89+
```
90+
91+
Set the limit to 8 hours, using the shorthand syntax for the time unit:
92+
93+
```sql
94+
ALTER MATERIALIZED VIEW trades_hourly_prices SET REFRESH LIMIT 8h;
95+
```

‎documentation/reference/sql/alter-mat-view-set-ttl.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
title: ALTER MATERIALIZED VIEW SET TTL
3+
sidebar_label: SET TTL
4+
description:
5+
ALTER MATERIALIZED VIEW SET TTL SQL keyword reference documentation.
6+
---
7+
8+
Sets the [time-to-live](/docs/concept/ttl/) (TTL) period on a materialized view.
9+
10+
## Syntax
11+
12+
![Flow chart showing the syntax of ALTER MATERIALIZED VIEW SET TTL command](/images/docs/diagrams/alterMatViewSetTtl.svg)
13+
14+
## Description
15+
16+
To keep only recent aggregated data in a materialized view, configure a
17+
time-to-live (TTL) period on the view using the
18+
`ALTER MATERIALIZED VIEW SET TTL` command.
19+
20+
The value follows the same rules as the one in the
21+
[`ALTER TABLE SET TTL`](/docs/reference/sql/alter-table-set-ttl) command.
22+
23+
:::note
24+
25+
QuestDB drops data that exceeded its TTL only a whole partition at a time. For
26+
this reason, the TTL period must be a whole number multiple of the view's
27+
partition size.
28+
29+
:::
30+
31+
## Examples
32+
33+
Set the TTL to 3 days:
34+
35+
```sql
36+
ALTER MATERIALIZED VIEW trades_hourly_prices SET TTL 3 DAYS;
37+
```
38+
39+
Set the TTL to 12 hours, using the shorthand syntax for the time unit:
40+
41+
```sql
42+
ALTER MATERIALIZED VIEW trades_hourly_prices SET TTL 12h;
43+
```

‎documentation/reference/sql/alter-table-set-ttl.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ QuestDB accepts both singular and plural forms:
5050
- `MONTH` or `MONTHS`
5151
- `YEAR` or `YEARS`
5252

53-
It also supports shorthand notation: `3H` for 3 hours, `2M` for 2 months.
53+
It also supports shorthand notation: `3h` for 3 hours, `2M` for 2 months.
5454

5555
:::note
5656

@@ -81,5 +81,5 @@ ALTER TABLE weather SET TTL 3 WEEKS;
8181
Set the TTL to 12 hours, using the shorthand syntax for the time unit:
8282

8383
```sql
84-
ALTER TABLE weather SET TTL 12H;
84+
ALTER TABLE weather SET TTL 12h;
8585
```

‎documentation/sidebars.js

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,8 @@ module.exports = {
177177
label: "ALTER MATERIALIZED VIEW",
178178
items: [
179179
"reference/sql/alter-mat-view-change-symbol-capacity",
180+
"reference/sql/alter-mat-view-set-refresh-limit",
181+
"reference/sql/alter-mat-view-set-ttl",
180182
"reference/sql/alter-mat-view-resume-wal",
181183
],
182184
},

‎static/images/docs/diagrams/.railroad

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,7 +385,13 @@ alterMatView
385385
::= 'ALTER' 'MATERIALIZED' 'VIEW' viewName
386386

387387
alterMatViewSymbolCapacity
388-
::= 'ALTER' 'MATERIALIZED' 'VIEW' viewName 'ALTER' 'COLUMN' columnName 'SYMBOL' 'CAPACITY' capacity
388+
::= 'ALTER' 'MATERIALIZED' 'VIEW' viewName 'ALTER' 'COLUMN' columnName 'SYMBOL' 'CAPACITY' capacity
389+
390+
alterMatViewSetTtl
391+
::= 'ALTER' 'MATERIALIZED' 'VIEW' viewName 'SET' 'TTL' n ('HOURS' | 'DAYS' | 'WEEKS' | 'MONTHS' | 'YEARS')
392+
393+
alterMatViewSetRefreshLimit
394+
::= 'ALTER' 'MATERIALIZED' 'VIEW' viewName 'SET' 'REFRESH' 'LIMIT' n ('HOURS' | 'DAYS' | 'WEEKS' | 'MONTHS' | 'YEARS')
389395

390396
refreshMatView
391397
::= 'REFRESH' 'MATERIALIZED' 'VIEW' viewName ('FULL' | 'INCREMENTAL')

‎static/images/docs/diagrams/alterMatViewSetRefreshLimit.svg

Lines changed: 73 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)