Skip to content

Commit 2da4c1d

Browse files
middtmiddt
committed
Add ClickHouse state store documentation
- Add comprehensive documentation for ClickHouse state store component - Include setup instructions for self-hosted, Kubernetes, and cloud deployments - Document supported features: CRUD, ETag, and TTL support - Add advanced configuration options and performance considerations - Update component registry with ClickHouse entry in generic.yaml Signed-off-by: Mehmet TOSUN <[email protected]>
1 parent 68f1be7 commit 2da4c1d

File tree

2 files changed

+215
-0
lines changed

2 files changed

+215
-0
lines changed

‎daprdocs/content/en/reference/components-reference/supported-state-stores/setup-clickhouse.md

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
---
2+
type: docs
3+
title: "ClickHouse"
4+
linkTitle: "ClickHouse"
5+
description: Detailed information on the ClickHouse state store component
6+
aliases:
7+
- "/operations/components/setup-state-store/supported-state-stores/setup-clickhouse/"
8+
---
9+
10+
## Component format
11+
12+
To setup ClickHouse state store create a component of type `state.clickhouse`. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration.
13+
14+
```yaml
15+
apiVersion: dapr.io/v1alpha1
16+
kind: Component
17+
metadata:
18+
name: <NAME>
19+
spec:
20+
type: state.clickhouse
21+
version: v1
22+
metadata:
23+
- name: clickhouseURL
24+
value: <CONNECTION_URL>
25+
- name: databaseName
26+
value: <DATABASE_NAME>
27+
- name: tableName
28+
value: <TABLE_NAME>
29+
- name: username # Optional
30+
value: <USERNAME>
31+
- name: password # Optional
32+
value: <PASSWORD>
33+
# Uncomment this if you wish to use ClickHouse as a state store for actors (optional)
34+
#- name: actorStateStore
35+
# value: "true"
36+
```
37+
38+
{{% alert title="Warning" color="warning" %}}
39+
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
40+
{{% /alert %}}
41+
42+
If you wish to use ClickHouse as an actor store, append the following to the yaml.
43+
44+
```yaml
45+
- name: actorStateStore
46+
value: "true"
47+
```
48+
49+
## Spec metadata fields
50+
51+
| Field | Required | Details | Example |
52+
|--------------------|:--------:|---------|---------|
53+
| clickhouseURL | Y | Connection URL for the ClickHouse server | `"clickhouse://localhost:9000"`, `"clickhouse://clickhouse-server:9000"` |
54+
| databaseName | Y | Name of the database to use | `"dapr_state"`, `"my_database"` |
55+
| tableName | Y | Name of the table to store state data | `"state_table"`, `"dapr_state_store"` |
56+
| username | N | Username for ClickHouse authentication. Can be `secretKeyRef` to use a secret reference | `"default"`, `"my_user"` |
57+
| password | N | Password for ClickHouse authentication. Can be `secretKeyRef` to use a secret reference | `"my_password"` |
58+
| actorStateStore | N | Consider this state store for actors. Defaults to `"false"` | `"true"`, `"false"` |
59+
60+
## Setup ClickHouse
61+
62+
Dapr can use any ClickHouse instance: containerized, running on your local dev machine, or a managed cloud service.
63+
64+
{{< tabs "Self-Hosted" "Kubernetes" "Cloud" >}}
65+
66+
{{% codetab %}}
67+
68+
1. Run an instance of ClickHouse. You can run a local instance of ClickHouse in Docker with the following command:
69+
70+
```bash
71+
docker run -d --name clickhouse-server \
72+
-p 8123:8123 -p 9000:9000 \
73+
-e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 \
74+
-e CLICKHOUSE_PASSWORD=my_password \
75+
clickhouse/clickhouse-server
76+
```
77+
78+
2. Create a database for state data (optional, as Dapr will create it automatically):
79+
80+
```sql
81+
CREATE DATABASE IF NOT EXISTS dapr_state;
82+
```
83+
84+
{{% /codetab %}}
85+
86+
{{% codetab %}}
87+
88+
You can use [Helm](https://helm.sh/) to quickly create a ClickHouse instance in your Kubernetes cluster. This approach requires [Installing Helm](https://github.com/helm/helm#install).
89+
90+
1. Add the ClickHouse Helm repository:
91+
```bash
92+
helm repo add clickhouse https://docs.altinity.com/clickhouse-operator/
93+
helm repo update
94+
```
95+
96+
2. Install ClickHouse into your cluster:
97+
```bash
98+
helm install clickhouse clickhouse/clickhouse
99+
```
100+
101+
3. Run `kubectl get pods` to see the ClickHouse containers now running in your cluster.
102+
103+
4. Add the ClickHouse service endpoint as the `clickhouseURL` in your component configuration. For example:
104+
```yaml
105+
metadata:
106+
- name: clickhouseURL
107+
value: "clickhouse://clickhouse:9000"
108+
```
109+
110+
{{% /codetab %}}
111+
112+
{{% codetab %}}
113+
114+
ClickHouse is available as a managed service from various cloud providers:
115+
116+
- [ClickHouse Cloud](https://clickhouse.com/cloud)
117+
- [Altinity.Cloud](https://altinity.com/cloud-database/)
118+
- [Yandex Managed Service for ClickHouse](https://cloud.yandex.com/services/managed-clickhouse)
119+
120+
When using a managed service, ensure you have the correct connection URL, database name, and credentials configured in your component metadata.
121+
122+
{{% /codetab %}}
123+
124+
{{< /tabs >}}
125+
126+
## Features
127+
128+
The ClickHouse state store supports the following features:
129+
130+
### ETags
131+
132+
The ClickHouse state store supports [ETags]({{< ref state-management-overview.md >}}) for optimistic concurrency control. ETags are automatically generated and updated when state data is modified.
133+
134+
### TTL (Time-To-Live)
135+
136+
This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to indicate after how many seconds the data should be considered "expired".
137+
138+
Example of setting TTL:
139+
140+
```json
141+
{
142+
"key": "my-key",
143+
"value": "my-value",
144+
"metadata": {
145+
"ttlInSeconds": "3600"
146+
}
147+
}
148+
```
149+
150+
Records with expired TTLs are automatically filtered out during read operations and are eligible for cleanup by ClickHouse's background processes.
151+
152+
## Advanced
153+
154+
### Table Schema
155+
156+
The ClickHouse state store creates a table with the following schema:
157+
158+
```sql
159+
CREATE TABLE IF NOT EXISTS <database>.<table> (
160+
key String,
161+
value String,
162+
etag String,
163+
expire DateTime64(3) NULL,
164+
PRIMARY KEY(key)
165+
) ENGINE = ReplacingMergeTree()
166+
ORDER BY key
167+
```
168+
169+
The table uses ClickHouse's `ReplacingMergeTree` engine, which automatically deduplicates rows with the same primary key during background merges.
170+
171+
### Connection URL Format
172+
173+
The ClickHouse connection URL follows the standard format:
174+
175+
```
176+
clickhouse://[username[:password]@]host[:port][/database][?param1=value1&...&paramN=valueN]
177+
```
178+
179+
Examples:
180+
- `clickhouse://localhost:9000`
181+
- `clickhouse://user:password@clickhouse-server:9000/my_db`
182+
- `clickhouse://localhost:9000?dial_timeout=10s&max_execution_time=60`
183+
184+
### Performance Considerations
185+
186+
- The ClickHouse state store is optimized for high-throughput scenarios
187+
- For better performance with large datasets, consider partitioning your table by date or other relevant columns
188+
- The `ReplacingMergeTree` engine provides eventual consistency for duplicate key handling
189+
- Background merges in ClickHouse will automatically clean up old versions of updated records
190+
191+
### Bulk Operations
192+
193+
The ClickHouse state store supports bulk operations for improved performance:
194+
195+
- `BulkGet`: Retrieve multiple keys in a single operation
196+
- `BulkSet`: Store multiple key-value pairs in a single operation
197+
- `BulkDelete`: Delete multiple keys in a single operation
198+
199+
## Related links
200+
201+
- [Basic schema for a Dapr component]({{< ref component-schema >}})
202+
- Read [this guide]({{< ref "howto-get-save-state.md#step-2-save-and-retrieve-a-single-state" >}}) for instructions on configuring state store components
203+
- [State management building block]({{< ref state-management >}})
204+
- [ClickHouse Official Documentation](https://clickhouse.com/docs)

‎daprdocs/data/components/state_stores/generic.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,17 @@
2020
etag: false
2121
ttl: true
2222
query: false
23+
- component: ClickHouse
24+
link: setup-clickhouse
25+
state: Alpha
26+
version: v1
27+
since: "1.15"
28+
features:
29+
crud: true
30+
transactions: false
31+
etag: true
32+
ttl: true
33+
query: false
2334
- component: CockroachDB
2435
link: setup-cockroachdb
2536
state: Stable

0 commit comments

Comments
 (0)