Skip to content

Commit ac5eba7

Browse files
author
Lauren
committed
CDC Updates
Updates to alpha docs based on user feedback
1 parent 7e3f521 commit ac5eba7

File tree

1 file changed

+34
-10
lines changed

1 file changed

+34
-10
lines changed

v2.1/change-data-capture.md

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,19 @@ The core feature of CDC is the [changefeed](create-changefeed.html). Changefeeds
3030
- Rows are sharded between Kafka partitions by the row’s [primary key](primary-key.html).
3131

3232
- The `WITH timestamps` option adds an **update timestamp** to each emitted row. It also causes periodic **resolved timestamp** messages to be emitted to each Kafka partition. A resolved timestamp is a guarantee that no (previously unseen) rows with a lower update timestamp will be emitted on that partition.
33+
For example:
34+
35+
~~~ json
36+
{"__crdb__": {"updated": "1532377312562986715.0000000000"}, "id": 1, "name": "Petee H"}
37+
{"__crdb__": {"updated": "1532377306108205142.0000000000"}, "id": 2, "name": "Carl"}
38+
{"__crdb__": {"updated": "1532377358501715562.0000000000"}, "id": 3, "name": "Ernie"}
39+
{"__crdb__":{"resolved":"1532379887442299001.0000000000"}}
40+
{"__crdb__":{"resolved":"1532379888444290910.0000000000"}}
41+
{"__crdb__":{"resolved":"1532379889448662988.0000000000"}}
42+
...
43+
{"__crdb__":{"resolved":"1532379922512859361.0000000000"}}
44+
{"__crdb__": {"updated": "1532379923319195777.0000000000"}, "id": 4, "name": "Lucky"}
45+
~~~
3346

3447
- Cross-row and cross-table order guarantees are not directly given. However, the resolved timestamp notifications on every Kafka partition can be used to provide strong ordering and global consistency guarantees by buffering records in between timestamp closures.
3548

@@ -94,7 +107,7 @@ In this example, you'll set up a changefeed for a single-node cluster that is co
94107
$ ./cockroach start --insecure
95108
~~~
96109

97-
2. Download and extract the [Confluent platform](https://www.confluent.io/download/) (which includes Kafka).
110+
2. Download and extract the [Confluent Open Source platform](https://www.confluent.io/download/) (which includes Kafka).
98111

99112
3. Start Confluent:
100113

@@ -105,28 +118,39 @@ In this example, you'll set up a changefeed for a single-node cluster that is co
105118

106119
Only `zookeeper` and `kafka` are needed. To troubleshoot Confluent, see [their docs](https://docs.confluent.io/current/installation/installing_cp.html#zip-and-tar-archives).
107120

108-
4. As the `root` user, open the [built-in SQL client](use-the-built-in-sql-client.html):
121+
4. Create a Kafka topic:
122+
123+
{% include copy-clipboard.html %}
124+
~~~
125+
$ ./confluent-4.0.0/bin/kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic office_dogs
126+
~~~
127+
128+
{{site.data.alerts.callout_info}}
129+
You are expected to create any Kafka topics with the necessary number of replications and partitions. [Topics can be created manually](https://kafka.apache.org/documentation/#basic_ops_add_topic) or [Kafka brokers can be configured to automatically create topics](https://kafka.apache.org/documentation/#topicconfigs) with a default partition count and replication factor.
130+
{{site.data.alerts.end}}
131+
132+
5. As the `root` user, open the [built-in SQL client](use-the-built-in-sql-client.html):
109133

110134
{% include copy-clipboard.html %}
111135
~~~ shell
112136
$ cockroach sql --insecure
113137
~~~
114138

115-
5. Create a database called `test`:
139+
6. Create a database called `test`:
116140

117141
{% include copy-clipboard.html %}
118142
~~~ sql
119143
> CREATE DATABASE cdc_demo;
120144
~~~
121145

122-
6. Set the database as the default:
146+
7. Set the database as the default:
123147

124148
{% include copy-clipboard.html %}
125149
~~~ sql
126150
> SET DATABASE = cdc_demo;
127151
~~~
128152

129-
7. Create a table and add data:
153+
8. Create a table and add data:
130154

131155
{% include copy-clipboard.html %}
132156
~~~ sql
@@ -147,7 +171,7 @@ In this example, you'll set up a changefeed for a single-node cluster that is co
147171
> UPDATE office_dogs SET name = 'Petee H' WHERE id = 1;
148172
~~~
149173

150-
8. Start the changefeed:
174+
9. Start the changefeed:
151175

152176
{% include copy-clipboard.html %}
153177
~~~ sql
@@ -164,7 +188,7 @@ In this example, you'll set up a changefeed for a single-node cluster that is co
164188

165189
This will start up the changefeed in the background and return the `job_id`. The changefeed writes to Kafka.
166190

167-
9. In a new terminal, start watching the Kafka topic:
191+
10. In a new terminal, start watching the Kafka topic:
168192

169193
{% include copy-clipboard.html %}
170194
~~~ shell
@@ -177,22 +201,22 @@ In this example, you'll set up a changefeed for a single-node cluster that is co
177201

178202
Note that the initial scan displays the state of the table as of when the changefeed started (therefore, the initial value of `"Petee"` is missing).
179203

180-
10. Back in the SQL client, insert more data:
204+
11. Back in the SQL client, insert more data:
181205

182206
{% include copy-clipboard.html %}
183207
~~~ sql
184208
> INSERT INTO office_dogs VALUES (3, 'Ernie');
185209
~~~
186210

187-
11. Back in the terminal where you're watching the Kafka topic, the following output has appeared:
211+
12. Back in the terminal where you're watching the Kafka topic, the following output has appeared:
188212
189213
~~~
190214
{"id": 3, "name": "Ernie"}
191215
~~~
192216
193217
## Known limitations
194218
195-
The following are limitations in July 2, 2018 alpha release, and will be addressed before the v2.1 release.
219+
The following are limitations in July 30, 2018 alpha release, and will be addressed before the v2.1 release.
196220
197221
- Changefeeds created with the alpha may not be compatible with future alphas and the final v2.1 release.
198222

0 commit comments

Comments
 (0)