Skip to content

[FLINK-37658][docs] Improve the Datagen connector docs #26455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 21, 2025
Merged

[FLINK-37658][docs] Improve the Datagen connector docs #26455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4f514b6
FLINK-37658: Improve the Datagen connector docs
davidradl Apr 15, 2025
3818b21
Update docs/content.zh/docs/connectors/table/datagen.md
davidradl Apr 18, 2025
c138a47
Update docs/content.zh/docs/connectors/table/datagen.md
davidradl Apr 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions docs/content.zh/docs/connectors/table/datagen.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,21 @@ CREATE TABLE Orders (
)
```

对于集合类型，你可以指定集合的大小（元素个数）。

```sql
CREATE TABLE Orders (
f0 Array<INT>,
f1 Map<INT, STRING>,
f2 MULTISET<INT>
) WITH (
'connector' = 'datagen',
'fields.f0.length' = '10',
'fields.f1.length' = '11',
'fields.f2.length' = '12'
);
```

字段类型
-----

Expand Down Expand Up @@ -285,21 +300,21 @@ CREATE TABLE Orders (
<td>可选</td>
<td style="word-wrap: break-word;">(Minimum value of type)</td>
<td>(Type of field)</td>
<td>随机生成器的最小值，适用于数字类型。</td>
<td>随机生成器的最小值，仅适用于数字类型。</td>
</tr>
<tr>
<td><h5>fields.#.max</h5></td>
<td>可选</td>
<td style="word-wrap: break-word;">(Maximum value of type)</td>
<td>(Type of field)</td>
<td>随机生成器的最大值，适用于数字类型。</td>
<td>随机生成器的最大值，仅适用于数字类型。</td>
</tr>
<tr>
<td><h5>fields.#.max-past</h5></td>
<td>可选</td>
<td style="word-wrap: break-word;">0</td>
<td>Duration</td>
<td>随机生成器生成相对当前时间向过去偏移的最大值，适用于 timestamp 类型。</td>
<td>对于 string/bytes 类型为 100，对于 array/map/multiset 类型为 3。</td>
</tr>
<tr>
<td><h5>fields.#.length</h5></td>
Expand Down Expand Up @@ -337,7 +352,7 @@ CREATE TABLE Orders (
<tr>
<td><h5>fields.#.null-rate</h5></td>
<td>optional</td>
<td style="word-wrap: break-word;">(none)</td>
<td style="word-wrap: break-word;">0</td>
<td>(Type of field)</td>
<td>空值比例。</td>
</tr>
Expand Down
31 changes: 23 additions & 8 deletions docs/content/docs/connectors/table/datagen.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,21 @@ CREATE TABLE Orders (
)
```

And for collections it is possible to specify different sized collections.

```sql
CREATE TABLE Orders (
f0 Array<INT>,
f1 Map<INT, STRING>,
f2 MULTISET<INT>
) WITH (
'connector' = 'datagen',
'fields.f0.length' = '10',
'fields.f1.length' = '11',
'fields.f2.length' = '12'
);
```

Types
-----

Expand Down Expand Up @@ -289,14 +304,14 @@ Connector Options
<td>optional</td>
<td style="word-wrap: break-word;">(Minimum value of type)</td>
<td>(Type of field)</td>
<td>Minimum value of random generator, work for numeric types.</td>
<td>Minimum value of random generator, only works for numeric types.</td>
</tr>
<tr>
<td><h5>fields.#.max</h5></td>
<td>optional</td>
<td style="word-wrap: break-word;">(Maximum value of type)</td>
<td>(Type of field)</td>
<td>Maximum value of random generator, work for numeric types.</td>
<td>Maximum value of random generator, only works for numeric types.</td>
</tr>
<tr>
<td><h5>fields.#.max-past</h5></td>
Expand All @@ -308,21 +323,21 @@ Connector Options
<tr>
<td><h5>fields.#.length</h5></td>
<td>optional</td>
<td style="word-wrap: break-word;">100</td>
<td style="word-wrap: break-word;">100 for string/bytes, 3 for array/map/multiset </td>
<td>Integer</td>
<td>
Size or length of the collection for generating varchar/varbinary/string/bytes/array/map/multiset types.
Please notice that for variable-length fields (varchar/varbinary), the default length is defined by the schema and cannot be set to a length greater than it.
for super-long fields (string/bytes), the default length is 100 and can be set to a length less than 2^31.
for constructed fields (array/map/multiset), the default number of elements is 3 and can be customized.
Please note that for variable-length fields (varchar/varbinary), the default length is defined by the schema and cannot be set to a length greater than it.
For super-long fields (string/bytes), the default length is 100 and can be set to a length less than 2^31.
For constructed fields (array/map/multiset), the default number of elements is 3.
</td>
</tr>
<tr>
<td><h5>fields.#.var-len</h5></td>
<td>optional</td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
<td>Whether to generate a variable-length data, please notice that it should only be used for variable-length types (varchar, string, varbinary, bytes).</td>
<td>Whether to generate a variable-length data, only works for variable-length types (varchar, string, varbinary, bytes).</td>
</tr>
<tr>
<td><h5>fields.#.start</h5></td>
Expand All @@ -341,7 +356,7 @@ Connector Options
<tr>
<td><h5>fields.#.null-rate</h5></td>
<td>optional</td>
<td style="word-wrap: break-word;">(none)</td>
<td style="word-wrap: break-word;">0</td>
<td>(Type of field)</td>
<td>The proportion of null values.</td>
</tr>
Expand Down