You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/guides/51-access-data-lake/02-iceberg.md
+77Lines changed: 77 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -281,6 +281,83 @@ CONNECTION=(
281
281
|`s3.disable-ec2-metadata`| Option to disable loading credentials from EC2 metadata (typically used with `s3.allow-anonymous`). |
282
282
|`s3.disable-config-load`| Option to disable loading configuration from config files and environment variables. |
283
283
284
+
### Catalog Types
285
+
286
+
Databend supports four types of Iceberg catalogs:
287
+
288
+
- REST Catalog
289
+
290
+
REST catalog uses a RESTful API approach to interact with Iceberg tables.
291
+
292
+
```sql
293
+
CREATE CATALOG iceberg_rest TYPE = ICEBERG CONNECTION = (
294
+
TYPE ='rest'
295
+
ADDRESS ='http://localhost:8181'
296
+
warehouse ='s3://warehouse/demo/'
297
+
"s3.endpoint"='http://localhost:9000'
298
+
"s3.access-key-id"='admin'
299
+
"s3.secret-access-key"='password'
300
+
"s3.region"='us-east-1'
301
+
)
302
+
303
+
- AWS Glue Catalog
304
+
For Glue catalogs, the configuration includes both Glue service parameters and storage (S3) parameters. The Glue service parameters appear first, followed by the S3 storage parameters (prefixed with "s3.").
305
+
306
+
```sql
307
+
CREATE CATALOG iceberg_glue TYPE = ICEBERG CONNECTION = (
308
+
TYPE = 'glue'
309
+
ADDRESS = 'http://localhost:5000'
310
+
warehouse = 's3a://warehouse/glue/'
311
+
"aws_access_key_id" = 'my_access_id'
312
+
"aws_secret_access_key" = 'my_secret_key'
313
+
"region_name" = 'us-east-1'
314
+
"s3.endpoint" = 'http://localhost:9000'
315
+
"s3.access-key-id" = 'admin'
316
+
"s3.secret-access-key" = 'password'
317
+
"s3.region" = 'us-east-1'
318
+
)
319
+
```
320
+
321
+
- Storage Catalog (S3Tables Catalog)
322
+
323
+
The Storage catalog requires a table_bucket_arn parameter. Unlike other buckets, S3Tables bucket is not a physical bucket, but a virtual bucket that is managed by S3Tables. You cannot directly access the bucket with a pathlike`s3://{bucket_name}/{file_path}`. All operations are performed with respect to the bucket ARN.
324
+
325
+
Properties Parameters
326
+
The following properties are available for the catalog:
327
+
328
+
```
329
+
profile_name: The name of the AWS profile to use.
330
+
region_name: The AWS region to use.
331
+
aws_access_key_id: The AWS access key ID to use.
332
+
aws_secret_access_key: The AWS secret access key to use.
333
+
aws_session_token: The AWS session token to use.
334
+
```
335
+
336
+
```sql
337
+
CREATE CATALOG iceberg_storage TYPE = ICEBERG CONNECTION = (
338
+
TYPE = 'storage'
339
+
ADDRESS = 'http://localhost:9111'
340
+
"table_bucket_arn" = 'my-bucket'
341
+
-- Additional properties as needed
342
+
)
343
+
```
344
+
345
+
- Hive Catalog (HMS Catalog)
346
+
347
+
The Hive catalog requires an ADDRESS parameter, which is the address of the Hive metastore. It also requires a warehouse parameter, which is the location of the Iceberg warehouse, usually an S3 bucket or compatible object storage system.
348
+
349
+
```sql
350
+
CREATE CATALOG iceberg_hms TYPE = ICEBERG CONNECTION = (
351
+
TYPE ='hive'
352
+
ADDRESS ='192.168.10.111:9083'
353
+
warehouse ='s3a://warehouse/hive/'
354
+
"s3.endpoint"='http://localhost:9000'
355
+
"s3.access-key-id"='admin'
356
+
"s3.secret-access-key"='password'
357
+
"s3.region"='us-east-1'
358
+
)
359
+
```
360
+
284
361
### SHOW CREATE CATALOG
285
362
286
363
Returns the detailed configuration of a specified catalog, including its type and storage parameters.
The DECODE function compares the select expression to each search expression in order. As soon as a search expression matches the selection expression, the corresponding result expression is returned. If no match is found and a default value is provided, the default value is returned.
-`expr`: The "select expression" that is compared against each search expression. This is typically a column, but can be a subquery, literal, or other expression.
16
+
-`searchN`: The search expressions to compare against the select expression. If a match is found, the corresponding result is returned.
17
+
-`resultN`: The values that will be returned if the corresponding search expression matches the select expression.
18
+
-`default`: Optional. If provided and no search expression matches, this default value is returned.
19
+
20
+
## Usage Notes
21
+
22
+
- Unlike `CASE`, a NULL value in the select expression matches a NULL value in the search expressions.
23
+
- If multiple search expressions would match, only the first match's result is returned.
24
+
25
+
## Examples
26
+
27
+
```sql
28
+
CREATETABLEt (a VARCHAR);
29
+
INSERT INTO t (a) VALUES
30
+
('1'),
31
+
('2'),
32
+
(NULL),
33
+
('4');
34
+
```
35
+
36
+
Example with a default value 'other' (note that NULL equals NULL):
0 commit comments