Skip to content

feat: add JSON lines, Arrow IPC stream, and Parquet output formats #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
esadek wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add JSON lines, Arrow IPC stream, and Parquet output formats #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 200 additions & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ arrow-schema = "58.1.0"
clap = { version = "4.6.1", features = ["derive"] }
comfy-table = "7.2.2"
nu-ansi-term = "0.50.3"
parquet = "58.1.0"
reedline = "0.47.0"
syntect = "5.3.0"
terminal-colorsaurus = "1.0.3"
Expand Down
5 changes: 4 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ A command-line tool for querying databases via [ADBC](https://arrow.apache.org/a
- **Interactive SQL shell** - Execute SQL queries with command history and intuitive navigation
- **Syntax highlighting** - SQL queries highlighted for improved readability
- **Formatted output** - Results displayed in clean, aligned tables with dynamic column width
- **File export** - Export query results to JSON, CSV, or Arrow IPC files
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files

@ianmcook ianmcook Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, or Parquet files

- **Fast and lightweight** - Built in Rust for high performance and minimal resource usage

## Installation
Expand Down Expand Up @@ -101,8 +101,11 @@ Execute a query and output the result to a file:

```sh
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.json
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.jsonl
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.csv
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.arrow
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.arrows
databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.parquet
```

## Reference
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@ databow is a command-line tool for querying databases.
- **Interactive SQL shell** - Execute SQL queries with command history and intuitive navigation
- **Syntax highlighting** - SQL queries highlighted for improved readability
- **Formatted output** - Results displayed in clean, aligned tables with dynamic column width
- **File export** - Export query results to JSON, CSV, or Arrow IPC files
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files

@ianmcook ianmcook Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's OK to be less pedantic about IPC stream vs. IPC file here and in the README

Suggested change
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files
- **File export** - Export query results to JSON, JSON lines, CSV, Arrow IPC, or Parquet files

- **Fast and lightweight** - Built in Rust for high performance and minimal resource usage
13 changes: 8 additions & 5 deletions docs/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,11 +108,14 @@ databow --driver duckdb --query "SELECT 42 AS the_answer" --output result.json

The output format is inferred from the file extension:

| Extension | Format |
|-----------------|-----------|
| `.json` | JSON |
| `.csv` | CSV |
| `.arrow`, `.ipc`| Arrow IPC |
| Extension | Format |
|-----------------|------------------|
| `.json` | JSON |
| `.jsonl` | JSON lines |
| `.csv` | CSV |
| `.arrow`, `.ipc`| Arrow IPC (file) |

@ianmcook ianmcook Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The naming conventions of the Rust library might have you believing otherwise, but unmodified "Arrow IPC" does not imply the file format, so I'd recommend dropping the parens here

Suggested change
| `.arrow`, `.ipc`| Arrow IPC (file) |
| `.arrow`, `.ipc`| Arrow IPC file |

| `.arrows` | Arrow IPC stream |
| `.parquet` | Parquet |

## --help

Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ $ databow --profile warehouse --file query.sql
└───────────────────┘
```

Instead of printing query results to stdout, the [`--output` argument](/reference/#-output) can be used to write results to JSON, CSV, or Arrow IPC files:
Instead of printing query results to stdout, the [`--output` argument](/reference/#-output) can be used to write results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files:

@ianmcook ianmcook Aug 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Instead of printing query results to stdout, the [`--output` argument](/reference/#-output) can be used to write results to JSON, JSON lines, CSV, Arrow IPC, Arrow IPC stream, or Parquet files:
Instead of printing query results to stdout, the [`--output` argument](/reference/#-output) can be used to write results to JSON, JSON lines, CSV, Arrow IPC file, Arrow IPC stream, or Parquet files:


```console
$ databow --profile warehouse --query "SELECT * FROM penguins" --output penguins.csv
Expand Down
Loading
Loading