Feature Request: DuckDB support

[mettekou]

Currently Data Wrangler does not support DuckDBPyRelation objects. It is therefore necessary to convert these objects into either Polars or Pandas DataFrame objects, which is inconvenient. It should instead be possible to inspect a DuckDBPyRelation object in Data Wrangler directly.

[pwang347]

Hi @mettekou, thank you for opening this feature request! Out of curiosity, are you looking for Data Wrangler to automatically convert to Pandas for convenience or are you looking for native support for DuckDBPyRelation objects (ie., not using Pandas)?

Also, what entrypoints are you currently using to launch Data Wrangler? Note that to support new data types, some of the entrypoints will also need change from the Jupyter extension.

Thanks!

[mettekou]

Hi @mettekou, thank you for opening this feature request!

@pwang347 Thank you for your work on Data Wrangler, which I thoroughly enjoy otherwise!

Out of curiosity, are you looking for Data Wrangler to automatically convert to Pandas for convenience or are you looking for native support for DuckDBPyRelation objects (ie., not using Pandas)?

I personally and professionally only use DuckDB and Polars, so I am looking for native support of DuckDBPyRelation objects. While Data Wrangler is already a step up from plain Python scripts and Jupyter Notebooks as-is, this addition could massively improve data science workflows with VS Code and draw in many more users. For instance, people now using SAS through SAS Studio could easily switch to Python with VS Code and Data Wrangler, if only it supported DuckDB, since that allows them to deal with data which does not fit in memory. This is incidentally the reason why going through Pandas does not work for me.

Also, what entrypoints are you currently using to launch Data Wrangler? Note that to support new data types, some of the entrypoints will also need change from the Jupyter extension.

I am using Data Wrangler from a Jupyter Notebook, through the shortcut which appears below a cell when the last expression in it is a variable of a compatible data type. Note that I also use DuckDB with JupySQL. Forgive my ignorance, but I do not think Data Wrangler presently supports SQL cells. To make Data Wrangler even better, you could, for instance, make this shortcut appear for a SQL cell containing a CREATE TABLE DDL statement.

It would also be nice to open DuckDB database files with Data Wrangler and I think this should be a nice challenge to you, since DuckDB database files can contain multiple tables. Perhaps you could also take a look at integrating Data Wrangler with SQLTools for this use case. It is early days for DuckDB support for SQLTools, so it may be the ideal time to tackle this integration.

[pwang347]

@mettekou that makes sense, thank you for providing a detailed response and for sharing more context about your usage. Also, glad to hear that you're enjoying Data Wrangler!

You're correct that Data Wrangler does not currently support SQL cells or SQL scenarios in general. We've added the items you mentioned to our backlog to followup for investigation and will let you know whenever we have any updates to share.

Thanks!

[mettekou]

@mettekou that makes sense, thank you for providing a detailed response and for sharing more context about your usage. Also, glad to hear that you're enjoying Data Wrangler!

You're correct that Data Wrangler does not currently support SQL cells or SQL scenarios in general. We've added the items you mentioned to our backlog to followup for investigation and will let you know whenever we have any updates to share.

Thanks!

@pwang347 @fubits1 Another source to draw inspiration from for the Jupyter Notebook experience: https://duckdb.org/2025/03/12/duckdb-ui.html.

[mettekou]

@mettekou that makes sense, thank you for providing a detailed response and for sharing more context about your usage. Also, glad to hear that you're enjoying Data Wrangler!
You're correct that Data Wrangler does not currently support SQL cells or SQL scenarios in general. We've added the items you mentioned to our backlog to followup for investigation and will let you know whenever we have any updates to share.
Thanks!

@pwang347 @fubits1 Another source to draw inspiration from for the Jupyter Notebook experience: https://duckdb.org/2025/03/12/duckdb-ui.html.

@pwang347 Not quite the experience I'm looking for, since it reinvents Jupyter Notebooks, but another possible source of inspiration for DuckDB support in Data Wrangler: https://github.com/sahilng/ducknb.

[ScottWilliamAnderson]

I agree! I'd find this useful for all the same use cases @mettekou shared. Data Wrangler is excellent but seems to be lacking this one main feature.