Interacting with Parquet Files

[bcrotty]

I have a data loader that produces a file large enough that loading seemed to time out when created as a CSV. Following Fil's advice here, I switched the data loader to create a parquet. I can use the parquet directly in something like Inputs.table, but I need to do some transformation on the data along with another file.

When I load the file as a parquet, it does not have any of the JS array functions like filter. How can I convert the parquet to an array or where can I see what functions the parquet object provides natively?

[Fil]

If you load it with the FileAttachment("").parquet() method, the result is an arrow table; see
https://observablehq.com/framework/lib/arrow for details.

You can transform this into a normal JS array by calling [...values], but if the number of rows is huge, it's not going to be efficient, and I'd recommend using tools that work directly with arrow tables, such as arquero or DuckDB.