[Parquet] pq.write_table fails with OSError: Lz4 compression failure.

[skaae]

Describe the bug, including details regarding any error messages, version, and platform.

I encountered a table with where pq.write_table throws compression error:

pq.write_table(table, "foo", compression='lz4')
*** OSError: Lz4 compression failure.

using compression='gzip' works.

The minimal table I was able to find contains 42 rows with a single columns of strings. Each string is 30-60 mb.
The strings in the table are xml and can be compressed to ~100mb in total.

I can share the table, but would like to not do it publicly.

Python 3.12.3
pyarrow 23.0.1
lz4 4.4.5
ubuntu 24.04

Component(s)

Parquet