Tins · File formats
parquet-full
Every Parquet codec the spec defines: parquet-mojo, plus ZSTD, BROTLI and LZ4 through libzstd, libbrotli and liblz4. It lives in the full/ directory of the parquet repository and depends on parquet-mojo, so adding it gives you both, and the import is unchanged.
What it does
Works today
- Take it unless you control the files you read: ZSTD is what most Parquet in the wild uses, Iceberg included
- A reader without it that meets one of these codecs raises an error naming it
- A consumer that never needs ZSTD never builds the three C libraries
Correctness
Checked against A write and read round trip through each of the three codecs, and the ZSTD-compressed Iceberg fixtures
Also in File formats
- parquet read · write · every encoding · Arrow out
- avro object container files, both ways
- arrow-mlake Arrow layout · C Data Interface · IPC
- roaring 32/64-bit · deletion vectors
- Every tin