# parquet

> parquet, a magmalake tin: The first native Apache Parquet reader and writer in Mojo.

[magmalake](https://magmalake.dev/) [parquet](https://magmalake.dev/s/parquet)

The first native Apache Parquet reader and writer in Mojo. It decodes the footer, the page headers, the levels and the values itself, and hands the result back in Arrow memory layout, exportable over the Arrow C Data Interface. That interface is the boundary, not a full Arrow library: there are no compute kernels, no IPC and no Flight.

Install

```
pixi shelf add parquet-mojo
```

Version

`0.11.7` on mojoshelf

Source

[GitHub](https://github.com/magmalake/parquet.mojo) [README](https://github.com/magmalake/parquet.mojo#readme) [Benchmark history](https://magmalake.github.io/parquet.mojo/benchmarks/)

## Works today

- Every physical and logical type, every encoding, v1 and v2 pages
- UNCOMPRESSED, SNAPPY and GZIP with no FFI; ZSTD, BROTLI and LZ4 with parquet-full
- Nested lists, maps and structs reconstructed from definition and repetition levels
- Row-group, page-index and bloom-filter pruning; field-id projection for Iceberg
- Arrow C Data Interface export, verified by pyarrow importing the arrays

## Not yet

- More than 2 GiB of string data in one Parquet column chunk

Checked against pyarrow, value-exact on 33 fixtures; pyarrow reads back every file it writes

Value-exact against pyarrow on 33 fixtures — every value of every column — and pyarrow reads back every file magmalake writes.

The Arrow export imports through pyarrow.Array.\_import\_from\_c.

### Building it found bugs upstream

Holding three implementations against each other surfaces disagreements, and not all of them turned out to be my fault. I cut each one below down to a standalone script or unit test with no magmalake code in it, then either reported it upstream or — where someone had got there first — matched it to the report that already existed.

- [Apache Arrow apache/arrow#51097  reported](https://github.com/apache/arrow/issues/51097) The Parquet writer undercounts nulls for a fixed-width leaf under list<struct> when some lists are null or empty. The BYTE\_ARRAY leaf beside it in the same struct is right, so two leaves disagree about the same level records.

- [parquet-full](https://magmalake.dev/s/parquet-full) parquet · plus ZSTD, BROTLI and LZ4
- [avro](https://magmalake.dev/s/avro) object container files, both ways
- [arrow-mlake](https://magmalake.dev/s/arrow-mlake) Arrow layout · C Data Interface · IPC
- [roaring](https://magmalake.dev/s/roaring) 32/64-bit · deletion vectors
- [Every tin](https://magmalake.dev/tins)

[A magmalake project](https://magmalake.dev/)

Apache 2.0 · © 2026 Marius Seritan