Measured against Ray and Rust, including the runs it lost

Distributed, 3 nodes ms, the eight queries, median pass

Three worker nodes, 16 task slots each, one task per data file, in two runs on separate clusters the same day. With 2 passes the median is the mean of both, so the slow first pass counts in it, and Rust and Ray lose more on that pass than Dista; with 5 the median is a warm pass. Even warm, Rust and Ray were slower on the first cluster: the clusters' own variation. Each query is from the 5-pass run. Every answer matches across the three frameworks: row counts exactly, sums to floating-point rounding. Results: matrix-2026-11-01 and gate-2026-11-01-readpath

Distributed, 3, 6 and 12 nodes ms, median of 5 passes

The eight queries' total on 3, 6 and 12 worker nodes; answers identical across every run. On 12 nodes, each framework's best of three runs: Ray's moves by up to 50% between runs. Results: matrix-2026-11-01

One process, 16 threads ms, the eight queries

The head node alone, no workers, every framework on its 16 cores: Dista's scan with 16 workers, Rust's multi-threaded tokio runtime with one task per core, pyarrow's default thread pools. Answers identical across the three. Rust's numbers are from local-2026-11-01-2, a re-run. Results: local-2026-11-01

One process, one thread, 8 files' reads in flight ms, the eight queries

One decoding thread each, with up to 8 files' reads in flight: Dista's files_at_once, iceberg-rust's concurrency limits on a current-thread runtime, pyarrow's I/O pool next to one CPU thread. Dista is level with Rust and 15% behind Ray; what's left is decode speed on one core. Results: local-2026-11-01

The second benchmark: a shuffle

The taxi queries spend most of their time reading Parquet. This one spends it in user code and in moving data between tasks: every trip grouped by pickup zone, dropoff zone and hour of day (1,044,000 groups), each group adding up its trips, speeds, tip ratios and fares. Map tasks read one data file each and split their groups by key into one partition per reducer; each reducer takes its partition from every map task and adds them up. Dista and Ray only, on the same table and nodes, from 2 November 2026. The query and its answer.

The shuffle, distributed on 3, 6 and 12 nodes ms, median of passes

One map task per data file, which splits its groups into one partition per reducer; one reducer per task slot, up to 96. More reducers made Dista about twice as slow, since 416 map tasks then make tens of thousands of partitions; Ray's best on 12 nodes, with 48 reducers, is 9,644 ms. The 12-node row is from shuffle-reducers-2026-11-02. Every answer matches both one-process references: groups and trips exactly, sums to about 1e-13. Results: shuffle-matrix-2026-11-02-2

The shuffle, one process ms

The head node alone. Each Dista thread reads whole files into arrays of its own, added up at the end. Ray's time barely moves with threads: its decoding uses them, but its per-batch NumPy work runs on one. Ray's 16- and 12-thread runs failed, likely from memory pressure on the head node, which also runs Kubernetes. Results: shuffle-local-2026-11-02-3

Where the time went

Dista records a timeline of every task, which opens in Perfetto. Before the read path, a file task waited on S3 for about 95% of its time: decoding the same file from local disk takes 12 to 40 ms. Planning each query read the table's 168 manifests one after another, and every task re-read the table's metadata. Now the footer comes in one read, a file's column chunks and a plan's manifests are fetched at once, and the metadata goes out with the task. Rust's iceberg-rust reader already did the first two.

Results and the trace: gate-2026-11-01-readpath

Building and shippingcold build on the head node

Dista's image is distroless: the binary and the libraries it links, one image for head, workers and driver. Ray's needs no compile and is the largest.

The runtime itself