Interop

DuckDB and Mojo: a sound example

The problem to solve: we have acoustic sensors at four construction sites, two seconds of sound an hour. A relational table says where each sensor is and when each clip was taken. Noise permits run 07:00 to 19:00.

This is a showcase of the Mojo mlake DuckDB extension. It can use any Mojo library — reading Iceberg tables through the magmalake stack, for one. The interesting addition is that it now also computes: five audio features over a BLOB column, and a table function over a directory of recordings. The kernel is Mojo, compiled into the extension, running on the bytes of the DuckDB vector where they already are.

The query

WITH measured AS (
    SELECT s.site, r.recorded_at, mlake_rms_db(r.clip) AS loudness_db
    FROM recordings r JOIN sensors s USING (sensor_id)
),
baseline AS (
    SELECT *, median(loudness_db) OVER (PARTITION BY site) AS site_median
    FROM measured
)
SELECT site, recorded_at, round(loudness_db, 1)
FROM baseline
WHERE hour(recorded_at) NOT BETWEEN 7 AND 18
  AND loudness_db - site_median > 12;

The site and the hour are relational. The loudness is a pass over 32,000 samples in Mojo. Comparing against the site’s own baseline is a window function over the column that pass produced — available only because the feature is an expression rather than a number computed elsewhere and loaded back in. It works in a WHERE, in a GROUP BY, inside a window, anywhere a column does.

The functions

mlake_rms_db, mlake_peak_db, mlake_centroid_hz, mlake_zcr and mlake_duration_s each take a WAV BLOB and return a DOUBLE. For samples still in files rather than in a column:

SELECT * FROM mlake_audio_features('recordings/*.wav');
-- path, sample_rate, channels, frames, duration_s,
-- rms_db, peak_db, centroid_hz, zcr, error

A clip that will not decode is a NULL from the scalar functions and a row with an error from the table function. Neither fails the query: one bad file in two thousand costs a row, and the table function names which row and why.

Correctness

Every feature is defined tightly enough to be reimplemented against, and is. The repository carries an independent numpy implementation — np.fft.rfft over Python’s wave module, sharing no code with the Mojo — and diffs the two over every clip in the dataset:

  ok  duration_s   max |mojo - numpy| = 0.000e+00
  ok  rms_db       max |mojo - numpy| = 0.000e+00
  ok  peak_db      max |mojo - numpy| = 0.000e+00
  ok  centroid_hz  max |mojo - numpy| = 2.274e-12
  ok  zcr          max |mojo - numpy| = 0.000e+00

Exact on four of the five. The centroid is a ratio of two sums over 513 bins, and two correct implementations may add them in either order.

Throughput

1152 clips, 74 MB of 16-bit PCM, M4, DuckDB 1.4.1. The same query and the same answer, four ways of getting the column — the extension, a DuckDB Python UDF handed whole pyarrow arrays, one handed a blob at a time, and fetching every blob into Python to loop there.

rms_db, one pass over the samples:

1 thread10 threads
mojo67.6 ms13.8 ms
py-arrow90.7 ms45.5 ms
py-native129.8 ms148.8 ms
fetch+numpy93.8 ms51.7 ms

centroid_hz, sixty-odd Fourier transforms per clip:

1 thread10 threads
mojo271.7 ms57.3 ms
py-arrow276.7 ms110.1 ms
py-native318.6 ms195.5 ms
fetch+numpy279.8 ms235.2 ms

Two different results, and the difference between them is the useful part.

On one thread, the Fourier transform is a tie. numpy’s is pocketfft and it is excellent; parity is the honest ceiling for a hand-written transform, and parity is what this gets. The single-threaded gain is on rms_db, at 1.4×, and that gain is not arithmetic — it is 1152 blobs not becoming Python objects.

With threads it is not close. DuckDB evaluates a scalar function from every worker, and there is no interpreter state to serialise on: 4.9× across ten cores, against 2.5× for the best Python UDF. That ratio is the GIL, and vectorising harder inside the UDF does not move it.

So the claim is not that Mojo beats numpy at FFTs. It is that a kernel compiled into the database parallelises with the query, does not cross a language boundary once per row, and is written in something other than C++.

What it costs to carry

MojoPython
decode + four features331 lines64 lines
leaning onnothingnumpy, 24 MB
shipped asa 2.1 MB shared libraryan interpreter and its site-packages

Five times the source but far fewer dependencies. Mojo does the work instead of delegating it: chunk-walking the RIFF container that Python’s wave module handles, and a radix-2 transform where numpy calls rfft. That is the trade — more code you own, and nothing to install beside the database.

Limits

16-bit PCM WAVE only. Anything else is refused by name rather than guessed at, so a file this cannot read says what it is.

Globs match one path component: clips/*.wav, not clips/**/*.wav. A union of two calls says the same thing out loud.

The audio table function has neither projection nor filter pushdown, so it decodes every clip the pattern matches and lets DuckDB discard the rest. The scalar functions have no such problem — a WHERE that eliminates a row before the expression runs eliminates the decode with it.

Running it

examples/soundlake is the whole thing: a generated dataset, eight queries in order, the numpy cross-check, and the benchmark above.

pixi run build
cd examples/soundlake && ./run.sh bench