# lz4 API reference

> The `lz4` package, from lz4 0.1.4, as published on mojoshelf (commit 23cd6e54967d), by `mojo doc` from Mojo 1.2.0.

`lz4` — Mojo bindings to liblz4 (block format + frame format) via a thin C shim (liblz4mojo.{dylib,so}), loaded through an `OwnedDLHandle`.

Two independent codecs live in one liblz4:

- **Block format** (`LZ4_RAW` in Parquet): a bare compressed blob with no
  size prefix — the uncompressed length must be supplied out of band (that's
  exactly what a Parquet page header carries). Legacy Parquet's `LZ4`
  compression carries the same blocks but wrapped in a Hadoop framing of
  `[BE u32 uncompressed_len][BE u32 compressed_len]` pairs.
- **Frame format** (`LZ4F`): a self-describing container with a magic
  number, optional content-size field, and (optionally) checksums. This is
  the *only* codec Apache Iceberg allows for Puffin file footers, which is
  why `compress_frame` always sets the content-size flag by default.

Mirrors zlib.mojo's FFI pattern: a single-call C wrapper
(shim/lz4_wrapper.c, built to $CONDA_PREFIX/lib/liblz4mojo.{dylib,so} by the
lz4-shim pixi package) so Mojo never reads back internal library state after
a foreign call. The handle is opened **once per process** and never closed
(`_LIB` below) — a `dlopen`/`dlclose` cycle costs hundreds of microseconds,
far more than compressing a Parquet page, so opening one per call made the
binding's fixed cost dwarf the work it wrapped. It is passed as a borrowed
(default-convention) parameter to each worker function, so Mojo's ASAP
destruction can't `dlclose` the library before the C call inside that worker
runs.

## Functions

### `compress_block`

```mojo
def compress_block(data: Span[UInt8], acceleration: Int = Int(1)) -> List[UInt8]
```

Compress to a raw LZ4 block (`LZ4_RAW`) — no size prefix. The caller (e.g. a Parquet page writer) must record the uncompressed length out of band; `decompress_block` needs it back to decode.

Args:

- `data`: Bytes to compress.
- `acceleration`: >=1; higher trades compression ratio for speed
    (1 matches `LZ4_compress_default`).

### `compress_block_hc`

```mojo
def compress_block_hc(data: Span[UInt8], level: Int = Int(9)) -> List[UInt8]
```

Compress to a raw LZ4 block using the high-compression (HC) encoder. Same wire format as `compress_block` — slower, smaller output.

Args:

- `data`: Bytes to compress.
- `level`: 1..12 (LZ4HC_CLEVEL_MIN..MAX); higher is slower and smaller.

### `decompress_block_into`

```mojo
def decompress_block_into(data: Span[UInt8], dst: Span[UInt8]) -> Int
```

Decompress a raw LZ4 block into a caller-supplied buffer. `dst` must be at least as large as the true uncompressed size — LZ4 blocks carry no size of their own, so this can't grow the buffer for you.

Returns the number of bytes written.

### `decompress_block`

```mojo
def decompress_block(data: Span[UInt8], uncompressed_size: Int) -> List[UInt8]
```

Decompress a raw LZ4 block. `uncompressed_size` must be exact — Parquet page headers carry it, since LZ4 blocks carry no size prefix.

### `compress_frame`

```mojo
def compress_frame(data: Span[UInt8], level: Int = Int(0), content_size: Bool = True) -> List[UInt8]
```

Compress to a single LZ4 frame (`LZ4F`) — the format Apache Iceberg requires for Puffin file footers. `content_size=True` (the default, and the Puffin requirement) writes the uncompressed length into the frame header so a reader can size its output buffer up front.

Args:

- `data`: Bytes to compress.
- `level`: 0 = fast mode (LZ4 default); >=1 selects the HC encoder
    (1..12, higher is slower and smaller).
- `content_size`: Whether to record the uncompressed length in the
    frame header.

### `frame_content_size`

```mojo
def frame_content_size(data: Span[UInt8]) -> Optional[Int]
```

Read the uncompressed content size from an LZ4 frame header without decompressing, if the frame carries one.

Returns `None` if the frame doesn't carry a content size (LZ4F itself
can't distinguish "unspecified" from a genuine zero-length payload —
both are stored as 0), or raises if `data` isn't a valid frame header.

### `is_lz4_frame`

```mojo
def is_lz4_frame(data: Span[UInt8]) -> Bool
```

Whether `data` starts with the LZ4 frame magic number (`0x184D2204`, little-endian) — the standard way to tell an LZ4F frame apart from a bare LZ4 block, which has no magic at all.

### `decompress_frame`

```mojo
def decompress_frame(data: Span[UInt8]) -> List[UInt8]
```

Decompress a single LZ4 frame. Uses the frame header's content-size field to size the output buffer exactly when present (the case for every frame `compress_frame` produces, and the Puffin convention); otherwise grows the buffer and retries until the whole frame fits.

### `decompress_hadoop`

```mojo
def decompress_hadoop(data: Span[UInt8], uncompressed_size: Int) -> List[UInt8]
```

Decompress legacy Parquet `LZ4` (Hadoop-framed): a sequence of `[BE u32 uncompressed_len][BE u32 compressed_len][block…]` chunks, each block an ordinary `LZ4_RAW` block. This is the format `parquet-mr` historically wrote for codec `LZ4`; new writers should use `LZ4_RAW` (`compress_block`/`decompress_block`) instead — see PARQUET-1974.