`orca.warehouse`

A dataset's current snapshot, read from its table's metadata file without iceberg, for <binary> sense.

A dataset ns.table is the filesystem catalog's table at <ORCA_WAREHOUSE>/ns/table, as in orca.tables. Its current metadata file is the one metadata/version-hint.text names, or else the highest version under metadata/, ties broken by last-updated-ms, as iceberg's catalog finds it. Of that file, current_snapshot reads the current snapshot's ID and two properties of its summary: whether an orca step made it (orca.run), and the partition keys it holds (orca.partitions), which a writer that isn't orca can set too.

The warehouse is a directory or a location in object storage (s3://bucket/warehouse), listed and read through objectstore's FileIOResolver, from the AWS_* environment.

orca's pipelines import iceberg only if they use tables, and every binary answers sense, so this reads the JSON itself: just enough of it to walk objects and arrays to the fields it needs.

Structs

`Sensed`

struct Sensed

A dataset's current snapshot, as sense prints it.

Implements Copyable, Movable

Fields: dataset: String; snapshot: Int64: -1 when the table doesn't exist or has no snapshot.; by_orca: Bool: Whether an orca step committed the snapshot.; partitions: String: The snapshot summary's orca.partitions, or empty.

Functions

`current_snapshot`

def current_snapshot(warehouse: String, dataset: String) -> Sensed

The dataset's current snapshot in the warehouse at warehouse.

`current_of`

def current_of(dataset: String, text: String) -> Sensed

The current snapshot a table metadata document names, and its summary's orca properties.