`orca.keyed`

Keyed partitions: keys that are names, read from a column of a dataset.

from orca.keyed import Keyed

comptime Labels = Pipeline[
    "labels", Label, Score, partitions=Keyed["av2.logs", "log_id"], keys_per_run=100
]

The keys are the distinct values of column in the current snapshot of the warehouse dataset dataset (ORCA_WAREHOUSE), each a Name. They live in data rather than in the source because there can be thousands and they grow. --partitions takes a list of names, @all for every current key, or @file:<path> for a list one per line; a range is refused, and so is a name the dataset doesn't have, so a typo doesn't make a run of nothing. <binary> keys prints the snapshot and the keys, for the scheduler, which links no iceberg.

A keyed pipeline has no calendar, so schedule= and fills= don't compile with one (Pipeline.check). This module imports iceberg, so only binaries with a keyed pipeline build it.

Structs

`Keyed`

struct Keyed[dataset_: StringSpan[ImmStaticOrigin], column_: StringSpan[ImmStaticOrigin]]

A partition per distinct value of column_ in the dataset dataset_ (namespace.table).

Implements Movable, Partitions

comptime K = Name

`kind`

def kind() -> StaticString

`start`

def start() -> StaticString

`dataset`

def dataset() -> StaticString

`column`

def column() -> StaticString

`parse`

def parse(s: StringSpan, check: Bool) -> List[Name]

`fill`

def fill(year: Int, month: Int, day: Int, previous: Bool) -> Name

`keys`

def keys() -> Tuple[Int64, List[String]]