# iceberg.batch

> A module of iceberg's API.

Building an Arrow `RecordBatch` from Mojo values.

The write path takes Arrow, which is the right input when the data already
*is* Arrow — a scan of another table, a Parquet file, anything speaking the C
Data Interface. When it is not, this is the bridge: a `ColumnBuilder` per
column, a `Datum` per cell, and `batch_of` to tie them together.

```mojo
var ids = ColumnBuilder(String("id"), 1, P_LONG)
var region = ColumnBuilder(String("region"), 2, P_STRING)
for k in range(n):
    ids.add(Datum.long_(Int64(k)))
    region.add(Datum.string_(names[k]))
var batch = batch_of([ids^, region^])
```

This is deliberately the *slow* side of the library — one tagged value per
cell, which is exactly what `iceberg.kernels` exists to avoid on the read
path. It is here because the alternative is asking a caller to lay out
validity bitmaps and offset buffers by hand, and a batch built once and
written once is not where the time goes.

## Structs

### `ColumnBuilder`

```mojo
struct ColumnBuilder
```

One column's values, before they become an Arrow array.

Implements: `Copyable`, `Movable`, `Sized`

Fields:

- `name: String`
- `field_id: Int`
- `kind: UInt8`
- `precision: Int`
- `scale: Int`
- `length: Int`
- `nullable: Bool`
- `values: List[Datum]`

#### `__init__`

```mojo
def __init__(out self, var name: String, field_id: Int, kind: UInt8, precision: Int = Int(0), scale: Int = Int(0), length: Int = Int(0), nullable: Bool = True)
```

#### `of`

```mojo
def of(schema: Schema, field_id: Int) -> Self
```

A builder typed by one of the table's columns.

#### `add`

```mojo
def add(mut self, var value: Datum)
```

#### `add_null`

```mojo
def add_null(mut self)
```

#### `__len__`

```mojo
def __len__(self) -> Int
```

#### `build_tree`

```mojo
def build_tree(self) -> ColumnTree
```

The same array, ready to sit beside a nested one.

#### `build`

```mojo
def build(self) -> ArrayData
```

### `NestedBuilder`

```mojo
struct NestedBuilder
```

A struct, list or map column, one JSON value per row.

A nested cell has no scalar to carry it, so the input here is text: the
same JSON shape `ScanResult.cell` prints, which makes a round trip through
this library one string per cell.

```mojo
var addr = NestedBuilder.of(schema, 4)
addr.add('{"city":"eu","zip":10}')
addr.add_null()
var batch = batch_of_columns([ids^.build_tree(), addr^.build()])
```

Structs are objects keyed by field **name**, lists are arrays, and a map is
either `{"keys": [...], "values": [...]}` — the spec's own shape — or a
plain object when its keys are strings.

Implements: `Copyable`, `Movable`, `Sized`

Fields:

- `name: String`
- `field_id: Int`
- `ctype: ColumnType`
- `nullable: Bool`
- `rows: List[String]`

#### `__init__`

```mojo
def __init__(out self, var name: String, field_id: Int, var ctype: ColumnType, nullable: Bool = True)
```

#### `of`

```mojo
def of(schema: Schema, field_id: Int) -> Self
```

A builder typed by one of the table's columns.

#### `add`

```mojo
def add(mut self, var json: String)
```

#### `add_null`

```mojo
def add_null(mut self)
```

#### `__len__`

```mojo
def __len__(self) -> Int
```

#### `build`

```mojo
def build(self) -> ColumnTree
```

## Functions

### `array_from_datums`

```mojo
def array_from_datums(name: String, field_id: Int, kind: UInt8, precision: Int, scale: Int, length: Int, nullable: Bool, values: List[Datum]) -> ArrayData
```

One Arrow array from a column of tagged values.

### `is_var_width_array`

```mojo
def is_var_width_array(a: ArrayData) -> Bool
```

### `append_datum`

```mojo
def append_datum(mut a: ArrayData, d: Datum, kind: UInt8)
```

Append one value to a primitive array, growing every buffer it has.

### `finish_array`

```mojo
def finish_array(mut a: ArrayData)
```

Pad the buffers a partly filled last byte leaves short.

### `batch_of`

```mojo
def batch_of(columns: List[ColumnBuilder]) -> RecordBatch
```

One `RecordBatch` from a set of equally long column builders.

### `push_json`

```mojo
def push_json(mut arena: ArrayArena, node: Int, store: TypeStore, ti: Int, doc: Json, at: Int)
```

Append one JSON value to a (possibly nested) Arrow array.

### `finish_tree`

```mojo
def finish_tree(mut arena: ArrayArena, node: Int)
```

Pad every buffer in a freshly built tree.

### `batch_of_columns`

```mojo
def batch_of_columns(var columns: List[ColumnTree]) -> RecordBatch
```

One `RecordBatch` from column trees, nested or not.