`thrift.parquet_types`

The Apache Parquet metadata schema as Mojo structs.

GENERATED by tools/gen_types.py from spec/parquet.thrift — do not edit by hand. Regenerate with pixi run gen-types after dropping a newer parquet.thrift into spec/.

Every Thrift struct, union and enum in the IDL is here. Optional fields are Optional[T]; required fields are validated on read; unknown field ids are skipped so a file written by a newer Parquet still decodes; write emits fields in ascending id order.

Enums are open: an unknown value round-trips as its integer rather than raising, because Parquet adds codecs and encodings over time.

Structs

`Type`

struct Type

Types supported by Parquet. These types are intended to be used in combination with the encodings to control the on disk storage format. For example INT16 is not included as a type since a good encoding of INT32 would handle this.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime BOOLEAN = Type(Int32(0))

comptime INT32 = Type(Int32(1))

comptime INT64 = Type(Int32(2))

comptime INT96 = Type(Int32(3))

comptime FLOAT = Type(Int32(4))

comptime DOUBLE = Type(Int32(5))

comptime BYTE_ARRAY = Type(Int32(6))

comptime FIXED_LEN_BYTE_ARRAY = Type(Int32(7))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or Type(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`ConvertedType`

struct ConvertedType

DEPRECATED: Common types used by frameworks (e.g. Hive, Pig) using parquet. ConvertedType is superseded by LogicalType. This enum should not be extended. See LogicalTypes.md for conversion between ConvertedType and LogicalType.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime UTF8 = ConvertedType(Int32(0))

comptime MAP = ConvertedType(Int32(1))

comptime MAP_KEY_VALUE = ConvertedType(Int32(2))

comptime LIST = ConvertedType(Int32(3))

comptime ENUM = ConvertedType(Int32(4))

comptime DECIMAL = ConvertedType(Int32(5))

comptime DATE = ConvertedType(Int32(6))

comptime TIME_MILLIS = ConvertedType(Int32(7))

comptime TIME_MICROS = ConvertedType(Int32(8))

comptime TIMESTAMP_MILLIS = ConvertedType(Int32(9))

comptime TIMESTAMP_MICROS = ConvertedType(Int32(10))

comptime UINT_8 = ConvertedType(Int32(11))

comptime UINT_16 = ConvertedType(Int32(12))

comptime UINT_32 = ConvertedType(Int32(13))

comptime UINT_64 = ConvertedType(Int32(14))

comptime INT_8 = ConvertedType(Int32(15))

comptime INT_16 = ConvertedType(Int32(16))

comptime INT_32 = ConvertedType(Int32(17))

comptime INT_64 = ConvertedType(Int32(18))

comptime JSON = ConvertedType(Int32(19))

comptime BSON = ConvertedType(Int32(20))

comptime INTERVAL = ConvertedType(Int32(21))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or ConvertedType(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`FieldRepetitionType`

struct FieldRepetitionType

Representation of Schemas.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime REQUIRED = FieldRepetitionType(Int32(0))

comptime OPTIONAL = FieldRepetitionType(Int32(1))

comptime REPEATED = FieldRepetitionType(Int32(2))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or FieldRepetitionType(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`EdgeInterpolationAlgorithm`

struct EdgeInterpolationAlgorithm

Edge interpolation algorithm for Geography logical type.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime SPHERICAL = EdgeInterpolationAlgorithm(Int32(0))

comptime VINCENTY = EdgeInterpolationAlgorithm(Int32(1))

comptime THOMAS = EdgeInterpolationAlgorithm(Int32(2))

comptime ANDOYER = EdgeInterpolationAlgorithm(Int32(3))

comptime KARNEY = EdgeInterpolationAlgorithm(Int32(4))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or EdgeInterpolationAlgorithm(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`Encoding`

struct Encoding

Encodings supported by Parquet. Not all encodings are valid for all types. These enums are also used to specify the encoding of definition and repetition levels. See the accompanying doc for the details of the more complicated encodings.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime PLAIN = Encoding(Int32(0))

comptime PLAIN_DICTIONARY = Encoding(Int32(2))

comptime RLE = Encoding(Int32(3))

comptime BIT_PACKED = Encoding(Int32(4))

comptime DELTA_BINARY_PACKED = Encoding(Int32(5))

comptime DELTA_LENGTH_BYTE_ARRAY = Encoding(Int32(6))

comptime DELTA_BYTE_ARRAY = Encoding(Int32(7))

comptime RLE_DICTIONARY = Encoding(Int32(8))

comptime BYTE_STREAM_SPLIT = Encoding(Int32(9))

comptime ALP = Encoding(Int32(10))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or Encoding(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`CompressionCodec`

struct CompressionCodec

Supported compression algorithms. Codecs added in format version X.Y can be read by readers based on X.Y and later. Codec support may vary between readers based on the format version and libraries available at runtime. See Compression.md for a detailed specification of these algorithms.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime UNCOMPRESSED = CompressionCodec(Int32(0))

comptime SNAPPY = CompressionCodec(Int32(1))

comptime GZIP = CompressionCodec(Int32(2))

comptime LZO = CompressionCodec(Int32(3))

comptime BROTLI = CompressionCodec(Int32(4))

comptime LZ4 = CompressionCodec(Int32(5))

comptime ZSTD = CompressionCodec(Int32(6))

comptime LZ4_RAW = CompressionCodec(Int32(7))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or CompressionCodec(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`PageType`

struct PageType

The Parquet PageType enum.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime DATA_PAGE = PageType(Int32(0))

comptime INDEX_PAGE = PageType(Int32(1))

comptime DICTIONARY_PAGE = PageType(Int32(2))

comptime DATA_PAGE_V2 = PageType(Int32(3))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or PageType(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`BoundaryOrder`

struct BoundaryOrder

Enum to annotate whether lists of min/max elements inside ColumnIndex are ordered and if so, in which direction.

Implements Copyable, Defaultable, Equatable, ImplicitlyCopyable, Movable, Writable

Fields: value: Int32

comptime UNORDERED = BoundaryOrder(Int32(0))

comptime ASCENDING = BoundaryOrder(Int32(1))

comptime DESCENDING = BoundaryOrder(Int32(2))

`init`

def __init__(out self)

def __init__(out self, value: Int32)

`eq`

def __eq__(self, other: Self) -> Bool

`ne`

def __ne__(self, other: Self) -> Bool

`name`

def name(self) -> String

The IDL spelling, or BoundaryOrder(n) for a value we do not know.

`write_to`

def write_to(self, mut writer: T)

`SizeStatistics`

struct SizeStatistics

A structure for capturing metadata for estimating the unencoded, uncompressed size of data written. This is useful for readers to estimate how much memory is needed to reconstruct data in their memory model and for fine grained filter pushdown on nested structures (the histograms contained in this structure can help determine the number of nulls at a particular nesting level and maximum length of lists).

Implements Copyable, Defaultable, Movable

Fields: unencoded_byte_array_data_bytes: Optional[Int64]; repetition_level_histogram: Optional[List[Int64]]; definition_level_histogram: Optional[List[Int64]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one SizeStatistics from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this SizeStatistics, fields in ascending id order.

`BoundingBox`

struct BoundingBox

Bounding box for GEOMETRY or GEOGRAPHY type in the representation of min/max value pair of coordinates from each axis.

Implements Copyable, Defaultable, Movable

Fields: xmin: Float64; xmax: Float64; ymin: Float64; ymax: Float64; zmin: Optional[Float64]; zmax: Optional[Float64]; mmin: Optional[Float64]; mmax: Optional[Float64]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BoundingBox from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BoundingBox, fields in ascending id order.

`GeospatialStatistics`

struct GeospatialStatistics

Statistics specific to Geometry and Geography logical types.

Implements Copyable, Defaultable, Movable

Fields: bbox: Optional[BoundingBox]; geospatial_types: Optional[List[Int32]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one GeospatialStatistics from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this GeospatialStatistics, fields in ascending id order.

`Statistics`

struct Statistics

Statistics per row group and per page All fields are optional.

Implements Copyable, Defaultable, Movable

Fields: max: Optional[List[UInt8]]; min: Optional[List[UInt8]]; null_count: Optional[Int64]; distinct_count: Optional[Int64]; max_value: Optional[List[UInt8]]; min_value: Optional[List[UInt8]]; is_max_value_exact: Optional[Bool]; is_min_value_exact: Optional[Bool]; nan_count: Optional[Int64]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one Statistics from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this Statistics, fields in ascending id order.

`StringType`

struct StringType

Empty structs to use as logical type annotations.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one StringType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this StringType, fields in ascending id order.

`UUIDType`

struct UUIDType

Allowed for BYTE_ARRAY, must be encoded with UTF-8.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one UUIDType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this UUIDType, fields in ascending id order.

`MapType`

struct MapType

Allowed for FIXED[16], must be encoded as raw UUID bytes.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one MapType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this MapType, fields in ascending id order.

`ListType`

struct ListType

See LogicalTypes.md.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ListType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ListType, fields in ascending id order.

`EnumType`

struct EnumType

See LogicalTypes.md.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one EnumType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this EnumType, fields in ascending id order.

`DateType`

struct DateType

Allowed for BYTE_ARRAY, must be encoded with UTF-8.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one DateType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this DateType, fields in ascending id order.

`Float16Type`

struct Float16Type

Allowed for INT32.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one Float16Type from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this Float16Type, fields in ascending id order.

`NullType`

struct NullType

Logical type to annotate a column that is always null. Sometimes when discovering the schema of existing data, values are always null and the physical type can't be determined. This annotation signals the case where the physical type was guessed from all null values.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one NullType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this NullType, fields in ascending id order.

`DecimalType`

struct DecimalType

Decimal logical type annotation Scale must be zero or a positive integer less than or equal to the precision. Precision must be a non- zero positive integer. To maintain forward-compatibility in v1, implementations using this logical type must also set scale and precision on the annotated SchemaElement. Allowed for physical types: INT32, INT64, FIXED_LEN_BYTE_ARRAY, and BYTE_ARRAY.

Implements Copyable, Defaultable, Movable

Fields: scale: Int32; precision: Int32

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one DecimalType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this DecimalType, fields in ascending id order.

`MilliSeconds`

struct MilliSeconds

Time units for logical types.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one MilliSeconds from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this MilliSeconds, fields in ascending id order.

`MicroSeconds`

struct MicroSeconds

The Parquet MicroSeconds struct.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one MicroSeconds from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this MicroSeconds, fields in ascending id order.

`NanoSeconds`

struct NanoSeconds

The Parquet NanoSeconds struct.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one NanoSeconds from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this NanoSeconds, fields in ascending id order.

`TimeUnit`

struct TimeUnit

The Parquet TimeUnit struct. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: MILLIS: Optional[MilliSeconds]; MICROS: Optional[MicroSeconds]; NANOS: Optional[NanoSeconds]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one TimeUnit from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this TimeUnit, fields in ascending id order.

`TimestampType`

struct TimestampType

Timestamp logical type annotation Allowed for physical types: INT64.

Implements Copyable, Defaultable, Movable

Fields: isAdjustedToUTC: Bool; unit: TimeUnit

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one TimestampType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this TimestampType, fields in ascending id order.

`TimeType`

struct TimeType

Time logical type annotation Allowed for physical types: INT32 (millis), INT64 (micros, nanos).

Implements Copyable, Defaultable, Movable

Fields: isAdjustedToUTC: Bool; unit: TimeUnit

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one TimeType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this TimeType, fields in ascending id order.

`IntType`

struct IntType

Integer logical type annotation bitWidth must be 8, 16, 32, or 64. Allowed for physical types: INT32, INT64.

Implements Copyable, Defaultable, Movable

Fields: bitWidth: Int8; isSigned: Bool

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one IntType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this IntType, fields in ascending id order.

`JsonType`

struct JsonType

Embedded JSON logical type annotation Allowed for physical types: BYTE_ARRAY.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one JsonType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this JsonType, fields in ascending id order.

`BsonType`

struct BsonType

Embedded BSON logical type annotation Allowed for physical types: BYTE_ARRAY.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BsonType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BsonType, fields in ascending id order.

`VariantType`

struct VariantType

Embedded Variant logical type annotation.

Implements Copyable, Defaultable, Movable

Fields: specification_version: Optional[Int8]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one VariantType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this VariantType, fields in ascending id order.

`GeometryType`

struct GeometryType

Embedded Geometry logical type annotation Geospatial features in the Well-Known Binary (WKB) format and edges interpolation is always linear/planar. A custom CRS can be set by the crs field. If unset, it defaults to "OGC:CRS84", which means that the geometries must be stored in longitude, latitude based on the WGS84 datum. Allowed for physical type: BYTE_ARRAY. See Geospatial.md for details.

Implements Copyable, Defaultable, Movable

Fields: crs: Optional[String]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one GeometryType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this GeometryType, fields in ascending id order.

`GeographyType`

struct GeographyType

Embedded Geography logical type annotation Geospatial features in the WKB format with an explicit (non-linear/non-planar) edges interpolation algorithm. A custom geographic CRS can be set by the crs field, where longitudes are bound by [-180, 180] and latitudes are bound by [-90, 90]. If unset, the CRS defaults to "OGC:CRS84". An optional algorithm can be set to correctly interpret edges interpolation of the geometries. If unset, the algorithm defaults to SPHERICAL. Allowed for physical type: BYTE_ARRAY. See Geospatial.md for details.

Implements Copyable, Defaultable, Movable

Fields: crs: Optional[String]; algorithm: Optional[EdgeInterpolationAlgorithm]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one GeographyType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this GeographyType, fields in ascending id order.

`FileType`

struct FileType

File logical type annotation Annotates a group that represents a reference to a file, or to a range of bytes that may be stored inline or in an external file. See LogicalTypes.md for details.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one FileType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this FileType, fields in ascending id order.

`LogicalType`

struct LogicalType

LogicalType annotations to replace ConvertedType. To maintain compatibility, implementations using LogicalType for a SchemaElement must also set the corresponding ConvertedType (if any) from the following table. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: STRING: Optional[StringType]: Use ConvertedType UTF8.; MAP: Optional[MapType]: Use ConvertedType MAP.; LIST: Optional[ListType]: Use ConvertedType LIST.; ENUM: Optional[EnumType]: Use ConvertedType ENUM.; DECIMAL: Optional[DecimalType]: Use ConvertedType DECIMAL + SchemaElement.{scale, precision}.; DATE: Optional[DateType]: Use ConvertedType DATE.; TIME: Optional[TimeType]; TIMESTAMP: Optional[TimestampType]; INTEGER: Optional[IntType]: Use ConvertedType INT_* or UINT_*.; UNKNOWN: Optional[NullType]: No compatible ConvertedType.; JSON: Optional[JsonType]: Use ConvertedType JSON.; BSON: Optional[BsonType]: Use ConvertedType BSON.; UUID: Optional[UUIDType]: No compatible ConvertedType.; FLOAT16: Optional[Float16Type]: No compatible ConvertedType.; VARIANT: Optional[VariantType]: No compatible ConvertedType.; GEOMETRY: Optional[GeometryType]: No compatible ConvertedType.; GEOGRAPHY: Optional[GeographyType]: No compatible ConvertedType.; FILE: Optional[FileType]: No compatible ConvertedType.

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one LogicalType from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this LogicalType, fields in ascending id order.

`SchemaElement`

struct SchemaElement

Represents an element inside a schema definition. - if it is a group (inner node) then type is undefined and num_children is defined - if it is a primitive type (leaf) then type is defined and num_children is undefined the nodes are listed in depth first traversal order.

Implements Copyable, Defaultable, Movable

Fields: type_: Optional[Type]; type_length: Optional[Int32]; repetition_type: Optional[FieldRepetitionType]; name: String; num_children: Optional[Int32]; converted_type: Optional[ConvertedType]; scale: Optional[Int32]; precision: Optional[Int32]; field_id: Optional[Int32]; logicalType: Optional[LogicalType]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one SchemaElement from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this SchemaElement, fields in ascending id order.

`DataPageHeader`

struct DataPageHeader

Data page header.

Implements Copyable, Defaultable, Movable

Fields: num_values: Int32; encoding: Encoding; definition_level_encoding: Encoding; repetition_level_encoding: Encoding; statistics: Optional[Statistics]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one DataPageHeader from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this DataPageHeader, fields in ascending id order.

`IndexPageHeader`

struct IndexPageHeader

The Parquet IndexPageHeader struct.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one IndexPageHeader from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this IndexPageHeader, fields in ascending id order.

`DictionaryPageHeader`

struct DictionaryPageHeader

The dictionary page must be placed at the first position of the column chunk if it is partly or completely dictionary encoded. At most one dictionary page can be placed in a column chunk.

Implements Copyable, Defaultable, Movable

Fields: num_values: Int32; encoding: Encoding; is_sorted: Optional[Bool]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one DictionaryPageHeader from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this DictionaryPageHeader, fields in ascending id order.

`DataPageHeaderV2`

struct DataPageHeaderV2

Alternate page format allowing reading levels without decompressing the data Repetition and definition levels are uncompressed The remaining section containing the data is compressed if is_compressed is true Implementation note - this header is not necessarily a strict improvement over DataPageHeader (in particular the original header might provide better compression in some scenarios). Page indexes require pages to start and end at row boundaries, regardless of which page header is used.

Implements Copyable, Defaultable, Movable

Fields: num_values: Int32; num_nulls: Int32; num_rows: Int32; encoding: Encoding; definition_levels_byte_length: Int32; repetition_levels_byte_length: Int32; is_compressed: Optional[Bool]; statistics: Optional[Statistics]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one DataPageHeaderV2 from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this DataPageHeaderV2, fields in ascending id order.

`SplitBlockAlgorithm`

struct SplitBlockAlgorithm

Block-based algorithm type annotation.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one SplitBlockAlgorithm from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this SplitBlockAlgorithm, fields in ascending id order.

`BloomFilterAlgorithm`

struct BloomFilterAlgorithm

The algorithm used in Bloom filter. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: BLOCK: Optional[SplitBlockAlgorithm]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BloomFilterAlgorithm from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BloomFilterAlgorithm, fields in ascending id order.

`XxHash`

struct XxHash

Hash strategy type annotation. xxHash is an extremely fast non- cryptographic hash algorithm. It uses 64 bits version of xxHash.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one XxHash from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this XxHash, fields in ascending id order.

`BloomFilterHash`

struct BloomFilterHash

The hash function used in Bloom filter. This function takes the hash of a column value using plain encoding. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: XXHASH: Optional[XxHash]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BloomFilterHash from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BloomFilterHash, fields in ascending id order.

`Uncompressed`

struct Uncompressed

The compression used in the Bloom filter.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one Uncompressed from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this Uncompressed, fields in ascending id order.

`BloomFilterCompression`

struct BloomFilterCompression

The Parquet BloomFilterCompression struct. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: UNCOMPRESSED: Optional[Uncompressed]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BloomFilterCompression from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BloomFilterCompression, fields in ascending id order.

`BloomFilterHeader`

struct BloomFilterHeader

Bloom filter header is stored at beginning of Bloom filter data of each column and followed by its bitset.

Implements Copyable, Defaultable, Movable

Fields: numBytes: Int32; algorithm: BloomFilterAlgorithm; hash: BloomFilterHash; compression: BloomFilterCompression

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one BloomFilterHeader from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this BloomFilterHeader, fields in ascending id order.

`PageHeader`

struct PageHeader

The Parquet PageHeader struct.

Implements Copyable, Defaultable, Movable

Fields: type_: PageType; uncompressed_page_size: Int32; compressed_page_size: Int32; crc: Optional[Int32]; data_page_header: Optional[DataPageHeader]; index_page_header: Optional[IndexPageHeader]; dictionary_page_header: Optional[DictionaryPageHeader]; data_page_header_v2: Optional[DataPageHeaderV2]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one PageHeader from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this PageHeader, fields in ascending id order.

`KeyValue`

struct KeyValue

Wrapper struct to store key values.

Implements Copyable, Defaultable, Movable

Fields: key: String; value: Optional[String]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one KeyValue from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this KeyValue, fields in ascending id order.

`SortingColumn`

struct SortingColumn

Sort order within a RowGroup of a leaf column.

Implements Copyable, Defaultable, Movable

Fields: column_idx: Int32; descending: Bool; nulls_first: Bool

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one SortingColumn from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this SortingColumn, fields in ascending id order.

`PageEncodingStats`

struct PageEncodingStats

Statistics of a given page type and encoding.

Implements Copyable, Defaultable, Movable

Fields: page_type: PageType; encoding: Encoding; count: Int32

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one PageEncodingStats from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this PageEncodingStats, fields in ascending id order.

`ColumnMetaData`

struct ColumnMetaData

Description for column metadata.

Implements Copyable, Defaultable, Movable

Fields: type_: Type; encodings: List[Encoding]; path_in_schema: List[String]; codec: CompressionCodec; num_values: Int64; total_uncompressed_size: Int64; total_compressed_size: Int64; key_value_metadata: Optional[List[KeyValue]]; data_page_offset: Int64; index_page_offset: Optional[Int64]; dictionary_page_offset: Optional[Int64]; statistics: Optional[Statistics]; encoding_stats: Optional[List[PageEncodingStats]]; bloom_filter_offset: Optional[Int64]; bloom_filter_length: Optional[Int32]; size_statistics: Optional[SizeStatistics]; geospatial_statistics: Optional[GeospatialStatistics]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ColumnMetaData from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ColumnMetaData, fields in ascending id order.

`EncryptionWithFooterKey`

struct EncryptionWithFooterKey

The Parquet EncryptionWithFooterKey struct.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one EncryptionWithFooterKey from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this EncryptionWithFooterKey, fields in ascending id order.

`EncryptionWithColumnKey`

struct EncryptionWithColumnKey

The Parquet EncryptionWithColumnKey struct.

Implements Copyable, Defaultable, Movable

Fields: path_in_schema: List[String]; key_metadata: Optional[List[UInt8]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one EncryptionWithColumnKey from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this EncryptionWithColumnKey, fields in ascending id order.

`ColumnCryptoMetaData`

struct ColumnCryptoMetaData

The Parquet ColumnCryptoMetaData struct. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: ENCRYPTION_WITH_FOOTER_KEY: Optional[EncryptionWithFooterKey]; ENCRYPTION_WITH_COLUMN_KEY: Optional[EncryptionWithColumnKey]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ColumnCryptoMetaData from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ColumnCryptoMetaData, fields in ascending id order.

`ColumnChunk`

struct ColumnChunk

The Parquet ColumnChunk struct.

Implements Copyable, Defaultable, Movable

Fields: file_path: Optional[String]; file_offset: Int64; meta_data: Optional[ColumnMetaData]; offset_index_offset: Optional[Int64]; offset_index_length: Optional[Int32]; column_index_offset: Optional[Int64]; column_index_length: Optional[Int32]; crypto_metadata: Optional[ColumnCryptoMetaData]; encrypted_column_metadata: Optional[List[UInt8]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ColumnChunk from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ColumnChunk, fields in ascending id order.

`RowGroup`

struct RowGroup

The Parquet RowGroup struct.

Implements Copyable, Defaultable, Movable

Fields: columns: List[ColumnChunk]; total_byte_size: Int64; num_rows: Int64; sorting_columns: Optional[List[SortingColumn]]; file_offset: Optional[Int64]; total_compressed_size: Optional[Int64]; ordinal: Optional[Int16]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one RowGroup from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this RowGroup, fields in ascending id order.

`TypeDefinedOrder`

struct TypeDefinedOrder

Empty struct to signal the order defined by the physical or logical type.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one TypeDefinedOrder from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this TypeDefinedOrder, fields in ascending id order.

`IEEE754TotalOrder`

struct IEEE754TotalOrder

Empty struct to signal IEEE 754 total order for floating point types.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one IEEE754TotalOrder from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this IEEE754TotalOrder, fields in ascending id order.

`Int96TimestampOrder`

struct Int96TimestampOrder

Empty struct to signal chronological ordering of physical type INT96.

Implements Copyable, Defaultable, Movable

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one Int96TimestampOrder from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this Int96TimestampOrder, fields in ascending id order.

`ColumnOrder`

struct ColumnOrder

Union to specify the order used for the min_value and max_value fields for a column. This union takes the role of an enhanced enum that allows rich elements (which will be needed for a collation-based ordering in the future). Possible values are: * TypeDefinedOrder - the column uses the order defined by its logical or physical type (if there is no logical type). * IEEE754TotalOrder - the floating point column uses IEEE 754 total order. * Int96TimestampOrder - the INT96 column uses chronological timestamp order. If the reader does not support the value of this union, min and max stats for this column should be ignored. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: TYPE_ORDER: Optional[TypeDefinedOrder]; IEEE_754_TOTAL_ORDER: Optional[IEEE754TotalOrder]; INT96_TIMESTAMP_ORDER: Optional[Int96TimestampOrder]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ColumnOrder from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ColumnOrder, fields in ascending id order.

`PageLocation`

struct PageLocation

The Parquet PageLocation struct.

Implements Copyable, Defaultable, Movable

Fields: offset: Int64; compressed_page_size: Int32; first_row_index: Int64

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one PageLocation from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this PageLocation, fields in ascending id order.

`OffsetIndex`

struct OffsetIndex

Optional offsets for each data page in a ColumnChunk. Forms part of the page index, along with ColumnIndex. OffsetIndex may be present even if ColumnIndex is not.

Implements Copyable, Defaultable, Movable

Fields: page_locations: List[PageLocation]; unencoded_byte_array_data_bytes: Optional[List[Int64]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one OffsetIndex from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this OffsetIndex, fields in ascending id order.

`ColumnIndex`

struct ColumnIndex

Optional statistics for each data page in a ColumnChunk. Forms part the page index, along with OffsetIndex. If this structure is present, OffsetIndex must also be present. For each field in this structure, <field>[i] refers to the page at OffsetIndex.page_locations[i].

Implements Copyable, Defaultable, Movable

Fields: null_pages: List[Bool]; min_values: List[List[UInt8]]; max_values: List[List[UInt8]]; boundary_order: BoundaryOrder; null_counts: Optional[List[Int64]]; repetition_level_histograms: Optional[List[Int64]]; definition_level_histograms: Optional[List[Int64]]; nan_counts: Optional[List[Int64]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one ColumnIndex from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this ColumnIndex, fields in ascending id order.

`AesGcmV1`

struct AesGcmV1

The Parquet AesGcmV1 struct.

Implements Copyable, Defaultable, Movable

Fields: aad_prefix: Optional[List[UInt8]]; aad_file_unique: Optional[List[UInt8]]; supply_aad_prefix: Optional[Bool]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one AesGcmV1 from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this AesGcmV1, fields in ascending id order.

`AesGcmCtrV1`

struct AesGcmCtrV1

The Parquet AesGcmCtrV1 struct.

Implements Copyable, Defaultable, Movable

Fields: aad_prefix: Optional[List[UInt8]]; aad_file_unique: Optional[List[UInt8]]; supply_aad_prefix: Optional[Bool]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one AesGcmCtrV1 from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this AesGcmCtrV1, fields in ascending id order.

`EncryptionAlgorithm`

struct EncryptionAlgorithm

The Parquet EncryptionAlgorithm struct. A Thrift union: exactly one member is set.

Implements Copyable, Defaultable, Movable

Fields: AES_GCM_V1: Optional[AesGcmV1]; AES_GCM_CTR_V1: Optional[AesGcmCtrV1]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one EncryptionAlgorithm from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this EncryptionAlgorithm, fields in ascending id order.

`FileMetaData`

struct FileMetaData

Description for file metadata.

Implements Copyable, Defaultable, Movable

Fields: version: Int32; schema: List[SchemaElement]; num_rows: Int64; row_groups: List[RowGroup]; key_value_metadata: Optional[List[KeyValue]]; created_by: Optional[String]; column_orders: Optional[List[ColumnOrder]]; encryption_algorithm: Optional[EncryptionAlgorithm]; footer_signing_key_metadata: Optional[List[UInt8]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one FileMetaData from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this FileMetaData, fields in ascending id order.

`FileCryptoMetaData`

struct FileCryptoMetaData

Crypto metadata for files with encrypted footer.

Implements Copyable, Defaultable, Movable

Fields: encryption_algorithm: EncryptionAlgorithm; key_metadata: Optional[List[UInt8]]

`init`

def __init__(out self)

`read`

def read[P: TProtocolReader, //](mut self, mut p: P)

Decode one FileCryptoMetaData from p, skipping fields we do not know.

`write`

def write[W: TProtocolWriter, //](self, mut p: W)

Encode this FileCryptoMetaData, fields in ascending id order.