Skip to content
Coordinator DB
View as Markdown
View as Markdown

parquet ​

Description ​

A Parquet file with column schema, row groups, data pages and footer. Use explicit target declarations when importing. Compression and row-group controls are statement settings.

Input and output. Select it with FORMAT parquet. Names ignore ASCII case; use the snake_case name shown here.

The file stores a column schema, row groups, data pages and a footer. Output types are inferred from runtime values; native collection members retain their declared types.

Create an explicit compatible destination before import, including when using file(). Missing, unknown and case-mismatched fields follow the listed options. The parquet_metadata format reads the footer as rows instead of decoding data columns.

text
SELECT expressions FORMAT parquet;
INSERT INTO table [(columns)] [SETTINGS ...] FORMAT parquet
<data>

Example usage ​

Use the native client's --query mode to preserve binary output bytes. The server must be running, the executable available as ./coordinator, and database data selected.

sh
./coordinator client --database data --query "SELECT CAST(7 AS Int64) AS id FORMAT parquet" > sample.parquet
./coordinator client --database data --query "CREATE TABLE format_import(id Int64)" > /dev/null
./coordinator client --database data --query "INSERT INTO format_import FORMAT parquet" < sample.parquet > /dev/null
./coordinator client --database data --query "SELECT id FROM format_import FORMAT csv"
text
7

Create format_import in a fresh database before running this example.

Format settings ​

Use a statement SETTINGS clause; these options are not persisted by SET.

Setting Type Default Meaning
ignore_unknown_fields Bool 0 Skip input keys and header names that match no target column instead of failing.
parquet_compression String zstd Compress column pages with snappy, lz4, brotli, zstd, gzip, or none.
parquet_detect_variant Bool 1 Recognize unannotated metadata/value binary structs as Variant values.
parquet_fixed_strings Bool 1 Write FixedString as fixed-width bytes instead of variable-width Binary.
parquet_ignore_case Bool 0 Ignore ASCII case when matching input fields to target column names.
parquet_missing_as_default Bool 1 Give missing input fields their SQL DEFAULT or native type default.
parquet_page_bytes UInt64 1048576 Target uncompressed bytes per output data page.
parquet_page_index Bool 1 Write column and offset indexes for individual data pages.
parquet_parse_json Bool 1 Decode JSON-annotated input fields as native JSON instead of text.
parquet_read_bytes UInt64 16744704 Target decoded batch bytes estimated from uncompressed row-group metadata.
parquet_read_geo Bool 1 Decode WKB or WKT columns identified by GeoParquet metadata; text and binary targets retain raw payloads.
parquet_read_rows UInt64 65409 Maximum rows per decoded input batch.
parquet_row_group_bytes UInt64 536870912 Target output row-group bytes before compression.
parquet_row_group_rows UInt64 1000000 Maximum rows per output row group.
parquet_strings_as_text Bool 1 Annotate text strings as UTF-8; BLOB keeps its exact binary representation.
parquet_wide_as_decimal Bool 0 Write 128/256-bit integers as big-endian DECIMAL instead of little-endian fixed-width bytes.
parquet_write_batch_rows UInt64 1024 Check output data-page size after this many rows.
parquet_write_geo Bool 1 Encode geometry as WKB and include GeoParquet column metadata.
snappy_mode String basic Read Snappy client files as basic (Hadoop blocks) or framed (standard streaming format).

Type mappings ​

Native value Parquet representation
Bool, integers through 64 bits, Float32/64 BOOLEAN, signed/unsigned INT32 or INT64, FLOAT/DOUBLE. Top-level narrow integers retain their runtime 64-bit class; collection members retain declared widths.
String, BLOB, FixedString UTF-8 BYTE_ARRAY, raw BYTE_ARRAY, FIXED_LEN_BYTE_ARRAY. Settings can remove text annotations or fixed widths. Native String requires UTF-8; use BLOB for arbitrary bytes.
Enum8/16, IPv4, IPv6, UUID Signed integer codes, UINT32, fixed 16-byte address, fixed 16-byte identifier. Input also accepts the UUID logical annotation.
Date/Date32, DateTime/DateTime64, Time/Time64 DATE, TIMESTAMP, TIME. Second-resolution timestamps/clocks use milliseconds; fractional values use the next exact supported unit. Clocks outside one day and overflowing conversions are rejected.
Decimal through precision 76 Standard DECIMAL with exact signed coefficients and scale.
Int128/UInt128/Int256/UInt256 Little-endian fixed 16/32-byte integers by default. parquet_wide_as_decimal=1 writes signed big-endian DECIMAL with 17/33 bytes and precision 39/77/78. Input accepts both representations; precision above 76 requires an explicit compatible integer target.
Array, Tuple, Map LIST, struct, MAP, including nested values and NULL members. Named Tuple fields retain their names.
JSON JSON-annotated binary JSON; parquet_parse_json=0 reads it as text before target conversion.
Nullable, LowCardinality, SimpleAggregateFunction Inner representation; scalar dictionary values are expanded before Parquet's own encoding.
Geometry aliases GeoParquet WKB plus geo metadata; input also accepts WKT. See the restrictions below.