Skip to content
Coordinator DB
View as Markdown
View as Markdown

msg_pack ​

Description ​

Consecutive positional MessagePack fields. The target column count determines row boundaries; this is not an outer array of row arrays.

Input and output. Select it with FORMAT msg_pack. Names ignore ASCII case; use the snake_case name shown here.

Fields are consecutive MessagePack objects in column order; every target column count groups the next objects into one row. There is no header or outer array of row arrays. Incomplete rows are rejected.

Native collection members use MessagePack containers. Exact wide numbers need matching destination types to interpret their binary coefficients. Numeric payloads and lengths use the MessagePack byte order; see the mappings below.

text
SELECT expressions FORMAT msg_pack;
INSERT INTO table [(columns)] [SETTINGS ...] FORMAT msg_pack
<data>

Example usage ​

Use the native client's --query mode to preserve binary output bytes. The server must be running, the executable available as ./coordinator, and database data selected.

sh
./coordinator client --database data --query "SELECT CAST(7 AS Int64) AS id FORMAT msg_pack" > sample.msg_pack
./coordinator client --database data --query "CREATE TABLE format_import(id Int64)" > /dev/null
./coordinator client --database data --query "INSERT INTO format_import FORMAT msg_pack" < sample.msg_pack > /dev/null
./coordinator client --database data --query "SELECT id FROM format_import FORMAT csv"
text
7

Create format_import in a fresh database before running this example.

Format settings ​

Use a statement SETTINGS clause; these options are not persisted by SET.

Setting Type Default Meaning
msg_pack_uuid_encoding String EXT Write UUID as EXT (extension type 2), BIN (16 binary bytes), or STR (canonical text).
snappy_mode String basic Read Snappy client files as basic (Hadoop blocks) or framed (standard streaming format).

Type mappings ​

Native value MessagePack representation
NULL Nil (0xc0).
INTEGER, UInt64, Enum8/16 The smallest signed or unsigned integer tag that holds the value, including positive and negative fixint.
Bool Integer 0 or 1 on output; input also accepts the boolean tags.
Float32/64 Float32/64 tags with exact big-endian IEEE payloads, including negative zero and infinities. Native target conversion stores NaN input as NULL.
String/TEXT, FixedString, BLOB Bin8/16/32; input also accepts Fixstr and Str8/16/32. BLOB and FixedString retain arbitrary bytes and padding; native String requires valid UTF-8.
Date/Date32, DateTime Integer epoch days or epoch seconds.
DateTime64 Unsigned integer carrying the raw signed tick bits; the target declaration supplies precision and timezone.
Decimal32/64 Integer coefficient; the target declaration supplies the scale.
Decimal128/256, Int128/UInt128/Int256/UInt256 Bin8 with exactly 16 or 32 little-endian coefficient bytes; signed coefficients use two's complement.
IPv4, IPv6 Unsigned address integer or 16 binary address bytes.
UUID Extension type 2 with 16 canonical bytes by default; the UUID setting can instead write 16 bytes as two little-endian halves or 36-character canonical text. Input accepts all three.
Array, Tuple Fixarray/Array16/Array32 with recursively encoded members. Named Tuple fields remain positional.
Map Fixmap/Map16/Map32 with recursively encoded keys and values; order and duplicate keys are retained.
Nullable, LowCardinality The inner representation.