Skip to content
Coordinator DB
View as Markdown
View as Markdown

protobuf ​

Description ​

Binary messages with a varint length prefix before each row. Use a supported literal protobuf_schema and protobuf_message, or the column-derived schema with protobuf_auto_schema. This format does not read arbitrary schema files.

Input and output. Select it with FORMAT protobuf. Names ignore ASCII case; use the snake_case name shown here.

Each row is a length-prefixed Protocol Buffers message. Choose a literal proto2/proto3 schema and message, or let protobuf_auto_schema derive a message from the columns.

protobuf_schema contains schema text, not a filename. Imports are limited to bundled Google definitions. The message, field mappings, wrappers and defaults must agree with the chosen destination declarations.

text
SELECT expressions FORMAT protobuf;
INSERT INTO table [(columns)] [SETTINGS ...] FORMAT protobuf
<data>

Example usage ​

Use the native client's --query mode to preserve binary output bytes. The server must be running, the executable available as ./coordinator, and database data selected.

sh
./coordinator client --database data --query "SELECT CAST(7 AS Int64) AS id SETTINGS protobuf_auto_schema=1 FORMAT protobuf" > sample.protobuf
./coordinator client --database data --query "CREATE TABLE format_import(id Int64)" > /dev/null
./coordinator client --database data --query "INSERT INTO format_import SETTINGS protobuf_auto_schema=1 FORMAT protobuf" < sample.protobuf > /dev/null
./coordinator client --database data --query "SELECT id FROM format_import FORMAT csv"
text
7

Create format_import in a fresh database before running this example.

Format settings ​

Use a statement SETTINGS clause; these options are not persisted by SET.

Setting Type Default Meaning
protobuf_auto_schema Bool 1 Derive a message schema from columns when protobuf_schema is empty.
protobuf_keep_default_wrappers Bool 0 Write present Google wrappers for default scalar values; NULL wrappers remain absent.
protobuf_message String '' Fully qualified message name in protobuf_schema; empty selects Row for an automatically derived schema.
protobuf_missing_as_default Bool 1 Apply SQL DEFAULT to target columns outside the message schema; when off use their native type defaults.
protobuf_oneof_tag Bool 0 Read the selected oneof field number into a column named after that oneof; unmatched branches yield zero.
protobuf_schema String '' Literal proto2 or proto3 schema; imports are limited to bundled Google definitions.
protobuf_unwrap_google Bool 0 Read Google scalar wrappers directly into scalar columns; absent nullable wrappers become NULL.
snappy_mode String basic Read Snappy client files as basic (Hadoop blocks) or framed (standard streaming format).