# arrow_stream

## Description

Arrow IPC stream with a schema, record batches and an end marker.

Input and output. Select it with `FORMAT arrow_stream`. Names ignore ASCII case; use the snake_case name shown here.

Use this layout with consumers expecting sequential Arrow IPC messages rather than a seekable IPC file. A stream begins with its schema, can contain several record batches and must end cleanly; trailing bytes after its end marker are rejected.

The value mappings are the same as arrow. Use explicit target declarations when importing and the native client exact-byte mode when exporting.

```text
SELECT expressions FORMAT arrow_stream;
INSERT INTO table [(columns)] [SETTINGS ...] FORMAT arrow_stream
<data>
```

## Example usage

Use the native client's `--query` mode to preserve binary output bytes. The server must be running, the executable available as `./coordinator`, and database `data` selected.

```sh
./coordinator client --database data --query "SELECT CAST(7 AS Int64) AS id FORMAT arrow_stream" > sample.arrow_stream
./coordinator client --database data --query "CREATE TABLE format_import(id Int64)" > /dev/null
./coordinator client --database data --query "INSERT INTO format_import FORMAT arrow_stream" < sample.arrow_stream > /dev/null
./coordinator client --database data --query "SELECT id FROM format_import FORMAT csv"
```

```text
7
```

Create `format_import` in a fresh database before running this example.

## Format settings

Use a statement `SETTINGS` clause; these options are not persisted by `SET`.

| Setting                           | Type     | Default     | Meaning                                                                                           |
| --------------------------------- | -------- | ----------- | ------------------------------------------------------------------------------------------------- |
| `arrow_compression`               | `String` | `lz4_frame` | Compress record batch buffers with lz4_frame or zstd; none writes uncompressed buffers.           |
| `arrow_date_as_number`            | `Bool`   | `0`         | Write Date as UInt16 days since the epoch instead of Date32; Date32 always remains Date32.        |
| `arrow_dictionary`                | `Bool`   | `0`         | Write LowCardinality as dictionary indices and values instead of expanding its scalars.           |
| `arrow_dictionary_64bit_indices`  | `Bool`   | `0`         | Use 64-bit dictionary indices instead of the smallest width that fits the dictionary.             |
| `arrow_dictionary_signed_indices` | `Bool`   | `1`         | Use signed dictionary indices; when off, use unsigned indices.                                    |
| `arrow_fixed_strings`             | `Bool`   | `1`         | Write FixedString with a fixed byte width instead of variable-length Binary.                      |
| `arrow_ignore_case`               | `Bool`   | `0`         | Ignore ASCII case when matching input field names to target columns.                              |
| `arrow_missing_as_default`        | `Bool`   | `1`         | Give target columns absent from the input schema their DEFAULT; when off, reject missing columns. |
| `arrow_strings_as_text`           | `Bool`   | `1`         | Write text strings as Utf8 instead of Binary; byte strings keep their exact bytes.                |
| `ignore_unknown_fields`           | `Bool`   | `0`         | Skip input keys and header names that match no target column instead of failing.                  |
| `snappy_mode`                     | `String` | `basic`     | Read Snappy client files as basic (Hadoop blocks) or framed (standard streaming format).          |