Set up a repeatable external Kafka import ​

A repeatable import needs a broker consumer, a documented row contract and a recovery policy. Coordinator DB provides the SQL destination. It does not provide a Kafka table engine or manage consumer groups.

Establish a baseline ​

First read a small message sample with your chosen consumer tooling. Record the topic and source position in your importer's own state. Create an explicit destination table, then upload those records as one bounded batch. Query the destination to verify identifiers, null handling, numeric ranges and nested values.

Use separate credentials for the broker and database. Check connectivity from the machine running the consumer; a successful SQL call on a developer laptop does not verify that machine's network route.

Exercise recovery before continuous use ​

Test an invalid message, a duplicate message and a destination connection failure. Decide when progress is acknowledged and how an ambiguous write is reconciled. Stable source identifiers let the consumer detect previously accepted records instead of blindly repeating a batch.

Batch size should be chosen from measured duration and memory, while respecting request admission and response limits. There is no asynchronous INSERT queue: a successful response follows completion of the statement. Monitor completed requests and errors to identify a change in behavior.

See batch inserts, query history and HTTP ingestion.