Import from a private managed Kafka source ​

A private managed Kafka cluster and Coordinator DB require separate client connections. Coordinator DB does not create Kafka consumers or private broker endpoints. Run a consumer in a network location that can reach the broker and an authenticated SQL endpoint.

Separate the two checks ​

First verify broker access with the consumer's own tools and credentials. Then run a small SQL request from the same host. A broker connection failure should be investigated in broker routing, certificates and consumer permissions; a SQL authentication error belongs to the destination connection.

Decode records in the consumer and turn them into rows matching an explicit table schema. json_each_row is convenient when each message already contains named fields. Schema-bearing formats are another option when their conversion rules match the destination types.

Commit progress after verification ​

Acknowledge source progress only after the destination operation succeeds according to your application's delivery policy. If the response is lost, inspect the destination using a stable event identifier before retrying. Coordinator DB does not couple a Kafka offset commit with a SQL transaction.

Keep broker credentials and database credentials separate. Test malformed messages, temporary network failures and duplicate delivery with a small dataset before relying on continuous import.

See JSON loading, INSERT INTO and HTTP requests.