I am using the docker developer image and want to write to Kafka:
DELIMITER //
CREATE OR REPLACE PROCEDURE SendToKafka(p_userid bigint, p_txnId TEXT) AS
--
BEGIN
--
SELECT userid, user_txn_id, txn_time, sessionid, approved_amount, spent_amount, purpose
FROM user_recent_transactions
WHERE userid = p_userid AND user_txn_id = p_txnId
INTO KAFKA '10.13.1.21:9092/charglt'
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n' STARTING BY '';
--
END//
I have tested the kafka server at 10.13.1.21:9092/charglt:
badger@badger:~/kafka_2.13-4.1.1$ bin/kafka-console-consumer.sh --topic charglt --from-beginning --bootstrap-server 10.13.1.21:9092
hello world
but when I try and call “SendToKafka” I get:
2025-11-26 09:09:21:java.sql.SQLException: (conn=203266) Unhandled exception
Type: ER_WRITER_SUBPROCESS_FOR_OUTBOUND_FILE_CLOSED (2871)
Message: Leaf Error (127.0.0.1:3307): The writer subprocess for the outbound file closed unexpectedly. Got following ERROR. Failed to flush to topic: Local: Timed out.
Got 2 errors, first (Local: Broker transport failure): localhost:9092/1: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 6ms in state CONNECT)
Callstack:
#0 Line 7 in `charglt`.`SendToKafka` called from
#1 Line 71 in `charglt`....(truncated)
Process finished with exit code 0
I first was running Kafka in docker alongside S2. I’ve since set up a separate Kafka instance on a different server in case there’s a weird docker issue. I also know from experience that Kafka can be odd about hostnames, but I’ve no clue here.
So: Is it because I am using the Developer Image? Or have I made an idiot mistake somewhere else?
David, I asked my new friend Claude.ai and he gave some suggestions:
Yes, I can see the issue here. The error message gives it away:
The connection error shows “localhost:9092/1: Connect to ipv4#127.0.0.1:9092 failed: Connection refused” singlestore even though the user is specifying 10.13.1.21:9092 in their SELECT ... INTO KAFKA statement.
The problem: This is a classic Docker networking issue. When SingleStore inside the Docker container tries to connect to Kafka, it’s resolving the broker address to localhost:9092 instead of 10.13.1.21:9092. This happens because:
Kafka brokers advertise their addresses to clients
The Kafka broker at 10.13.1.21:9092 is likely advertising itself as localhost:9092 or the hostname that resolves to localhost
When SingleStore connects and gets the broker metadata, it tries to use that advertised address
From inside the Docker container, localhost refers to the container itself, not the host machine
Solutions:
Fix Kafka’s advertised listeners - Configure Kafka’s advertised.listeners to use the actual IP address (10.13.1.21) instead of localhost/hostname
Use Docker host networking - Run the SingleStore container with --network host so it shares the host’s network stack
Use host.docker.internal - If Kafka is on the Docker host machine, use host.docker.internal:9092 instead of the IP address (this works on Docker Desktop)
Put both in the same Docker network - If running Kafka in Docker too, create a Docker network and put both containers in it, then use the Kafka container name as the hostname
The developer image itself shouldn’t have limitations on SELECT ... INTO KAFKA - this is purely a networking configuration issue.