Hello Team…
i am new to singlestore and planning to explore kafka pipeline.
Background of current setup -
kafka setup is on AWS EMR
singlestore db is setup on AWS EC2 nodes
kafka and singlestore db are in same AWS VPC
Copied ca.pem to all the singlestore ec2 nodes
Using DbVisualizer to connect to singlestore db
I am using below code to create pipeline and facing issues. Please suggest if i have to be aware any configuration part in below code and also in AWS.
CREATE PIPELINE kafka_pipeline
AS LOAD DATA KAFKA ‘kafkabroker1-XXXX.com:19093,kafkabroker2-XXXX.com:19093,kafkabroker3-XXXX.com:19093/test_topic’
CONFIG ‘{“security.protocol”: “ssl”,
“ssl.ca.location”: “/etc/memsql/extra/CARoot.pem”}’
CREDENTIALS ‘{“ssl.key.password”: “XXXX”}’
INTO table demo;
ERROR :
[Code: 1933, SQL State: HY000] (conn=177124) Cannot get source metadata for pipeline. Could not fetch Kafka metadata; are the Kafka brokers reachable from the Master Aggregator? Is ssl.ca.location correct (tried ‘/etc/memsql/extra/CARoot.pem’)? Kafka error Local: Broker transport failure