I am watching Video: Ingesting JSON Data Loading. While following this video, I am creating a pipeline to load json data from S3 bucket. I am doing the same as in the video but facing this error.
ERROR 1970 ER_SUBPROCESS_TIMEOUT_ERROR: Forwarding Error (node-6feb1713-52a1-44ba-9d1f-b09640029f41-master-0.svc-6feb1713-52a1-44ba-9d1f-b09640029f41:3306): Remote connection timed out because no metadata was received from source in pipelines_extractor_get_offsets_timeout_ms (35000 ms). Check your pipeline configuration. No stderr returned.
Here is my code:
create pipeline json_test
as load data S3 ‘testing-ingest-examples/a_file.json’
CREDENTIALS ‘{“aws_access_key_id”: “”, “aws_secret_access_key”: “”}’
INTO TABLE t1(a ← a::b) FORMAT JSON;
Anyone please help me what is wrong with this code. I am new to this platform. I am writing this code on https://portal.singlestore.com/ SQL editor.
I’m not sure if it’s the forum’s formatting or something else but you have a few wrong characters in your code.
" ` " instead of " ’ " and " “ " instead of " " ", if that makes sense
here’s how it should look like:
CREATE PIPELINE json_test
AS LOAD DATA S3 'testing-ingest-examples/a_file.json'
CONFIG '{"region": "us-east-1"}'
CREDENTIALS '{"aws_access_key_id": "", "aws_secret_access_key": ""}'
INTO TABLE t1(a <- a::b) FORMAT JSON;
That’s a bit odd. Just tested it on my cluster, the bucket is fine.
Could be that your cluster can’t reach the bucket for some reason. Are you running on SingleStoreDB Cloud?
@Kostya I tried it but no success. Created new workspace, new database and re-run the code but it’s giving the same error. I am install on-prem server of singlestore. I’ll let you know once I tried on on-prem server.
