Lab: Ingesting JSON Data

Hi Guys,

I am watching Video: Ingesting JSON Data Loading. While following this video, I am creating a pipeline to load json data from S3 bucket. I am doing the same as in the video but facing this error.

ERROR 1970 ER_SUBPROCESS_TIMEOUT_ERROR: Forwarding Error (node-6feb1713-52a1-44ba-9d1f-b09640029f41-master-0.svc-6feb1713-52a1-44ba-9d1f-b09640029f41:3306): Remote connection timed out because no metadata was received from source in pipelines_extractor_get_offsets_timeout_ms (35000 ms). Check your pipeline configuration. No stderr returned.

Here is my code:

create pipeline json_test
as load data S3 ‘testing-ingest-examples/a_file.json’
CREDENTIALS ‘{“aws_access_key_id”: “”, “aws_secret_access_key”: “”}’
INTO TABLE t1(a ← a::b) FORMAT JSON;

Anyone please help me what is wrong with this code. I am new to this platform. I am writing this code on https://portal.singlestore.com/ SQL editor.

Hi BilalWaris,

I’m not sure if it’s the forum’s formatting or something else but you have a few wrong characters in your code.
" ` " instead of " " and " " instead of " " ", if that makes sense :slight_smile:

here’s how it should look like:

CREATE PIPELINE json_test
AS LOAD DATA S3 'testing-ingest-examples/a_file.json'
CONFIG '{"region": "us-east-1"}'
CREDENTIALS '{"aws_access_key_id": "", "aws_secret_access_key": ""}'
INTO TABLE t1(a <- a::b) FORMAT JSON;
1 Like

My code is same as of yours. I tried your code too but same error. I think problem is with S3 bucket file path.

That’s a bit odd. Just tested it on my cluster, the bucket is fine.
Could be that your cluster can’t reach the bucket for some reason. Are you running on SingleStoreDB Cloud?

Yes, I am using SingleStore Cloud. It’s on https://portal.singlestore.com/

You’re missing the CONFIG line in your code.


Still same error. I am from Pakistan, so tell me what region should I use if my region is not " us-east-1".

Could you try please creating another workspace and re-run it there?
You can detach the database from this one and attach it to the new one.

@Kostya I tried it but no success. Created new workspace, new database and re-run the code but it’s giving the same error. I am install on-prem server of singlestore. I’ll let you know once I tried on on-prem server.

One other thing to try is to add actual AWS credentials to the CREDENTIALS line.

1 Like

@rachelgomez161999 Can you please add Screen Shot of it. I am unable to find Data or All action section on SingleStore cloud service.