I thought I’d share something I learned. I was struggling figuring out how to pass the right information into the pipeline to get access to the minio bucket. I saw another post related to S3, but this didn’t quite get me there. This is for ingesting a csv file. After some trial and error, this is what I ended up with:
create or replace pipeline mypipeline as
load data S3 ‘mybucket/myfolderinbucket’
CONFIG ‘{“endpoint_url”:“http://192.168.1.xxx:9000/”}’
CREDENTIALS ‘{“aws_access_key_id”: “accesskeyfromminio”, “aws_secret_access_key”: “secretkeyfromminio”}’
replace into table mytable
columns terminated by ‘,’
OPTIONALLY ENCLOSED BY ‘"’
ignore 1 lines
(column1,
column2,
column3,
column4);
1 Like