After a few days of running the pipeline just stop with the following error in GCS. We are using the latest version 7.3.3. It seems that it started to happen after we upgraded the version.
To fix this we drop and recreate the pipeline, and all the data is ingested correctly but after 2 or 3 days… it fails again with the same error. Is there some workaround for this? Thanks!
Leaf Error (127.0.0.1:3307): Leaf Error (127.0.0.1:3307): Cannot **extract** **data** **for** pipeline. AWS request failed:
NotFound: **Not** **Found**
status code: 404, request id: , host id:
Cannot **extract** **data** **for** pipeline. AWS request failed:
NotFound: **Not** **Found**
status code: 404, request id: , host id:
This is the pipeline statement:
CREATE PIPELINE `ingested_data`
AS LOAD DATA GCS 'bucket/folder1/folder2/'
CONFIG '{\"region\": \"us-east4\"}'
CREDENTIALS '{"access_id": "sdfdfs", "secret_key": "sdfdfs+sfdfs"}'
BATCH_INTERVAL 200
SKIP ALL ERRORS
INTO TABLE `table`
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n' STARTING BY ''
(
`table`.`field`
);