Error handling - insert error information into a table

Hello,

We have a pipeline which inserts event data from kafka into several tables via execution of several stored procedures. We need to store failed batches errors into a table of errors. How can we extract the error from a failed procedure?

We want to do something like this:
INSERT INTO [LearningErrorLog]
(
ErrorNumber
,ErrorDescription
,ErrorProcedure
,ErrorState
,ErrorSeverity
,ErrorLine
,ErrorTime
)
VALUES
(
ERROR_NUMBER()
,ERROR_MESSAGE()
,ERROR_PROCEDURE()
,ERROR_STATE()
,ERROR_SEVERITY()
,ERROR_LINE()
,GETDATE()
);

I see the singlestore studio auto completes the function ERROR_MESSAGE, but then says it’s not defined.

Any help would be appreciated.

Hi Maximg and welcome aboard! :wave:
We’d love to help with this issue.
Are you running on the managed or self-hosted service and what version number?

Hi Maria,

I’m running in the self hosted currently.
SingleStore DB Version: 7.8.14
Studio Version: 4.0.7

Thanks

hi @maximg did you check the pipeline error table that already captures the errors messages ?

Regards,
Manish Kumar

Hello @mkumar, no the insertion errors are not present there. what about my original question, getting last error information?

Hi Maximg. Hope you find this helpful!

You may be able to get the information you need within the information_schema and insert those values into another table as needed.

All errors from pipelines can be viewed on a table called

information_schema.PIPELINE_ERRORS

Here is a useful query to show all of the errors that occur in the pipeline:

SELECT DATABASE_NAME, PIPELINE_NAME, ERROR_UNIX_TIMESTAMP, BATCH_ID, PARTITION, BATCH_SOURCE_PARTITION_ID, ERROR_KIND, ERROR_CODE, ERROR_MESSAGE, LOAD_DATA_LINE_NUMBER, LOAD_DATA_LINE FROM information_schema.PIPELINES_ERRORS;

Here is page that goes over how to handle pipeline errors.

If you can share what worked for you, others in the community may benefit from your postings. Cheers :beers:

Hey @mkumar and @MariaSilverhardt,

Thanks for the support. The default pipeline errors are not enough. I want to add custom data that’s available during the error handling.

That’s why I’m asking about a way to extract the error data. To add it to the custom stuff.

Cheers, Maxim

Hi Maxim, can you please paste your code and the ‘not defined’ error that you are getting back here? this will help us determine the issue you are having

I see the singlestore studio auto completes the function ERROR_MESSAGE, but then says it’s not defined.

1 Like