Using MemSQL in AWS Glue (as source and target)

saurshan · January 27, 2021, 4:25pm

Hi Ömer,

I hope you were able to use SingleStore with AWS Glue. You would also find the AWS blog post on this topic useful https://aws.amazon.com/blogs/big-data/building-fast-etl-using-singlestore-and-aws-glue/

Thanks.

oemer.percin · February 15, 2021, 2:26pm

Hi saurshan,
this looks very promising. We will check and get back soon.
However I do not see how you are querying on the MemSQL/Singlestore database there natively. As far as I understood, you are doing this via Python in Glue.
Thanks,
Ömer

oemer.percin · February 22, 2021, 10:03am

Hi saurshan,
we have tested this in our environment.
Having the need to create schemas manually is very cumbersome. Is there a way how this can be achieved automatically?
Also, we do not want to transform the data in Glue. We want to run SQL queries and only orchestrate the process through Glue. Is there an easy way just to connect to SingleStore and run SQL queries with that marketplace extension? We have tried but were not able to. Please reach out to us.
Thanks,
Ömer