"Failed to find data source: memsql" error is observed while connecting to "memsql" from "PySpark"

[vishwajeetdabholkar]

Hi Ivan,

Please see the details at following location:

Using PySpark to write spark dataframe into memsql

Hi, I have a local memsql cluster setup on my Ubuntu 18 VM with 1 masternode, 1 aggregator node and 1 leaf node. I want to read a parquet file from s3 and save that data frame into memsql table using spark.write() following is what I have done and its working All I want to know if it is correct and is it okay to use this in real time. def read_from_s3(objName): # Reading from s3 df = spark.read.parquet(“s3n://python-s3-upload-test/”+objName) return df df = read_from_s3(‘sample.parquet’) …

Thanks,
Vishwajeet