Using PySpark to write spark dataframe into memsql

Help Connectors and Pipelines
15

Hi Ivan,

Thank you for all the help.
I am able to read data from memsql table withe the SPARK connector details you gave.
and for the show() issue, I did downgraded my java version (from java11 to java8) and now everything is working fine!!

For others who might face the same issue, I used the following configurations:

for Spark 2.4:
memsql 7
Python 3.7
Java 8

and while starting the spark terminal use:

pyspark --packages com.memsql:memsql-spark-connector_2.11:3.0.5-spark-2.4.4

Thanks,
Vishwajeet

1 Like