Hi,
Please find the Answers here:
Q)When you say that you are using memsql version 2.0.1, are you referring to the version of the MemSQL Spark Connector?
Ans:Yes.memsql-connector_2.11,version 2.0.1
Q)If so, can you confirm what version of MemSQL you are using as well? You can do so by running select @@memsql_version
in the database.
Ans:6.8.12
Q)Can you think of any changes made to the Spark data frame, Connector settings, the MemSQL database, or the table you are loading into in MemSQL on April 16, 2020?
Ans:No changes made
Q)Can you provide the makeup of the table you are loading into in MemSQL by running SHOW CREATE TABLE <table_name>?
Ans:
CREATE TABLE sm_meta_table
(
app_name
text CHARACTER SET utf8 COLLATE utf8_general_ci,
table_name
text CHARACTER SET utf8 COLLATE utf8_general_ci,
record_count
bigint(20) DEFAULT NULL,
hdfs_path
text CHARACTER SET utf8 COLLATE utf8_general_ci,
timestamp
text CHARACTER SET utf8 COLLATE utf8_general_ci,
write_success
bit(1) DEFAULT NULL,
memsql2write_status
text CHARACTER SET utf8 COLLATE utf8_general_ci
/*!90618 , SHARD KEY () */
)
Q)Can you provide a sample output of your data frame that you are loading from in spark using df.show()?
--------±-------------------±-----------±-------------------±------------------±------------±------------------+
|app_name| table_name|record_count| hdfs_path| timestamp|write_success|memsql2write_status|
±-------±-------------------±-----------±-------------------±------------------±------------±------------------+
| My_App|my_table | 39162|hdfs://test/… |2020-04-19 08-41-54| true| STARTED|
±-------±-------------------±-----------±-------------------±------------------±------------±------------------
The same values reflect on Memsql table too except write_success,which gets converted to 0
Thanks
Smitha