Hi,there is a problem in Cluster Monitoring.
I’ve followed the instruction of the manual on SinglesStore web site to configure monitoring.
At the beginning,it is normally,but a problem occurred after a few minutes.
Then i did it again,but the problem still exists .
I don’t know where the problem is and i don’t know how to find the cause of the problem.
Thank you!
SELECT * FROM information_schema.pipelines_errors;
Here is CSV file:
DATABASE_NAME,PIPELINE_NAME,ERROR_UNIX_TIMESTAMP,ERROR_TYPE,ERROR_CODE,ERROR_MESSAGE,ERROR_KIND,STD_ERROR,LOAD_DATA_LINE,LOAD_DATA_LINE_NUMBER,BATCH_ID,ERROR_ID,BATCH_SOURCE_PARTITION_ID,BATCH_EARLIEST_OFFSET,BATCH_LATEST_OFFSET,HOST,PORT,PARTITION
metrics,192.168.150.174_9104_blobs,1626612002.12413,Error,1844,Leaf Error (192.168.150.174:3307): Invalid JSON value for column ‘vals’,Internal,NULL,NULL,NULL,9,53,NULL,NULL,NULL,192.168.150.174,3306,NULL
metrics,192.168.150.174_9104_blobs,1626612002.124064,Error,1844,
Leaf Error (192.168.150.174:3307): Leaf Error (192.168.150.174:3307): Invalid JSON value for column ‘vals’,
Internal,NULL,NULL,NULL,9,44,NULL,NULL,NULL,192.168.150.174,3306,NULL
metrics,192.168.150.174_9104_blobs,1626612002.1179,Error,1844,Invalid JSON value for column ‘vals’,Load,NULL,
“{”“keys”“:”“{"“activity_name"”:"“Select_metrics__et_al_e529638156a3a4a3"”}”“,”“memsql_tags”“:”“{"“cluster"”:"“memsql_cluster"”,"“host"”:"“192.168.150.174:9104"”,"“port"”:"“3306"”,"“push_job"”:"”"“,"“role"”:"“ma"”}”“,”“time_sec”“:”“1626612002"”,““type””:““query””,““vals””:“”{"“plan_warnings"”:"“"”,"“query_text"”:"“select * from (\\nselect time_sec,\\nCONCAT(^,host) as metric,\\nmemsql_sysinfo_mem_used_b / memsql_sysinfo_mem_total_b as used\\nFROM\\n(SELECT\\nmax(time_sec) time_sec,\\navg(value) value,\\nmetric,\\nhost\\nFROM\\n(SELECT\\n time_sec,\\n time_sec DIV @ * @ tg,\\n intval as value,\\n name as metric,\\n CONCAT(^, trim_host(host), ^) as host\\nFROM metrics\\nWHERE cluster=^\\nand (host in (^) or ^ = ^)\\nand extractor = ^\\nand subsystem = ^\\nand name like ^\\nand time_sec \u003e= @ AND time_sec \u003c= @\\n) X\\nGROUP BY tg, metric, host\\n) as pvt_data\\nPIVOT (\\n AVG(value) \\n FOR metric \\n IN (\\"“memsql_sysinfo_mem_total_b\\"”, \\"“memsql_sysinfo_mem_used_b\\"”)\\n) as pvt_table) a\\norder by time_sec"”}“”}",
metrics,192.168.150.174_9104_blobs,1626611986.667857,Error,1844,Leaf Error (192.168.150.174:3307): Invalid JSON value for column ‘vals’,Internal,NULL,NULL,NULL,8,35,NULL,NULL,NULL,192.168.150.174,3306,NULL
metrics,192.168.150.174_9104_blobs,1626611986.667789,Error,1844,Leaf Error (192.168.150.174:3307): Leaf Error (192.168.150.174:3307): Invalid JSON value for column ‘vals’,Internal,NULL,NULL,NULL,8,26,NULL,NULL,NULL,192.168.150.174,3306,NULL
metrics,192.168.150.174_9104_blobs,1626611986.65621,Error,1844,Invalid JSON value for column ‘vals’,Load,NULL,
“{”“keys”“:”“{"“activity_name"”:"“Select_metrics__et_al_e529638156a3a4a3"”}”“,”“memsql_tags”“:”“{"“cluster"”:"“memsql_cluster"”,"“host"”:"“192.168.150.174:9104"”,"“port"”:"“3306"”,"“push_job"”:"”"“,"“role"”:"“ma"”}”“,”“time_sec”“:”“1626611986"”,““type””:““query””,““vals””:“”{"“plan_warnings"”:"“"”,"“query_text"”:"“select * from (\\nselect time_sec,\\nCONCAT(^,host) as metric,\\nmemsql_sysinfo_mem_used_b / memsql_sysinfo_mem_total_b as used\\nFROM\\n(SELECT\\nmax(time_sec) time_sec,\\navg(value) value,\\nmetric,\\nhost\\nFROM\\n(SELECT\\n time_sec,\\n time_sec DIV @ * @ tg,\\n intval as value,\\n name as metric,\\n CONCAT(^, trim_host(host), ^) as host\\nFROM metrics\\nWHERE cluster=^\\nand (host in (^) or ^ = ^)\\nand extractor = ^\\nand subsystem = ^\\nand name like ^\\nand time_sec \u003e= @ AND time_sec \u003c= @\\n) X\\nGROUP BY tg, metric, host\\n) as pvt_data\\nPIVOT (\\n AVG(value) \\n FOR metric \\n IN (\\"“memsql_sysinfo_mem_total_b\\"”, \\"“memsql_sysinfo_mem_used_b\\"”)\\n) as pvt_table) a\\norder by time_sec"”}“”}",
Grafan screenshot
