We are running memsql 6.8(128 GB free license) with a heterogeneous workload and we see the leaves occasionally exceeding the provisioned memory limit leading to query errors. None of the tables are using a lot of memory and profiling hasn’t revealed any bad actors.
memsql-report is showing the following errors:
FAIL Found 14999 'buffer manager memory allocation failure' errors in last 7 days for 18.104.22.168:3306 (C99E9EBDD2) FAIL Found 14350 'buffer manager memory allocation failure' errors in last 7 days for 22.214.171.124:3306 (C99E9EBDD2) FAIL Malloc_active_memory too high on node C99E9EBDD25CFC195BDDD65DB0D7551E7BA3CA0E (5.16 GB) FAIL Malloc_active_memory too high on node 01D22BEB264B9C8991927CCFDD2027E51552F3B4 (5.29 GB)
From show status extended the below component looks large on both the leaves…
Alloc_durability_large 22101.626 MB
Please let me know how we can go about debugging this further