As we are coming from Hadoop world, we have been there with 3 copy of data redundancy to handle double fault scenario.
On going through the High Availability solution of SinglestoreDB, I understand that we can configure only maximum 2 copy of data for high availability. But as the cluster size grows more the risk of node failure increases and proportionally we should also secure the data as going offline because of two node failure will be costly for production deployment.
Query : Can I please know, is there a configuration available right now to increase the replication count to 3 or 5 (for very large cluster) ? If not, is there any roadmap item planned to make it flexible here?
We support maximum redundancy of two right now. At some point we plan to support redundancy 3. However, compared with Hadoop, there are a few important things to know:
If you use our cloud service with unlimited storage databases, data more than about two minutes old will be on object storage, such as S3, which has extremely high durability (lots of nines) – effectively you will never lose data on the object store unless the whole region is lost.
We heal reduced redundancy back to redundancy two automatically, and quickly. We have some customers with up to 200 nodes, though the large majority are satisfied with under twenty notes. Our QP is so much faster than Hadoop, large clusters are not usually necessary. I personally have never seen a customer lose data due to lost redundancy in the last five years.