Hi.
I am not comfortable knowing that with High Availability set to True, there are only two copies of my data in the Leaf Nodes.
As the Leaf Nodes are paired, each pair keeps the only copies of the shards assigned to them.
Under these rules how can another separate copy of those shards be kept?
Can more Leaf Nodes be added to keep another two copies of all the shards in the cluster?
I do understand a Database can be replicated to another cluster. This has obvious advantages but is also an expensive way of keeping extra copies of one’s data.
What would the normal setup be to solve this problem please?
Hi jlb,
Right now we only support storing 2 copies within one cluster. As you mentioned running a 2nd cluster and replicating the database is one way of getting a second copy, but that has other trade-offs (the replication link between the two cluster is asynchronous for one thing).
You can manually create more copies of each partition, but singlestore won’t maintain this configuration for you (in the sense that if nodes die or you expand the cluster… it won’t create or move the extra copies around to keep the cluster balanced. You would need to do it manually), so you would have to script it out.
We will add support for storing more copies in a future release.
-Adam
Thank you very much Adam.
Hi Adam.
Do the following references therefore imply manually copying shards(partitions):
(Manual copying seems to me to be quite an arduous option but is maybe not actually so)
- Blog
Replication at Speed – System of Record Capabilities for SingleStore DB 7.0
[Replication at Speed – System of Record Capabilities for SingleStoreDB Self-Managed 7.0]
"Because SingleStore is a distributed database, it can implement a highly available system by keeping multiple copies of the data, and then failing over to another copy in the event that it detects a machine has failed."
- YouTube
SingleStore: Separation of Storage and Compute for Transactions and Analytics (Joyo Victor)
[SingleStore: Separation of Storage and Compute for Transactions and Analytics (Joyo Victor) - YouTube]
At time segment 2 minutes 30 seconds Joyo Victor states "...So each partition has one primary replica and one or two secondary replicas..."
This is also listed as an entry in the slide at that same time point as follows:
Thanks very much again for helping me to understand the whole system more extensively.