CockroachDB and nodes at sea

replication

#1

Just wondering about the suitability of CDB for an at sea application. Satellite leases to ship can be intermittent and have high latency.

If I understand the scenario, if a ship loses connectivity (loses quorum) it would continue to service reads for a while until the read lease expired. Then it would go dormant until it could see other nodes and reach consensus.

But what happens if you set your replication factor very low. Say a shore facility has 5 nodes and there are three ships at play. All ships lose connectivity to the shore, but they can still communicate with each other. Locality is used to make sure that the ships have full data.

If I understand this correctly, the replication factor was 3, both segments of network would maintain quorum, would rebalance and continue to accept reads and writes…but what would happen when the two segments were rejoined into one happy network again.

Would writes obtained during the network outage be populated to amongst all nodes? Would rebalancing occur again?

tks!


(Ron Arévalo) #2

Hey @Tarpond,

First off, very interesting scenario! Now to answer your questions. If a ship loses connectivity, and by extension quorum it would not serve writes as you need quorum for writes but it would still serve reads if a leaseholder is present. If there is no leaseholder then you would lose reads and writes from a ship.

If a leaseholder is present and you have a quorum or replicas then you would be able to both read and write.

If a node is unavailable, and becomes available again, you would see that data and the cluster would rebalance again, however if for whatever reason that node is lost or damaged beyond repair, then that data on that node would be lost.

Let me know if you have any other questions.

Ron


(Ron Arévalo) #3

@Tarpond,

Also to clarify, in a partitioned network, only one partition will have the liveness range lease, and only that partition will be able to make progress. Consensus is not possible at the range level.