I have deployed Cockroachdb on 6 nodes, across 2 sites in one cluster. We have almost zero latency between the sites but need to account for either DC potentially becoming unavailable.
What would be the best way to configure the cluster.
Currently, I have setup 2 zones with 3 nodes each, but when we simulate a zone failure the other site becomes unavailable. We have tried replication factors of 3,5 and 6. but none seem to help.
Would it be better to rather create one zone with 6 nodes and set replication factor to 3? Would that be more survivable? Currently there is no option for a 3rd DC, so creating a 3rd zone is out of the question.
Any advice would be appreciated, thanks.