CockroachDB CAP tradeoffs

I first learned about CockroachDB in the fall when the BSDNow podcast covered it running on freebsd and the “Critters in a Jar” blog post. The project looks interesting so i have been lurking.

One thing i have never seen discussed are the CAP tradeoffs that CockroachDB makes. After asking on twitter I was informed that official documentation is in the works, and in the mean time to ask here for more information.

For those that dont know CAP theorem, in the event of a failure take

  • Consistency
  • Availability
  • Partition tolerance

Pick any two but one of them needs to be Partition tolerance

The response i got on twitter says that is a CP system yet still had High Availability, what tradeoffs are made to accomplish this

Yes, CockroachDB is a CP system. That means that in the presence of partitions, the system will become unavailable rather than do anything which might cause inconsistent results. For example, writes require acknowledgements from a majority of replicas, and reads require a lease (which can only be transferred to a different node when writes are possible).

Separately, CockroachDB is also Highly Available, although “available” here means something a bit different than the way that term is used in the CAP theorem. In the CAP theorem, availability is a binary property, but for HA we talk about availability as a spectrum (using terms like “five nines” for a system that is available 99.999% of the time).

So being both CP and HA means that whenever a majority of replicas can talk to each other, they should be able to make progress. For example, if you deploy CockroachDB to three datacenters and the network link to one of them fails, the other two datacenters should be able to operate normally with only a few seconds’ disruption. We do this by attempting to detect partitions and failures quickly and efficiently, transferring leadership to nodes that are able to communicate with the majority, and routing internal traffic away from nodes that are partitioned away.

3 Likes