Client-side load balancing

Based on the docs I’ve read, it sounds like the client needs to connect directly to a CRDB node, and I guess if the node contacted is not the raft leader of the underlying data in the operation(s), the request is forwarded to the leader. Does CRDB have plans to introduce a load-balancing policy like Cassandra? What’s the recommended way to do load balancing? Just use something like HA proxy and round-robin the requests to the available nodes? Will CRDB introduce a thick client at some point that has load balancing policies?

2 Likes

@msmith10101, in CockroachDB the node you talk to does the large share of the work (e.g., forwarding requests to replica leaders, as you say), so it’s definitely best to use some type of load balancing to distribute connections across nodes. You’ll get better performance (queries per second), and the client will be resilient to specific nodes failing.

We’re actually just starting to draft some docs on load balancing (GitHub issue). We’ll likely cover HAproxy with round-robin, which we’ve tested and can recommend, as well as connection pooling at the client driver level. I’ll update this thread once those docs are live.

As for a cockroachDB thick client, @peter, @marc, @dianasaur323, do you have any details?

@msmith10101 I’ve had some success using Lyft’s Envoy as a TCP proxy and their Service Discovery Service (SDS) checking against the health endpoints of Cockroach. You can choose the type of load balancing you require.

Doing load balancing at the client driver level would certainly be possible. This would require enhancing every language driver, though I suppose that can be done incrementally. There are no concrete plans for doing this work yet.

There has been discussion about adding a Cockroach Proxy node that would internally maintain a cache of range locations in order to reduce network hops. This would be similar to having a Cockroach node that has no disk storage. There are also no concrete plans to work on this as general performance and stability work is currently taking precedence.

Created a feature request for a Cockroach Proxy mode https://github.com/cockroachdb/cockroach/issues/24001