Cockroach db multi region expose

Hello,

I have set up a cockroach DB multi-region database on GCP Kubernetes. One is in Asia and the other is in the US region. Now I want to expose the database to the public because my front end apps are running on AWS. My question is can I expose the database to the global load balancer of GCP?

If I can’t expose it to the GCP global load balancer and use regional TCP load balancer then How my app will forward traffic to the nearest database location? Because I have to expose 2 TCP load balancer in each region.

Kindly share the best practice to access a multi-region database publically using a single IP address or any other way to access the database.

Hello, Cockroach db support team,

I am new to cockroach DB and exploring some of your new databases features and apply them in our production services. You should help us to explore more of your products. We want to use cockroach in our organization. I think i am not getting good support from your forum.

If we implemented this service successfully in our products then we are planning to buy enterprise features also.

Hi shivraj,

Sorry it took so long to get back to you. Generally it’s not a great idea to expose the single Cockroach cluster behind a single load balancer. In order to best achieve optimal latency with crdb you’ll want to configure application servers to communicate with cockroach nodes which are physically proximate. This generally means that the best practice is to deploy regional TCP load balancers which each have their own IP. If I understand correctly, I believe this is what you say you’ve done. Then in AWS you should configure your application servers to use the IP address of the closer cockroach region.

In front of your application servers it then makes sense to use a global layer 7 load balancer.

Consider consulting https://www.cockroachlabs.com/docs/v19.2/topology-patterns.html#multi-region-patterns for cockroach deployment tips or https://www.cockroachlabs.com/blog/follower-reads/ for an example of deploying an application against multiple regions.

Let me know if you have any follow up questions and I’ll be happy to help.

-Andrew

1 Like

Thanks for the suggestion. But still I am not able to understand the flow of multi region cluster on kubernetes. How to connect all the clusters in our app.

I have frontend applications deploy in asia and europe region. Now i want same for databases, Cockroach db will be deployed on asia and europe region. I am using .env file for frontend app to use db.

Below is my env file. for single region cluster i have expose the cockroachdb-public service to the cluster ip to LoadBalancer and used LB ip in my env file to connect DB from My apps. But when multi region involve in this, I am confise that how can i connect cockroach multi region from my apps.

ENV= production
DBHOST=cockroach-lb-ip
DBPORT=26257
DBUSER=my-user
DBNAME=my-db
DBPASSWORD=mypassword

One way to achieve your goals is to deploy each region of your database as a separate k8s stateful set. You’ll need to make sure that the nodes in Asia and Europe can communicate with each other. Each region should have its own load balancer. For example cockroach-lb-europe and cockroach-lb-asia. Application servers in Asia should connect to the cockroach servers in Asia and application servers in Europe should connect to the cockroach servers in Europe.

@ajwerner I have set up the same as you said. But what if Asia DB down? my application server from Asia will be able to communicate with Europe?

Because I have deployed multi-region applications in both regions, So data is the same for both the application.

I followed this documentation.

https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb-with-kubernetes-multi-cluster.html

You could implement failover logic to detect that case in the application but I wouldn’t recommend it. Instead I’d recommend that you use a layer-7 load balancer in front of all of your application servers (e.g. https://cloud.google.com/load-balancing/). You can have the load-balancer health check the different backends to which it routes. If your asia database becomes unhealthy due to a region failure then your application servers in asia should fail their health check. That should lead to the global load balancer routing traffic to european application servers and ultimately to european database nodes.

I prefer this approach both because of its simplicity and because multi-statement operations will be much slower if the application server communicates with a faraway database node if you were using a topology pattern which was providing fast global reads such as follower reads or duplicate indexes. If you had been using geopartitioning then the failure of the asian database nodes will incur at least partial unavailability in your cluster for asian users regardless.