Proxying cockroach connection based on SNI routing

I currently have a CockroachDB cluster running in an Istio-enabled Kubernetes cluster, mostly with success.

I am trying to use SNI for routing from my ingress gateway, however it appears this doesn’t work. My guess is that it’s something do with the Postgres protocol (it’s a bit old, but see this post PostgreSQL: Re: Introducing SNI in TLS handshake for SSL connections) however I thought I’d check with the CockroachDB community to see if I’m right.

If I use the wildcard * in my hosts for the Gateway and VirtualService, this disables SNI-based routing and configures a direct TCP proxy from my gateway to the cockroach pods. With this setup I can successfully connect to the cockroach cluster, however this means I can only run a single cluster per port.

With SNI-based routing configured in my cluster, I can run openssl s_client -connect and I am served with the expected certificates, which confirms my Istio/Envoy configuration is correct. However when I use cockroach sql ... (using combinations of explicit URL or CLI params), the connection fails and my gateway reports the connection as not TLS (confirmed by the Envoy stat tls_inspector.tls_not_found). There’s nothing wrong with my CLI parameters because this works when not using SNI-based routing.

Does anyone have any knowledge of this kind of issue? Am I just doing something wrong or are my suspicions around it not being possible due to the postgres protocol correct?

Welcome to the forum @dan-j!

I’m sorry I don’t have a better answer for you, but CockroachDB doesn’t support SNI yet. To do so would break our Postgres compatibility in some places. We are waiting until SNI is fully supported by Postgres.

1 Like

Ah ok, I feared as much.

It has taken me many hours to actually convince myself that this was the case and not just a mistake on my part. It might be worth adding something to the docs to save others like me going down the same route. It wouldn’t take much explaining, so long as the keywords are there on the site Google searches would help users find the information they’re looking for.

I don’t mind contributing this, just would need a pointer to the right place.

Cheers

@dan-j that is a great suggestion. I will bring this to the docs team to make it more obvious that we don’t support SNI just yet. I appreciate your willingness to contribute! If there is a place in the docs that you think would be an good place for this, feel free to to use the Contribute dropdown to offer an update. Our team is always interested in hearing about how we can make things better.

1 Like