authentication handshake failed: x509: certificate signed by unknown authority

While deploying cockroachdb using helm chart on kubernetes cluster, we are getting the error “authentication handshake failed: x509: certificate signed by unknown authority”

We did manually approve all the certificate requests

Any help will be highly appreciated

W200127 16:45:29.688740 494 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:29.690303 553 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:30.282543 502 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.14.37:52490”: remote error: tls: bad certificate
W200127 16:45:30.690504 560 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:30.690548 553 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:30.941396 534 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.9.70:51478”: remote error: tls: bad certificate
W200127 16:45:31.111157 507 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.11.73:42980”: remote error: tls: bad certificate
W200127 16:45:31.690816 560 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:31.693978 508 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:32.694262 508 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:32.699227 580 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:32.944057 566 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.9.70:51562”: remote error: tls: bad certificate
W200127 16:45:33.116260 586 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.11.73:43140”: remote error: tls: bad certificate
W200127 16:45:33.699589 580 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:33.703220 599 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:34.703524 599 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:34.706965 610 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:34.948297 589 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.9.70:51672”: remote error: tls: bad certificate
W200127 16:45:35.124941 591 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.11.73:43348”: remote error: tls: bad certificate
W200127 16:45:35.362191 605 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.14.37:52634”: remote error: tls: bad certificate
W200127 16:45:35.707192 610 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:35.714371 642 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:36.712756 647 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:36.715114 642 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:36.951048 608 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.9.70:51758”: remote error: tls: bad certificate
W200127 16:45:37.127330 658 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.11.73:43498”: remote error: tls: bad certificate
W200127 16:45:37.713087 647 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:37.715144 652 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:38.715574 652 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:38.716027 640 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
W200127 16:45:38.955968 624 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.9.70:51834”: remote error: tls: bad certificate
W200127 16:45:39.133264 690 vendor/google.golang.org/grpc/server.go:649 grpc: Server.Serve failed to complete security handshake from “172.16.11.73:43668”: remote error: tls: bad certificate
W200127 16:45:39.716410 640 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-1.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: Error while dialing cannot reuse client connection”. Reconnecting…
W200127 16:45:39.724821 676 vendor/google.golang.org/grpc/clientconn.go:1206 grpc: addrConn.createTransport failed to connect to {s7e-crdb-cockroachdb-2.s7e-crdb-cockroachdb.demo-devtest.svc.cluster.local:26257 0 }. Err :connection error: desc = “transport: authentication handshake failed: x509: certificate signed by unknown authority”. Reconnecting…
I200127 16:45:40.250352 1 cli/start.go:865 received signal ‘terminated’
I200127 16:45:40.267601 1 cli/start.go:930 initiating graceful shutdown of server
initiating graceful shutdown of server
I200127 16:45:40.290445 1 cli/start.go:981 too early to drain; used hard shutdown instead

hi @hound,

Welcome to the Cockroach Forum! Sorry for the delay in reponse.

You mentioned that you manually approved all the certificate requests.
To better assist you, would you be able to check the CSR requests by running
kubectl get csr
?

The CSRs should have CONDITION set to Approved and Issued

Please let us know if this is the case.

Regards,
Florence