CDB secure cockroach in k8s prometheus scrap failed with cert error

Secure CDB running in K8s with provided certs, the node certs are as per the example provided in the documentation
cockroach cert create-node --certs-dir=certs --ca-key=root-cert/cockroach.key localhost 127.0.0.1 cockroachdb-public cockroachdb-public.default cockroachdb-public.default.svc.cluster.local *.cockroachdb *.cockroachdb.default *.cockroachdb.default.svc.cluster.local
Everything is working except Prometheus scrap (prometheus-operator) in tlsConfig is set to ignore cert validation, error message Get https://10.11.108.3:8080/_status/vars: x509: certificate is valid for 127.0.0.1, 127.0.0.1, not 10.11.108.3