Multi region cluster with pd-ssd k8s

I am trying to deploy multi region cockroach db cluster in k8s GCP. It was working fine with default settings with standard storageclass. But we want to use pd-ssd staorage class. I have made some changes for the same.

Create storage class :

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-ssd
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

and then change i changed in cockroachdb-statefulset-secure.yaml.

      terminationGracePeriodSeconds: 60
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
      - name: certs
        emptyDir: {}
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes:
        - "ReadWriteOnce"
      storageClassName: pd-ssd
      resources:
        requests:
          storage: 50Gi

above setting works with single region cluster. but when i setup multi region , Pods not getting started. below are the logs.

2019-12-12T06:13:39.639781145Z Check your certificate settings, set --certs-dir, or use --insecure for insecure clusters.
 E 
2019-12-12T06:13:39.639786988Z 
 E 
2019-12-12T06:13:39.639842005Z failed to start server: problem with CA certificate: not found
 E 
2019-12-12T06:13:39.641742535Z Error: cannot load certificates.
 E 
2019-12-12T06:13:39.641759821Z Check your certificate settings, set --certs-dir, or use --insecure for insecure clusters.
 E 
2019-12-12T06:13:39.641765818Z 
 E 
2019-12-12T06:13:39.641770565Z failed to start server: problem with CA certificate: not found
 E 
2019-12-12T06:13:39.641775654Z Failed running "start"
 E 

Help me to create ssd storage with multi region setup on k8s.

I am stuck with SSD volume in multi region setup on kubernetes. need some help for the same.

1 Like

@shivraj Seems like your certs not mounted on volume. You can try below config. It should work.

open your cockroachdb-statefulset-secure.yaml and replace it with below.

      # No pre-stop hook is required, a SIGTERM plus some time is all that's
      # needed for graceful shutdown of a node.
      terminationGracePeriodSeconds: 60
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
      - name: certs
        secret:
          secretName: cockroachdb.node
          defaultMode: 256
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes:
        - "ReadWriteOnce"
      storageClassName: pd-ssd
      resources:
        requests:
          storage: 50Gi

1 Like