I’m evaluating Cockroach for use in one of our products. We are using helm, so I’m following the helm based kubernetes orchestration guides from the website: https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb-with-kubernetes.html
We are using a fairly standard EKS setup, using Kubernetes 1.11.
Setting up an insecure cluster works perfectly. However, setting up a secure cluster runs into issues with volume binding. The only difference between the two setups is that the secure flag is enabled.
This is what my values looks like for the secure cluster, just fyi:
Secure: Enabled: true Storage: "32Gi" StorageClass: gp2
The specific error message (with details censored) I’m running into with the helm chart is:
pod has unbound PersistentVolumeClaims (repeated 4 times) AttachVolume.Attach failed for volume "pvc-****" : "Error attaching EBS volume \"vol-****\"" to instance "i-******" since volume is in "creating" state Readiness probe failed: Get https://*****:8080/health?ready=1: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
I’m really not sure why the secure flag causes this behaviour. The pods never recover from this state, and deleting pods obviously just results in the same thing happening to the new pod.