I have 3 nodes where each node got 6 SSD disks.
As per production recommendations https://www.cockroachlabs.com/docs/stable/recommended-production-settings.html we run one instance per node with multiple stores.
Command line looks like that
/usr/local/bin/cockroach start --certs-dir=/etc/cockroach/ssl --http-host=127.0.0.1
–http-port=8080 --cache=5% --max-sql-memory=80%
What I see, when cluster is initialized, and start filling data, is that, instead of spreading ranges across multiple disks, it is selecting 1 disk per node and start filling it.
This can be verified by tools like iostat, node exporter, df etc where we can see that all of IO goes to one disk, usage goes to one disk etc.
This is especially true at the beginning when database is freshly initialized and I start filling it.
Is there a way I can control the way the disks should be utilized something like creating new ranges in round robin.
What do you think about this in general? Does this make any sense at all?
My idea is that if we have multiple disks if we write into them simultaneously we will get higher write throughput.