I’ve read the hardware recommendations and other posts on this forum about node size, and still wonder how best to handle a situation where the only storage available is on-premise servers that have big JBOD disks on them. In some cases people are simply limited to specific hardware given to us (e.g. ops department has limited rackspace and wants to buy hosts with 22 disks attached). It would be nice to know the best way to operate in these conditions, even if it isn’t the ideal recommendation.
The documentation clearly states that individual cockroach nodes should not exceed 4TB of data, and that we shouldn’t run multiple instances on the same physical host. But clearly we can’t satisfy both of those conditions with this kind of hardware.
It looks like putting multiple cockroach instances on a single host can work okay as long as we are using locality effectively and including the hostname as the last key in our locality spec. This should ensure we don’t end up with all the replicas of a range being on the same host together, right? This seems like a better way to run than to try to have one cockroach instance handle all that data.