Is there any good way of tracking the status of replication/rebalancing when scaling out? What I’m interested in is learning when a particular rebalance has finished. I have a “feel” for it by looking into the replica count on the node list of the Web UI but it’s not very intuitive and I’m not 100% sure what these numbers represent and what’s their precise relation with a rebalancing process.
** Note: this is a copy of a community asked question on a different communication channel
You can watch the “Replicas Per Store” graph on the admin UI. Rebalancing is “done” when all the nodes stabilize near each other.
Just to add a bit more here. Rebalancing is always occurring. It’s not a one time operation. When any store starts to fill up more than another, it will automatically rebalance away from replicas. And if the network topology changes, such as adding a new node or removing one, rebalancing with automatically occur.
I think the real question here is “can we have a green light in the UI to tell us ‘nodes are currently well-balanced’”.
And/or perhaps a red light per node to tell us ‘this node has too few/too many replicas compared to other nodes’.