What is the best way to monitor the overhead of the background block copying that will go on when adding or deleting nodes from cluster?
I’m not sure I understand your question, could you clarify a bit further? Are you asking about CPU usage during rebalancing?
The question may be more like ‘what metric do i watch to try to quantify performance degradation during rebalancing’? I know there will be some, and its actually been mentioned in a webinar by one of your customers. Blocks are being copied between nodes and ranges are being reorganized. I need to know what that impact will be as we would be scaling up and down periodically to accommodate very high traffic events. Having that flexibility is the main reason we have deployed in GKE. Ive tried adding and dropping nodes during a ‘workload’ run but the way the results are reported its not readily evident what the impact is.
Metrics that would be helpful to watch are CPU, network and disk io latency. These metrics can give you a good idea if there is any performance degradation happening when a node is added or removed.
Let me now if you have any other questions.