Decommissioned nodes still affect admin UI cluster overview

I recently updated a cluster to v1.1.2, and decommissioned all dead nodes from previous versions (which works now, thanks @tschottdorf!). However, the decommissioned nodes still affect several things in the UI. First, the cluster still believes it has mixed versions, and the “Staggered Version” banner is continually displayed.

Also, the decommissioned nodes continue to appear in the “Graph” dropdown, adding quite a bit of clutter when trying to filter down to a single graph’s metrics.

Finally, (and most confusingly), the “Capacity Used” section in the summary pane on the right side shows “NaN%” indicating that both the used bytes and the capacity are believed to be 0. I’ve seen this happen while updating clusters before, but had normally resolved itself when the rolling upgrade was completed.

The storage charts appear correct, and checking the _status/node/{id} endpoint for the live nodes show that the nodes are correctly reporting their storage info:

        "capacity": {
          "capacity": "100000000000",
          "available": "98113769721",
          "used": "1886230279",
          "logicalBytes": "12400585235",
          "rangeCount": 1129,
          "leaseCount": 220,
          "writesPerSecond": 208.35817056121618,
          "bytesPerReplica": {
            "p10": 0,
            "p25": 0,
            "p50": 0,
            "p75": 16684462,
            "p90": 40041730
          "writesPerReplica": {
            "p10": 0,
            "p25": 0,
            "p50": 0,
            "p75": 0.5057812631832431,
            "p90": 0.5072402159927035

Screenshot showing several of these issues:

Hi Taylor,

good and bad news, starting with the good: we’re aware of these issues, and some (well, one) has been fixed (though unfortunately not in 1.1.2; will be in 1.1.3 though): the staggered version warning. For the rest, various bugs are filed (see roughly here:✓&q=is%3Aissue%20is%3Aopen%20decommissioned)

I filed something for the NaN bug though, haven’t seen that previously.

Thanks for reporting!