Replica ranges and understanding of replication (replication lag)

Am newbie to CRDB environment, still getting deep into it - Could anybody kindly get me understand around the ranges/replicas_leaders/replicas_leaseholders.

Here is the snapshot of node status --all command and the values differ on replica nodes which I suspect result in the inconsistent result set from all nodes of command “SELECT COUNT(*)” - Am I missing anything as I am aware around the fact that there is no-lag found any time in any of the CRDB node from a cluster?

FYI - I executed “node status command” and SELECT count(*) command in single statement from terminal as follows - Kindly suggest

###################################

/tmp/cockroach node status --all --insecure --host=1.1.1.1 --port=26257 --format pretty; /tmp/cockroach sql --insecure --host=1.1.1.1 --port=26257 -d db-mvt -e “SELECT count(1) FROM TT_YY”; /tmp/cockroach sql --insecure --host=2.2.2.2 --port=26257 -d db-mvt -e “SELECT count(1) FROM TT_YY”; /tmp/cockroach sql --insecure --host=3.3.3.3 --port=26257 -d db-mvt -e “SELECT count(1) FROM TT_YY”;

±—±---------------------±-------±--------------------±--------------------±-----------------±----------------------±-------±-------------------±-----------------------±-----------±----------±------------±-------------±-------------±--------±------------------±-------------------±------------+
| id | address | build | updated_at | started_at | replicas_leaders | replicas_leaseholders | ranges | ranges_unavailable | ranges_underreplicated | live_bytes | key_bytes | value_bytes | intent_bytes | system_bytes | is_live | gossiped_replicas | is_decommissioning | is_draining |
±—±---------------------±-------±--------------------±--------------------±-----------------±----------------------±-------±-------------------±-----------------------±-----------±----------±------------±-------------±-------------±--------±------------------±-------------------±------------+
| 1 | 1.1.1.1:26257 | v1.1.5 | 2018-02-20 07:50:00 | 2018-02-20 07:02:10 | 31 | 31 | 33 | 0 | 0 | 2406718972 | 49845338 | 2376280281 | 0 | 126189 | true | 104 | false | false |
| 2 | 2.2.2.2:26257 | v1.1.5 | 2018-02-20 07:49:53 | 2018-02-12 07:45:07 | 41 | 36 | 41 | 0 | 0 | 4212638733 | 42604006 | 4189440465 | 69 | 126957 | true | 104 | false | false |
| 3 | 3.3.3.3:26257 | v1.1.5 | 2018-02-20 07:50:02 | 2018-02-20 07:10:11 | 30 | 29 | 30 | 0 | 0 | 2406719137 | 49845431 | 2376280458 | 34 | 126395 | true | 104 | false | false |
±—±---------------------±-------±--------------------±--------------------±-----------------±----------------------±-------±-------------------±-----------------------±-----------±----------±------------±-------------±-------------±--------±------------------±-------------------±------------+
(3 rows)

Cluster ID: x-x-x-x

±---------+
| count(1) |
±---------+
| 596166 |
±---------+
(1 row)

Cluster ID: x-x-x-x

±---------+
| count(1) |
±---------+
| 596175 |
±---------+
(1 row)

Cluster ID: x-x-x-x

±---------+
| count(1) |
±---------+
| 596178 |
±---------+
(1 row)

Cluster ID: x-x-x-x

±---------+
| count(1) |
±---------+
| 596186 |
±---------+
(1 row)

###################################

Kindly suggest on the understanding.

Thanks

Hi @stdranwl!

Were there writes being processed on this cluster as you ran this command?

While you did run all the queries in a single terminal command, the terminal
won’t run the next query until the previous one has finished. COUNT(*) can
take a while to run, and it’s possible that the state of the cluster changed
in the process of running each query (new rows were inserted, for example).
You might see this discrepancy lessen if you kick off the queries in separate
terminals at the same time (though it still might not be precisely the same).

If the cluster wasn’t processing any writes at the time you ran these queries,
there might be a more significant issue afoot.

While I don’t yet suspect that there’s a relationship with the ranges, replica
leaders, and leaseholders to what you’re seeing, if you’d like to learn more
about them we have some documentation about them on our
website
.
As far as understanding the replication guarantees provided by CockroachDB,
CRDB provides serializability, rather than linearizability (at least by default).
This means that while individual transactions will always see consistent results,
an outside observer running multiple transactions may not see results that
necessarily line up with real time.

Let me know if you have any more questions!

Justin