Understanding Multi Raft Implementation

General
#1

Hi ,
I was building a small nosql db (for learning purpose) backed by raft consensus for replication . I was going to through various document and links how cockroachdb has optimized Heartbeat mechanism .

Here is my understanding from design docs :

  1. Each Node will represent single store.

  2. Each store consists of Regions (Range of keys)
    2.1 Leader Regions
    2.2 Replica Regions ( Replica or peers of regions which are leaders on other nodes/store)

  3. Each Region (Leader or Replica ) will be backed by single raft node . So for each store (no of raft node == no of regions )

  4. Since each raft node coordinate via messages sent through heartbeats , cockroachdb coalesces all heartbeats of single node /store and send to other store . Message will contain region id and node id which will multiplexed at receiver end and changes in state machines will happen .

Let me know if anything is misunderstood .

What I am not able to understand is how “coalesces all heartbeats” is happening at low level (code level). It would be great help if I get some details and code pointers .

Thanks
Sohi

#3

You’ve got the right idea. To find the code that implements this, search for coalesce in pkg/storage/{store,replica}.go.

#4

Thanks Ben. I looked at code pointed by you. I will go more in depth .

What I understand is, Cockroach db is using gRpc Server and client for sending / receiving coalesced
Heartbeats between nodes .
Just wanted to know if I am on right path .

Thanks again
Sohi

#5

Yes, GRPC is used for all inter-node communication.