Hi @tim-o - Please find the answers below.
How are you balancing load between nodes?
I am not doing anything explicitly for load balancing. In my Go Microservice, I am using GORM which opens “postgresql” connection.
The connection string is postgresql://firstname.lastname@example.org:26257/mydb?sslmode=disable . Where the url is Kubernetes Service name.
What does the output of the sql connections graph on the sql dash look like?
I believe there is some imbalance in serving the query.
I ran a test, this time n1 shows very latency where as n2 has very high latency. Though there are connections to all 3 nodes.
What’s the schema for conversion_currency_rates?
How many rows / how much data is contained in the table?
The table has only one row
Where are the nodes located?
The CRDB Nodes running Kubernetes across 3 availability zones.
AWS Region - ap-southeast-1
n1 AZ - ap-southeast-1c
n2 AZ - ap-southeast-1a
n3 AZ - ap-southeast-1b
What is the latency between them?
admin@ip-10-0-86-66:~$ ping ip-10-0-68-213.ap-southeast-1.compute.internal
64 bytes from ip-10-0-68-213.ap-southeast-1.compute.internal (10.0.68.213): icmp_seq=18 ttl=64 time=0.562 ms
— ip-10-0-68-213.ap-southeast-1.compute.internal ping statistics —
18 packets transmitted, 18 received, 0% packet loss, time 17006ms
rtt min/avg/max/mdev = 0.540/0.593/0.656/0.036 ms
admin@ip-10-0-86-66:~$ ping ip-10-0-56-60.ap-southeast-1.compute.internal
15 packets transmitted, 15 received, 0% packet loss, time 14015ms
rtt min/avg/max/mdev = 1.236/1.335/1.425/0.060 ms
admin@ip-10-0-68-213:~$ ping ip-10-0-56-60.ap-southeast-1.compute.internal
— ip-10-0-56-60.ap-southeast-1.compute.internal ping statistics —
18 packets transmitted, 18 received, 0% packet loss, time 17017ms
rtt min/avg/max/mdev = 1.229/1.392/3.095/0.415 ms
Did you detect any 180ms response times from the db in your application?
I didn’t captured that in log. I will add that and update. But I can correlate the query latency with my API response time. Whenever there is a spike in CRDB Service latency, my average response increases drastically.
Were there any other queries on the statements page?
No, only one query in statements page