Heavy business analytics query solution?

Currently, we have a PostgreSQL cluster with 1 master, 2 read-only slaves and 1 BI slave.
BI slave is just a slave with huge amount of memory and only for BI team.

With cockroach, what are the solutions for BI? (since we don’t want heavy query from BI team affect PROD performance)

Hi @nvcnvn,

What sorts of BI queries are you intending to run, on what DDL? CRDB is currently not well optimized for OLAP workloads. If you want the consistency and survivability of CRDB but need to summarize large sets of data, we typically recommend using CDC to sync changes to another DB optimized for OLAP workloads.

Hope that helps!

Hi, it CDC seem a great feature…
But I’m a free (as beer) user, do CRDB have any way… let say to read the Raft log?

Connect the dots for me - what are you considering with the raft logs?

Assuming the OLAP queries don’t need to be accurate in real time, using the core version you can also dump data from the database and load it into an analytic database asynchronously.

Forgive me if I’m wrong, I assumed CRDB raft log is somewhat similar to MySQL bin log. So I guess if we able to read (and understand) the raft log, we may build a tool to replicate data.

Our OLAP workloads not really require real time, but dump the full db and import them every hour seem too much,

https://www.cockroachlabs.com/docs/dev/changefeed-for is the non-enterprise version of CDC. It doesn’t have the same guarantees, but it does allow for streaming changes through a SQL client. That is probably your best option to solve your problem.

1 Like