Memory Usage(Memory management)

Hi All,

I have question around development of memory management in CockroachDB. I was going though a blog Memory Usage in CockroachDB.

  1. what package did cockraoch db use to track memory allocation in the code or did cockroach db created their own memory allocation framework to track allocation??

@sushant1988 You could say we created our own, but it’s really not much
code. The hooks into the various parts of the system are more extensive
than the library. You can see an overview of how it works at


and the monitor library itself at

So Are you tracking only In-Memory row storage or everything?? Although In-Memory row storage matters since that is the only variable part. The rest can be assumed as constant value.
Right??

Correct, we only track the parts we expect to be variable, which generally
is the data that is temporarily batched at various places in our sql
evaluation.

Hi,

I was going through the documentation provided to me above. Come up with bunch of questions related to memory usage on CockroachDB.

  1. What is the time complexity for tracking memory usage.
  2. How do you decide which session query to be killed?? Is there any priority set for session based query or you kill on first come first basis :- who so ever asked for more memory and there is no memory left then you that return with error and kill that query ??

Thanks for the follow-up questions, @Sushant198. @knz wrote the memory usage blog post, so I’m hoping he might be able to help further.

It’s a constant factor on every row value being manipulated, so a constant factor on the time complexity of entire queries.

We do not kill session nor queries. IT doesn’t work like the linux kernel (which kills processes that exhaust memory). Instead, CockroachDB will cause a large query to fail with a SQL error (and the SQL transaction will abort). The client can continue issuing more statements/transactions in the same transaction.

We also have a blog post that explains the mechanisms, the costs and how to reason about memory:

Can you check this out and see if it answers your questions?

Yes I have already gone through this blog. It was pretty helpful.