64kb/1mb Bytes limit on performance

Hey all,

While reading the docs, I found recommendations that String and Bytes objects be under 64kb and 1mb respectively, otherwise there is performance degradation.

In my use case, I have a couple (on order 2-5) very large string or bytes objects (they’re just DNA sequence, so ATGC) that are about 4mb to 10mb in size. In that table, all other rows are sized in the ~3kb range.

Will having a few large string objects decrease performance by a large degree? Or would that only really happen if I had many rows with lots of data?

Thanks

Hey @koeng

In CockroachDB, the database sits on top of a key value store comprised of ranges. Each range can grow to a maximum of 64 MB by default (docs page here describes this). Due to this, there is a maximum row size limit that is in place (docs page here). There is also MVCC history for each key-value pair that needs to stay with the pair, and what may occur, if these large rows are updated often, is that at some point the range may attempt to split and will not be able to, as splitting it may separate the history from the pair(details are here. One thing that you can try to help against this is creating a column family, which is described in the documentation here, so that the column will all be a part of the same KV pair, which may help.

Please review all the information, and let me know if there are any other questions.

Hi @rickrock,

I read the documentation, and just want to make sure I’m reading this correctly:

  • After columns are 64mb in size, they get split into ranges
  • If you exceed that size AND if you update during a small period of time, errors can occur

However, I won’t be updating the information in those large rows nearly ever (it’s pretty static), and thus, I won’t have to worry about it.

Hey @koeng

That is mostly correct. I would add that typically if you exceed the size, update in that time frame, and hit one of those limitations on sizing, then it could error out.

Since you aren’t updating very often then, reads should occur without too much trouble. Let me know if there are any other questions.

2 Likes