RocksDB prefix compression

Hi,

On https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mapping-table-data-to-key-value-storage/ you write:

You might be thinking that the common prefixes (/1000/10) for the keys are wasting storage, but RocksDB our underlying storage engine, eliminates almost all of the overhead via prefix compression of keys.

But I couldn’t find an in-depth description of this feature in RocksDB (searched for prefix compression and some similar queries). Could you please provide a pointer, or a better term to search for?

Hi @bra! Thank you for your interest in CockroachDB and RocksDB.

The prefix compression for keys is documented here:

"

// When we store a key, we drop the prefix shared with the previous
// string.  This helps reduce the space requirement significantly.
// Furthermore, once every K keys, we do not apply the prefix
// compression and store the entire key.  We call this a "restart
// point".  The tail end of the block stores the offsets of all of the
// restart points, and can be used to do a binary search when looking
// for a particular key.  

"

Does this answer your question?

Yes, thanks a lot for the link!