I’m new to CRDB.
I have configured CRDB to store logfiles and temporary data in directories outside of the one defined by the startup parameter “–store” and now almost everything looks neat.
Now in the “storage” directory referred to by the parameter “–store” I see some files (e.g. many “.sst"-files, one MANIFEST and OPTIONS files, etc…).
In addition to that I see some ".log”-files, for example:
# ls -lh *.log -rw-r----- 1 crdb crdb 91M Jul 19 20:08 019347.log -rw-r----- 1 crdb crdb 90M Jul 19 20:09 019353.log -rw-r----- 1 crdb crdb 89M Jul 19 20:09 019363.log -rw-r----- 1 crdb crdb 91M Jul 19 20:10 019367.log -rw-r----- 1 crdb crdb 82M Jul 19 20:10 019372.log
They seem to host binary data.
What are those “*.log”-files? Maybe some kind of WAL?
I saw that they keep being written to (a little bit) even when the DB is idle, and that they are written a lot when performing DML operations.
Can I store those files in a separate directory?
I’m using normal (slow) HDDs for the “storage” directory => as those files are written a lot during DMLs, but as they seem to disappear as well quite quickly, it would be perfect if I could write them onto my small SSD (therefore freeing up I/O for the final storage hosted on HDDs).
Maybe this thread is referring to those files?
In particular I’ve generally assumed that a JBOD such as this commonly has a relatively small SSD which could be utilized for the commit log and then the disks for the data. This is something I would emphatically like to see but my guess is we’re about a year from it. Some of the internals of that idea are discussed in #38322