I’m migrating our telemetry database from RethinkDB to CockroachDB, and am having some performance problems.
Our telemetry data is stored as JSON payloads which do not adhere to a rigid structure. Rethink was perfect for this, because we could create indexes on specific fields within the JSON document, rather than the whole thing. Some JSON documents may be up to 10 MB large, although most are less than 500 KB.
I’m finding that writes larger than about 500 KB are taking minutes, even though only a single column is written – the JSON document – for up to 100 or so rows.
By removing the inverted index on that field, however, writes accelerated to < 50ms.
I only need to index a few fields within the large JSON structure – for queries on the rest, which are very very rare, I can accept a table-wide scan. Is creating such an index possible with Cockroach?