Does IMPORT CSV support compressed files?


(Travis) #1

Just wondering if the IMPORT statement supports compressed files, e.g. table.csv.gz
I have some fairly large tables (~20 GB) to import and it would make it easier if they could be compressed.

Also is there a way to specify a different column order? It seems like the CSV has to be in the same order as the create table statement.


(david) #2

IMPORT does not currently support compressed files.

That said, some of the external storage implementations might offer on-the-fly decompression e.g. the http external storage provider uses Go’s HTTP client, and it automatically handles gzip’ed http responses, and some of the cloud providers might offer similar?

We’re currently collecting feedback on our initial version of IMPORT, to guide where to focus resources for additional improvements, so thanks for the suggestion! I’ve filed it here: https://github.com/cockroachdb/cockroach/issues/25149

As far as column remapping, that is also not yet supported, but there’s some discussion of adding it on https://github.com/cockroachdb/cockroach/issues/24426