How about cloning prod -> test

Can we add test-nodes to the prod-cluster that are to receive a copy of the full data set and separate them from the prod-cluster again to make a test cluster from the test-nodes?
This would be nice to be able to quickly create a test system from an existing prod environment.

Hi @ik_zelf, that is not possible. it looks like what you’re looking for is to backup the dataset (either using DUMP or enterprise BACKUP) and to load it into a new test cluster. That should achieve exactly what you’re looking for. You should be regularly backing up your production cluster as best practices any way, so restoring from the latest copy into your dev environment should be a single step.

That is a pity. I could imagine that something like adding nodes and somehow creating a splitbrain would make it possible.
For small databases dumps can do the job. For larger db’s it’s no fun at all.

@ik_zelf some of the subcomponents of the precise mechanism you’re describing (adding nodes that receive copies of the data but are not part of the main cluster) are on our roadmap, but others are not (promoting them to a fresh test cluster - with a “new brain” so to speak).

If your concern is having the freshest possible copy of a test cluster, we are planning to build Change Data Capture (CDC) in our 2.1 release (Oct 2018). This would enable a live production cockroach cluster to stream changes into the test cluster as they occurred. But you would still have to kick things off with a backup/restore to get the initial data. CDC would then cover all the changes that occurred since the backup/restore.

If your concern is speed, our enterprise backup/restore is a lot faster than DUMP, and takes full advantage of all the nodes in the cluster to parallelize the backup and restore, so it would be fun even at larger scales.

@arjun Change Data Capture sounds like a great feature. Would you be able to use CDC to fail a CRDB cluster on the east coast to a CRDB cluster on the west coast and vice versa?

That is our intention - to be able to tail a CRDB cluster into a backup, or into Kafka, or into another CRDB cluster. Just one heads up: we’re still in the design phase for CDC, so it’s not as concrete, although it is our current intention to ship it in the October 2018 timeframe.