Combining two nodes into one

hello - i have successfully installed cockroachdb on a google cloud instance following these instructions. this is my initial impression .

i have issued the following two commands as per the instructions:

cockroach start --background ;

cockroach start
–store=node2
–insecure
–port=26258
–http-host=10.138.0.2
–http-port=8081
–join=localhost:26257
–background;

i used –insecure so i could access the Admin UI since this is a google-cloud instance.

i have also created the little “bank” database with one simple table, and everything worked great.

my question: do i really need two nodes running now just to see my little bank database? is it now possible to start up cockroach with just one node? if i try something like this:

cockroach start
–store=cockroach-data
–insecure
–port=26257
–http-host=10.138.0.2
–http-port=8082
–background ;

i cannot access my database. i tried substituting both of the following:
cockroach sql ;
set database = bank;

i can only see my database again if i start the cockroach processes using the identical startup commands from my original startup.

i should think that the database would be wherever the store is pointing to, but when i just use either of the following:

  • store=cockroach-data
  • store=node2

i can no longer find my bank table, and cockroach node ls no longer works either. it appears i need to start up BOTH nodes identically to how i originally started them, or nothing works.

what must i do to “combine” the two original cockroach start command to just run one node?

This sounds like it’s operating as designed - Your cluster was of size 2, and you restarted it with a single node. As far of the database is concerned, half its cluster is missing! As such, it can’t be sure of getting consensus of greater than half the cluster for later requests. If instead of intentionally turning a node off, you actually encountered a network partition, and if the nodes continued to accept requests they would get into a split brain scenario, and consistency is out the window.

As an aside, this is why it’s generally recommended to run odd-sized clusters - the second node doesn’t actually get you anything if a node fails. This is also why the minimum recommended cockroach cluster size is 3 - you can then survive a single node failure without issue.

I’m not sure what the policy is for gradually decreasing cluster size however. I would assume that once given time to rebalance data, the cluster would be able to continue to have nodes pruned away slowly until you get to 2. However, at that point, removing nodes puts the cluster into an invalid state (unable to connect to greater than half the nodes), so I don’t think it’s possible to go down to a single node. I’d love to have a member of the dev team confirm that however :slight_smile:

thank you very much. i get the impression now its probably better just to begin with one process, at least until i really know what i am doing.

Hey,

thanks for giving us a try! What @twroebel says is correct: If you run your cluster with default settings and you add a second node, all the data will use two copies and you can’t lose one of the copies as that means losing the quorum (quorum for 2 nodes is 2).

The procedure to get such a cluster back down to single-node is via the zone config - you’d give the two nodes different attributes (say node1 and node2), and then decree (via zone config) that all of your data should only have one copy on node1. After a while, that should drain all your data off node2 and you could destroy it.

That said, that last part (waiting) is largely manual and there isn’t a really good way of knowing when it’s done, so until we improve that you may want to just keep your cluster single-node for simplicity if that’s all you need.

1 Like