Add the ability for operators of natc in consensus mode to remove
servers from the raft cluster config, without losing other state.
Updates #14667
Signed-off-by: Fran Bull <fran@tailscale.com>
Currently consensus has a bootstrap routine where a tsnet node tries to
join each other node with the cluster tag, and if it is not able to join
any other node it starts its own cluster.
That algorithm is racy, and can result in split brain (more than one
leader/cluster) if all the nodes for a cluster are started at the same
time.
Add a FollowOnly argument to the bootstrap function. If provided this
tsnet node will never lead, it will try (and retry with exponential back
off) to follow any node it can contact.
Add a --follow-only flag to cmd/natc that uses this new tsconsensus
functionality.
Also slightly reorganize some arguments into opts structs.
Updates #14667
Signed-off-by: Fran Bull <fran@tailscale.com>
We have been unintentionally ignoring errors from calling bootstrap.
bootstrap sometimes calls raft.BootstrapCluster which sometimes returns
a safe to ignore error, handle that case appropriately.
Updates #14667
Signed-off-by: Fran Bull <fran@tailscale.com>
tsconsensus enables tsnet.Server instances to form a consensus.
tsconsensus wraps hashicorp/raft with
* the ability to do discovery via tailscale tags
* inter node communication over tailscale
* routing of commands to the leader
Updates #14667
Signed-off-by: Fran Bull <fran@tailscale.com>
Fran Bull