cmd/k8s-operator,k8s-operator: Allow the use of multiple tailnets (#18344)

This commit contains  the implementation of multi-tailnet support within the Kubernetes Operator

Each of our custom resources now expose the `spec.tailnet` field. This field is a string that must match the name of an existing `Tailnet` resource. A `Tailnet` resource looks like this:

```yaml
apiVersion: tailscale.com/v1alpha1
kind: Tailnet
metadata:
  name: example  # This is the name that must be referenced by other resources
spec:
  credentials:
    secretName: example-oauth
```

Each `Tailnet` references a `Secret` resource that contains a set of oauth credentials. This secret must be created in the same namespace as the operator:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: example-oauth # This is the name that's referenced by the Tailnet resource.
  namespace: tailscale
stringData:
  client_id: "client-id"
  client_secret: "client-secret"
```

When created, the operator performs a basic check that the oauth client has access to all required scopes. This is done using read actions on devices, keys & services. While this doesn't capture a missing "write" permission, it catches completely missing permissions. Once this check passes, the `Tailnet` moves into a ready state and can be referenced. Attempting to use a `Tailnet` in a non-ready state will stall the deployment of `Connector`s, `ProxyGroup`s and `Recorder`s until the `Tailnet` becomes ready.

The `spec.tailnet` field informs the operator that a `Connector`, `ProxyGroup`, or `Recorder` must be given an auth key generated using the specified oauth client. For backwards compatibility, the set of credentials the operator is configured with are considered the default. That is, where `spec.tailnet` is not set, the resource will be deployed in the same tailnet as the operator. 

Updates https://github.com/tailscale/corp/issues/34561

cmd/k8s-operator/depaware.txt

@@ -725,7 +725,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
         k8s.io/utils/net                                             from k8s.io/apimachinery/pkg/util/net+
         k8s.io/utils/ptr                                             from k8s.io/client-go/tools/cache+
         k8s.io/utils/trace                                           from k8s.io/client-go/tools/cache
-        sigs.k8s.io/controller-runtime/pkg/builder                   from tailscale.com/cmd/k8s-operator
+        sigs.k8s.io/controller-runtime/pkg/builder                   from tailscale.com/cmd/k8s-operator+
         sigs.k8s.io/controller-runtime/pkg/cache                     from sigs.k8s.io/controller-runtime/pkg/cluster+
         sigs.k8s.io/controller-runtime/pkg/cache/internal            from sigs.k8s.io/controller-runtime/pkg/cache
         sigs.k8s.io/controller-runtime/pkg/certwatcher               from sigs.k8s.io/controller-runtime/pkg/metrics/server+
@@ -821,10 +821,12 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
         tailscale.com/ipn/store                                      from tailscale.com/ipn/ipnlocal+
         tailscale.com/ipn/store/kubestore                            from tailscale.com/cmd/k8s-operator
         tailscale.com/ipn/store/mem                                  from tailscale.com/ipn/ipnlocal+
-        tailscale.com/k8s-operator                                   from tailscale.com/cmd/k8s-operator
+        tailscale.com/k8s-operator                                   from tailscale.com/cmd/k8s-operator+
         tailscale.com/k8s-operator/api-proxy                         from tailscale.com/cmd/k8s-operator
         tailscale.com/k8s-operator/apis                              from tailscale.com/k8s-operator/apis/v1alpha1
         tailscale.com/k8s-operator/apis/v1alpha1                     from tailscale.com/cmd/k8s-operator+
+        tailscale.com/k8s-operator/reconciler                        from tailscale.com/k8s-operator/reconciler/tailnet
+        tailscale.com/k8s-operator/reconciler/tailnet                from tailscale.com/cmd/k8s-operator
         tailscale.com/k8s-operator/sessionrecording                  from tailscale.com/k8s-operator/api-proxy
         tailscale.com/k8s-operator/sessionrecording/spdy             from tailscale.com/k8s-operator/sessionrecording
         tailscale.com/k8s-operator/sessionrecording/tsrecorder       from tailscale.com/k8s-operator/sessionrecording+