I saw this panic while writing a new test for #14715:
panic: send on closed channel
goroutine 826 [running]:
tailscale.com/tsnet.(*listener).handle(0x1400031a500, {0x1035fbb00, 0x14000b82300})
/Users/bradfitz/src/tailscale.com/tsnet/tsnet.go:1317 +0xac
tailscale.com/wgengine/netstack.(*Impl).acceptTCP(0x14000204700, 0x14000882100)
/Users/bradfitz/src/tailscale.com/wgengine/netstack/netstack.go:1320 +0x6dc
created by gvisor.dev/gvisor/pkg/tcpip/transport/tcp.(*Forwarder).HandlePacket in goroutine 807
/Users/bradfitz/go/pkg/mod/gvisor.dev/gvisor@v0.0.0-20240722211153-64c016c92987/pkg/tcpip/transport/tcp/forwarder.go:98 +0x32c
FAIL tailscale.com/tsnet 0.927s
Updates #14715
Change-Id: I9924e0a6c2b801d46ee44eb8eeea0da2f9ea17c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
cmd/k8s-operator: add logic to parse L7 Ingresses in HA mode
- Wrap the Tailscale API client used by the Kubernetes Operator
into a client that knows how to manage VIPServices.
- Create/Delete VIPServices and update serve config for L7 Ingresses
for ProxyGroup.
- Ensure that ingress ProxyGroup proxies mount serve config from a shared ConfigMap.
Updates tailscale/corp#24795
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Adds a new Hostinfo.IngressEnabled bool field that holds whether
funnel is currently enabled for the node. Triggers control update
when this value changes.
Bumps capver so that control can distinguish the new field being false
vs non-existant in previous clients.
This is part of a fix for an issue where nodes with any AllowFunnel
block set in their serve config are being displayed as if actively
routing funnel traffic in the admin panel.
Updates tailscale/tailscale#11572
Updates tailscale/corp#25931
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
We throw error early with a warning if users attempt to enable background funnel
for a node that does not allow incoming connections
(shields up), but if it done in foreground mode, we just silently fail
(the funnel command succeeds, but the connections are not allowed).
This change makes sure that we also error early in foreground mode.
Updates tailscale/tailscale#11049
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Updates tailscale/corp#25936
This defines a new syspolicy 'Hostname' and allows an IT administrator to override the value we normally read from os.Hostname(). This is particularly useful on Android and iOS devices, where the hostname we get from the OS is really just the device model (a platform restriction to prevent fingerprinting).
If we don't implement this, all devices on the customer's side will look like `google-pixel-7a-1`, `google-pixel-7a-2`, `google-pixel-7a-3`, etc. and it is not feasible for the customer to use the API or worse the admin console to manually fix these names.
Apply code review comment by @nickkhyl
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
Co-authored-by: Nick Khyl <1761190+nickkhyl@users.noreply.github.com>
Since 5297bd2cff, tstun.Wrapper has required its Start
method to be called for it to function. Failure to do so just
results in weird hangs and I've wasted too much time multiple
times now debugging. Hopefully this prevents more lost time.
Updates tailscale/corp#24454
Change-Id: I87f4539f7be7dc154627f8835a37a8db88c31be0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Metrics currently exist for dropped packets by reason, and total
received packets by kind (e.g., `disco` or `other`), but relating these
two together to gleam information about the drop rate for specific
reasons on a per-kind basis is not currently possible.
Change `derp_packets_dropped` to use a `metrics.MultiLabelMap` to
track both the `reason` and `kind` in the same metric to allow for this
desired level of granularity.
Drop metrics that this makes unnecessary (namely `packetsDroppedReason`
and `packetsDroppedType`).
Updates https://github.com/tailscale/corp/issues/25489
Signed-off-by: Mario Minardi <mario@tailscale.com>
Most users should not run into this because it's set in the helm chart
and the deploy manifest, but if namespace is not set we get confusing
authz errors because the kube client tries to fetch some namespaced resources
as though they're cluster-scoped and reports permission denied. Try to
detect namespace from the default projected volume, and otherwise fatal.
Fixes #cleanup
Change-Id: I64b34191e440b61204b9ad30bbfa117abbbe09c3
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
If the server was in use at the time of the initial check, but disconnected and was removed
from the activeReqs map by the time we registered a waiter, the ready channel will never
be closed, resulting in a deadlock. To avoid this, we check whether the server is still busy
after registering the wait.
Fixes#14655
Signed-off-by: Nick Khyl <nickk@tailscale.com>
I made a last-minute change in #14626 to split a single loop that created 1_000 concurrent
connections into an inner and outer loop that create 100 concurrent connections 10 times.
This introduced a race because the last user's connection may still be active (from the server's
perspective) when a new outer iteration begins. Since every new client gets a unique ClientID,
but we reuse usernames and UIDs, the server may let a user in (as the UID matches, which is fine),
but the test might then fail due to a ClientID mismatch:
server_test.go:232: CurrentUser(Initial): got &{S-1-5-21-1-0-0-1001 User-4 <nil> Client-2 false false};
want &{S-1-5-21-1-0-0-1001 User-4 <nil> Client-114 false false}
In this PR, we update (*testIPNServer).blockWhileInUse to check whether the server is currently busy
and wait until it frees up. We then call blockWhileInUse at the end of each outer iteration so that the server
is always in a known idle state at the beginning of the inner loop. We also check that the current user
is not set when the server is idle.
Updates tailscale/corp#25804
Updates #14655 (found when working on it)
Signed-off-by: Nick Khyl <nickk@tailscale.com>
As we look to add github.com/prometheus/client_golang/prometheus to
more parts of the codebase, lock in that we don't use it in tailscaled,
primarily for binary size reasons.
Updates #12614
Change-Id: I03c100d12a05019a22bdc23ce5c4df63d5a03ec6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This test verifies, among other things, that init functions cannot be deferred after (*DeferredFuncs).Do
has already been called and that all subsequent calls to (*DeferredFuncs).Defer return false.
However, the initial implementation of this check was racy: by the time (*DeferredFuncs).Do returned,
not all goroutines that successfully deferred an init function may have incremented the atomic variable
tracking the number of deferred functions. As a result, the variable's value could differ immediately
after (*DeferredFuncs).Do returned and after all goroutines had completed execution (i.e., after wg.Wait()).
In this PR, we replace the original racy check with a different one. Although this new check is also racy,
it can only produce false negatives. This means that if the test fails, it indicates an actual bug rather than
a flaky test.
Fixes#14039
Signed-off-by: Nick Khyl <nickk@tailscale.com>
There's at least one example of stored routes and advertised routes
getting out of sync. I don't know how they got there yet, but this would
backfill missing advertised routes on startup from stored routes.
Also add logging in LocalBackend.AdvertiseRoute to record when new
routes actually get put into prefs.
Updates #14606
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
I moved the actual rename into separate, GOOS-specific files. On
non-Windows, we do a simple os.Rename. On Windows, we first try
ReplaceFile with a fallback to os.Rename if the target file does
not exist.
ReplaceFile is the recommended way to rename the file in this use case,
as it preserves attributes and ACLs set on the target file.
Updates #14428
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
The --mesh-with flag now supports the specification of hostname tuples like
derp1a.tailscale.com/derp1a-vpc.tailscale.com, which instructs derp to mesh
with host 'derp1a.tailscale.com' but dial TCP connections to 'derp1a-vpc.tailscale.com'.
For backwards compatibility, --mesh-with still supports individual hostnames.
The logic which attempts to auto-discover '[host]-vpc.tailscale.com' dial hosts
has been removed.
Updates tailscale/corp#25653
Signed-off-by: Percy Wegmann <percy@tailscale.com>
We have observed some clients with extremely large lists of IPv6
endpoints, in some cases from subnets where the machine also has the
zero address for a whole /48 with then arbitrary addresses additionally
assigned within that /48. It is in general unnecessary for reachability
to report all of these addresses, typically only one will be necessary
for reachability. We report two, to cover some other common cases such
as some styles of IPv6 private address rotations.
Updates tailscale/corp#25850
Signed-off-by: James Tucker <james@tailscale.com>
In this commit, we add a failing test to verify that ipn/ipnserver.Server correctly
sets and unsets the current user when two different clients send requests concurrently
(A sends request, B sends request, A's request completes, B's request completes).
The expectation is that the user who wins the race becomes the current user
from the LocalBackend's perspective, remaining in this state until they disconnect,
after which a different user should be able to connect and use the LocalBackend.
We then fix the second of two bugs in (*Server).addActiveHTTPRequest, where a race
condition causes the LocalBackend's state to be reset after a new client connects,
instead of after the last active request of the previous client completes and the server
becomes idle.
Fixestailscale/corp#25804
Signed-off-by: Nick Khyl <nickk@tailscale.com>
In this commit, we add a failing test to verify that ipn/ipnserver.Server correctly
sets and unsets the current user when two different users connect sequentially
(A connects, A disconnects, B connects, B disconnects).
We then fix the test by updating (*ipn/ipnserver.Server).addActiveHTTPRequest
to avoid calling (*LocalBackend).ResetForClientDisconnect again after a new user
has connected and been set as the current user with (*LocalBackend).SetCurrentUser().
Since ipn/ipnserver.Server does not allow simultaneous connections from different
Windows users and relies on the LocalBackend's current user, and since we already
reset the LocalBackend's state by calling ResetForClientDisconnect when the last
active request completes (indicating the server is idle and can accept connections
from any Windows user), it is unnecessary to track the last connected user on the
ipnserver.Server side or call ResetForClientDisconnect again when the user changes.
Additionally, the second call to ResetForClientDisconnect occurs after the new user
has been set as the current user, resetting the correct state for the new user
instead of the old state of the now-disconnected user, causing issues.
Updates tailscale/corp#25804
Signed-off-by: Nick Khyl <nickk@tailscale.com>
We update client/tailscale.LocalClient to allow specifying an optional Transport
(http.RoundTripper) for LocalAPI HTTP requests, and implement one that injects
an ipnauth.TestActor via request headers. We also add several functions and types
to make testing an ipn/ipnserver.Server possible (or at least easier).
We then use these updates to write basic tests for ipnserver.Server,
ensuring it works on non-Windows platforms and correctly sets and unsets
the LocalBackend's current user when a Windows user connects and disconnects.
We intentionally omit tests for switching between different OS users
and will add them in follow-up commits.
Updates tailscale/corp#25804
Signed-off-by: Nick Khyl <nickk@tailscale.com>
In preparation for adding test coverage for ipn/ipnserver.Server, we update it
to use ipnauth.Actor instead of its concrete implementation where possible.
Updates tailscale/corp#25804
Signed-off-by: Nick Khyl <nickk@tailscale.com>
We build up maps of both the existing MagicDNS configuration in hosts
and the desired MagicDNS configuration, compare the two, and only
write out a new one if there are changes. The comparison doesn't need
to be perfect, as the occasional false-positive is fine, but this
should greatly reduce rewrites of the hosts file.
I also changed the hosts updating code to remove the CRLF/LF conversion
stuff, and use Fprintf instead of Frintln to let us write those inline.
Updates #14428
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
This deprecates the old "DERP string" packing a DERP region ID into an
IP:port of 127.3.3.40:$REGION_ID and just uses an integer, like
PeerChange.DERPRegion does.
We still support servers sending the old form; they're converted to
the new form internally right when they're read off the network.
Updates #14636
Change-Id: I9427ec071f02a2c6d75ccb0fcbf0ecff9f19f26f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This doesn't seem to have any immediate impact, but not allowing access via the IPv6 masquerade
address when an IPv4 masquerade address is also set seems like a bug.
Updates #cleanup
Updates #14570 (found when working on it)
Signed-off-by: Nick Khyl <nickk@tailscale.com>
This finishes the work started in #14616.
Updates #8632
Change-Id: I4dc07d45b1e00c3db32217c03b21b8b1ec19e782
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In this PR, we add a generic views.ValuePointer type that can be used as a view for pointers
to basic types and struct types that do not require deep cloning and do not have corresponding
view types. Its Get/GetOk methods return stack-allocated shallow copies of the underlying value.
We then update the cmd/viewer codegen to produce getters that return either concrete views
when available or ValuePointer views when not, for pointer fields in generated view types.
This allows us to avoid unnecessary allocations compared to returning pointers to newly
allocated shallow copies.
Updates #14570
Signed-off-by: Nick Khyl <nickk@tailscale.com>
This amends commit b7e48058c8.
That commit broke all documented ways of starting Tailscale on gokrazy:
https://gokrazy.org/packages/tailscale/ — both Option A (tailscale up)
and Option B (tailscale up --auth-key) rely on the tailscale CLI working.
I verified that the tailscale CLI just prints it help when started
without arguments, i.e. it does not stay running and is not restarted.
I verified that the tailscale CLI successfully exits when started with
tailscale up --auth-key, regardless of whether the node has joined
the tailnet yet or not.
I verified that the tailscale CLI successfully waits and exits when
started with tailscale up, as expected.
fixes https://github.com/gokrazy/gokrazy/issues/286
Signed-off-by: Michael Stapelberg <michael@stapelberg.de>
This will enable Prometheus queries to look at the bandwidth over time windows,
for example 'increase(derp_bw_bytes_total)[1h] / increase(derp_bw_transfer_time_seconds_total)[1h]'.
Fixes commit a51672cafd.
Updates tailscale/corp#25503
Signed-off-by: Percy Wegmann <percy@tailscale.com>
We still use josharian/native (hi @josharian!) via
netlink, but I also sent https://github.com/mdlayher/netlink/pull/220
Updates #8632
Change-Id: I2eedcb7facb36ec894aee7f152c8a1f56d7fc8ba
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
sync.OnceValue and slices.Compact were both added in Go 1.21.
cmp.Or was added in Go 1.22.
Updates #8632
Updates #11058
Change-Id: I89ba4c404f40188e1f8a9566c8aaa049be377754
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Bump the versions to pick up some CVE patches. They don't affect us, but
customer scanners will complain.
Updates #cleanup
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
This commit updates the return body of c2n endpoint /vip-services to keep hash generation logic on client side.
Updates tailscale/corp#24510
Signed-off-by: KevinLiang10 <37811973+KevinLiang10@users.noreply.github.com>
Brad Fitzpatrick
dependabot[bot]
Irbe Krumina
Mike O'Driscoll
Andrea Gottardo
Mario Minardi
Aaron Klotz
Tom Proctor
Jordan Whited
Nick Khyl
Andrew Lytvynov
Percy Wegmann
James Tucker
Anton Tolchanov
Michael Stapelberg
KevinLiang10