In v1.78, we started acquiring the GP lock when reading policy settings. This led to a deadlock during
Tailscale installation via Group Policy Software Installation because the GP engine holds the write lock
for the duration of policy processing, which in turn waits for the installation to complete, which in turn
waits for the service to enter the running state.
In this PR, we prevent the acquisition of GP locks (aka EnterCriticalPolicySection) during service startup
and update the Windows Registry-based util/syspolicy/source.PlatformPolicyStore to handle this failure
gracefully. The GP lock is somewhat optional; it’s safe to read policy settings without it, but acquiring
the lock is recommended when reading multiple values to prevent the Group Policy engine from modifying
settings mid-read and to avoid inconsistent results.
Fixes#14416
Signed-off-by: Nick Khyl <nickk@tailscale.com>
This was a slow memory leak on busy tailnets with lots of tagged
ephemeral nodes.
Updates tailscale/corp#26058
Change-Id: I298e7d438e3ffbb3cde795640e344671d244c632
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Still behind the same ts_omit_tap build tag.
See #14738 for background on the pattern.
Updates #12614
Change-Id: I03fb3d2bf137111e727415bd8e713d8568156ecc
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
If we fail to parse the upstream DNS response in an app connector, we
might miss new IPs for the target domain. Log parsing errors to be able
to diagnose that.
Updates #14606
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
Remove "unexpected" labelling of PeerGoneReasonNotHere.
A peer being no longer connected to a DERP server
is not an unexpected case and causes confusion in looking at logs.
Fixestailscale/corp#25609
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
The new ProxyGroup-based Ingress reconciler is causing a fatal log at
startup because it has the same name as the existing Ingress reconciler.
Explicitly name both to ensure they have unique names that are consistent
with other explicitly named reconcilers.
Updates #14583
Change-Id: Ie76e3eaf3a96b1cec3d3615ea254a847447372ea
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
This pulls out the Wake-on-LAN (WoL) code out into its own package
(feature/wakeonlan) that registers itself with various new hooks
around tailscaled.
Then a new build tag (ts_omit_wakeonlan) causes the package to not
even be linked in the binary.
Ohter new packages include:
* feature: to just record which features are loaded. Future:
dependencies between features.
* feature/condregister: the package with all the build tags
that tailscaled, tsnet, and the Tailscale Xcode project
extension can empty (underscore) import to load features
as a function of the defined build tags.
Future commits will move of our "ts_omit_foo" build tags into this
style.
Updates #12614
Change-Id: I9c5378dafb1113b62b816aabef02714db3fc9c4a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
* Reapply "ipn/ipnlocal: re-advertise appc routes on startup (#14609)"
This reverts commit 51adaec35a.
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
* ipn/ipnlocal: fix a deadlock in readvertiseAppConnectorRoutes
Don't hold LocalBackend.mu while calling the methods of
appc.AppConnector. Those methods could call back into LocalBackend and
try to acquire it's mutex.
Fixes https://github.com/tailscale/corp/issues/25965Fixes#14606
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
---------
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
Updates tailscale/corp#25278
Adds definitions for new CLI commands getting added in v1.80. Refactors some pre-existing CLI commands within the `configure` tree to clean up code.
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
Rather than using a string everywhere and needing to clarify that the
string should have the svc: prefix, create a separate type for Service
names.
Updates tailscale/corp#24607
Change-Id: I720e022f61a7221644bb60955b72cacf42f59960
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
This commit intend to provide support for TCP and Web VIP services and also allow user to use Tun
for VIP services if they want to.
The commit includes:
1.Setting TCP intercept function for VIP Services.
2.Update netstack to send packet written from WG to netStack handler for VIP service.
3.Return correct TCP hander for VIP services when netstack acceptTCP.
This commit also includes unit tests for if the local backend setServeConfig would set correct TCP intercept
function and test if a hander gets returned when getting TCPHandlerForDst. The shouldProcessInbound
check is not unit tested since the test result just depends on mocked functions. There should be an integration
test to cover shouldProcessInbound and if the returned TCP handler actually does what the serveConfig says.
Updates tailscale/corp#24604
Signed-off-by: KevinLiang10 <37811973+KevinLiang10@users.noreply.github.com>
We previously baked in the LetsEncrypt x509 root CA for our tlsdial
package.
This moves that out into a new "bakedroots" package and is now also
shared by ipn/ipnlocal's cert validation code (validCertPEM) that
decides whether it's time to fetch a new cert.
Otherwise, a machine without LetsEncrypt roots locally in its system
roots is unable to use tailscale cert/serve and fetch certs.
Fixes#14690
Change-Id: Ic88b3bdaabe25d56b9ff07ada56a27e3f11d7159
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Both @agottardo and I tripped over this today.
Updates #cleanup
Change-Id: I64380a03bfc952b9887b1512dbcadf26499ff1cd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
If unable to accept a connection from the bandwidth probe listener,
return from the goroutine immediately since the accepted connection
will be nil.
Updates tailscale/corp#25958
Signed-off-by: Percy Wegmann <percy@tailscale.com>
I saw this panic while writing a new test for #14715:
panic: send on closed channel
goroutine 826 [running]:
tailscale.com/tsnet.(*listener).handle(0x1400031a500, {0x1035fbb00, 0x14000b82300})
/Users/bradfitz/src/tailscale.com/tsnet/tsnet.go:1317 +0xac
tailscale.com/wgengine/netstack.(*Impl).acceptTCP(0x14000204700, 0x14000882100)
/Users/bradfitz/src/tailscale.com/wgengine/netstack/netstack.go:1320 +0x6dc
created by gvisor.dev/gvisor/pkg/tcpip/transport/tcp.(*Forwarder).HandlePacket in goroutine 807
/Users/bradfitz/go/pkg/mod/gvisor.dev/gvisor@v0.0.0-20240722211153-64c016c92987/pkg/tcpip/transport/tcp/forwarder.go:98 +0x32c
FAIL tailscale.com/tsnet 0.927s
Updates #14715
Change-Id: I9924e0a6c2b801d46ee44eb8eeea0da2f9ea17c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
cmd/k8s-operator: add logic to parse L7 Ingresses in HA mode
- Wrap the Tailscale API client used by the Kubernetes Operator
into a client that knows how to manage VIPServices.
- Create/Delete VIPServices and update serve config for L7 Ingresses
for ProxyGroup.
- Ensure that ingress ProxyGroup proxies mount serve config from a shared ConfigMap.
Updates tailscale/corp#24795
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Adds a new Hostinfo.IngressEnabled bool field that holds whether
funnel is currently enabled for the node. Triggers control update
when this value changes.
Bumps capver so that control can distinguish the new field being false
vs non-existant in previous clients.
This is part of a fix for an issue where nodes with any AllowFunnel
block set in their serve config are being displayed as if actively
routing funnel traffic in the admin panel.
Updates tailscale/tailscale#11572
Updates tailscale/corp#25931
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
We throw error early with a warning if users attempt to enable background funnel
for a node that does not allow incoming connections
(shields up), but if it done in foreground mode, we just silently fail
(the funnel command succeeds, but the connections are not allowed).
This change makes sure that we also error early in foreground mode.
Updates tailscale/tailscale#11049
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Updates tailscale/corp#25936
This defines a new syspolicy 'Hostname' and allows an IT administrator to override the value we normally read from os.Hostname(). This is particularly useful on Android and iOS devices, where the hostname we get from the OS is really just the device model (a platform restriction to prevent fingerprinting).
If we don't implement this, all devices on the customer's side will look like `google-pixel-7a-1`, `google-pixel-7a-2`, `google-pixel-7a-3`, etc. and it is not feasible for the customer to use the API or worse the admin console to manually fix these names.
Apply code review comment by @nickkhyl
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
Co-authored-by: Nick Khyl <1761190+nickkhyl@users.noreply.github.com>
Since 5297bd2cff, tstun.Wrapper has required its Start
method to be called for it to function. Failure to do so just
results in weird hangs and I've wasted too much time multiple
times now debugging. Hopefully this prevents more lost time.
Updates tailscale/corp#24454
Change-Id: I87f4539f7be7dc154627f8835a37a8db88c31be0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Metrics currently exist for dropped packets by reason, and total
received packets by kind (e.g., `disco` or `other`), but relating these
two together to gleam information about the drop rate for specific
reasons on a per-kind basis is not currently possible.
Change `derp_packets_dropped` to use a `metrics.MultiLabelMap` to
track both the `reason` and `kind` in the same metric to allow for this
desired level of granularity.
Drop metrics that this makes unnecessary (namely `packetsDroppedReason`
and `packetsDroppedType`).
Updates https://github.com/tailscale/corp/issues/25489
Signed-off-by: Mario Minardi <mario@tailscale.com>
Most users should not run into this because it's set in the helm chart
and the deploy manifest, but if namespace is not set we get confusing
authz errors because the kube client tries to fetch some namespaced resources
as though they're cluster-scoped and reports permission denied. Try to
detect namespace from the default projected volume, and otherwise fatal.
Fixes #cleanup
Change-Id: I64b34191e440b61204b9ad30bbfa117abbbe09c3
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
If the server was in use at the time of the initial check, but disconnected and was removed
from the activeReqs map by the time we registered a waiter, the ready channel will never
be closed, resulting in a deadlock. To avoid this, we check whether the server is still busy
after registering the wait.
Fixes#14655
Signed-off-by: Nick Khyl <nickk@tailscale.com>
I made a last-minute change in #14626 to split a single loop that created 1_000 concurrent
connections into an inner and outer loop that create 100 concurrent connections 10 times.
This introduced a race because the last user's connection may still be active (from the server's
perspective) when a new outer iteration begins. Since every new client gets a unique ClientID,
but we reuse usernames and UIDs, the server may let a user in (as the UID matches, which is fine),
but the test might then fail due to a ClientID mismatch:
server_test.go:232: CurrentUser(Initial): got &{S-1-5-21-1-0-0-1001 User-4 <nil> Client-2 false false};
want &{S-1-5-21-1-0-0-1001 User-4 <nil> Client-114 false false}
In this PR, we update (*testIPNServer).blockWhileInUse to check whether the server is currently busy
and wait until it frees up. We then call blockWhileInUse at the end of each outer iteration so that the server
is always in a known idle state at the beginning of the inner loop. We also check that the current user
is not set when the server is idle.
Updates tailscale/corp#25804
Updates #14655 (found when working on it)
Signed-off-by: Nick Khyl <nickk@tailscale.com>
As we look to add github.com/prometheus/client_golang/prometheus to
more parts of the codebase, lock in that we don't use it in tailscaled,
primarily for binary size reasons.
Updates #12614
Change-Id: I03c100d12a05019a22bdc23ce5c4df63d5a03ec6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This test verifies, among other things, that init functions cannot be deferred after (*DeferredFuncs).Do
has already been called and that all subsequent calls to (*DeferredFuncs).Defer return false.
However, the initial implementation of this check was racy: by the time (*DeferredFuncs).Do returned,
not all goroutines that successfully deferred an init function may have incremented the atomic variable
tracking the number of deferred functions. As a result, the variable's value could differ immediately
after (*DeferredFuncs).Do returned and after all goroutines had completed execution (i.e., after wg.Wait()).
In this PR, we replace the original racy check with a different one. Although this new check is also racy,
it can only produce false negatives. This means that if the test fails, it indicates an actual bug rather than
a flaky test.
Fixes#14039
Signed-off-by: Nick Khyl <nickk@tailscale.com>
There's at least one example of stored routes and advertised routes
getting out of sync. I don't know how they got there yet, but this would
backfill missing advertised routes on startup from stored routes.
Also add logging in LocalBackend.AdvertiseRoute to record when new
routes actually get put into prefs.
Updates #14606
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
I moved the actual rename into separate, GOOS-specific files. On
non-Windows, we do a simple os.Rename. On Windows, we first try
ReplaceFile with a fallback to os.Rename if the target file does
not exist.
ReplaceFile is the recommended way to rename the file in this use case,
as it preserves attributes and ACLs set on the target file.
Updates #14428
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
Nick Khyl
Brad Fitzpatrick
Andrew Lytvynov
Mike O'Driscoll
Tom Proctor
Andrea Gottardo
Adrian Dewhurst
dependabot[bot]
KevinLiang10
James Tucker
Percy Wegmann
Irbe Krumina
Mario Minardi
Aaron Klotz
Jordan Whited