Avery found a bunch of tests that fail with -count=2.
Updates tailscale/corp#40176 (tracks making our CI detect them)
Change-Id: Ie3e4398070dd92e4fe0146badddf1254749cca20
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Co-authored-by: Avery Pennarun <apenwarr@tailscale.com>
TestLookupMetric was added in e8d140654 (2023-08-17) without
initializing the dnsCache and dnsCacheBytes globals. When run in
isolation, handleBootstrapDNS writes a nil body (from the
uninitialized dnsCacheBytes), causing getBootstrapDNS to fail
decoding an empty response with EOF.
Add a setDNSCache test helper that stores the dnsEntryMap, marshals
dnsCacheBytes, and registers a t.Cleanup to nil both out, so tests
that forget to call it will hit the dnsCache-nil fatal in
getBootstrapDNS rather than silently depending on prior test state.
Also add AssertNotParallel and a dnsCache-nil fatal check to
getBootstrapDNS, the central helper all bootstrap DNS tests flow
through, to prevent future tests from running in parallel (they
all mutate package-level DNS caches and metrics) and to give a
clear error if a test forgets to initialize the DNS caches.
Fixes#19388
Change-Id: I8ad454ec6026c71f13ecfa14d25925df5478b908
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Co-authored-by: Avery Pennarun <apenwarr@tailscale.com>
The natlab-integrationtest CI job frequently flakes by exhausting its
3m go test timeout. The root cause is that the QEMU VMs run under
pure software emulation (TCG) with no KVM. Under TCG, the guest
kernel's timer calibration busy-loops are at the mercy of host CPU
scheduling. When two VMs boot simultaneously on a 2-core CI runner,
one VM's calibration gets starved and produces wrong results, leaving
the kernel with broken timers that prevent it from ever completing
boot — even after the other VM finishes and frees up CPU.
Additionally, the microvm machine type doesn't provide HPET hardware,
but the kernel command line specified clocksource=hpet. And the VM
image build (make natlab) ran inside the test itself, consuming most
of the 3m timeout budget before the actual test started.
Fix by:
- Enabling KVM when /dev/kvm is available, so timer calibration
uses real hardware timers unaffected by host CPU scheduling.
- Adding a CI step to set /dev/kvm permissions on the GitHub
Actions runner (ubuntu-latest provides KVM but needs a udev rule).
- Pre-building the VM image in a separate CI step so it doesn't
cut into the go test -timeout budget.
- Replacing the hardcoded 60s context timeout with one derived from
t.Deadline(), so the test uses the full -timeout budget.
- Adding VM boot progress detection (AwaitFirstPacket) and QMP
diagnostics, so boot failures produce clear errors instead of
opaque "context deadline exceeded" messages.
With KVM enabled, the test passes reliably even on a single CPU core
with 3 parallel workers — a scenario that was 100% broken under TCG.
Fixes#18906
Change-Id: I4c87631a9c9678d185b9f30cb05c0f7bfa9f5c62
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
For tests to loudly declare (and panic on violation) when they're doing
something that's not safe in a parallel test.
Fixes#19385
Change-Id: If79693b0c235c146871a05ed74fa9ea75bb500f9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The maxInFlightConnectionAttemptsForTest and
maxInFlightConnectionAttemptsPerClientForTest globals were plain ints
read by background gVisor TCP handler goroutines (via
wrapTCPProtocolHandler) and written by tstest.Replace cleanup in
TestTCPForwardLimits_PerClient. When a gVisor goroutine outlived the
test cleanup window, the race detector caught the unsynchronized
access.
The race-prone code was introduced in c5abbcd4b4 (2024-02-26,
"wgengine/netstack: add a per-client limit for in-flight TCP
forwards") which added both the plain int globals and the
TestTCPForwardLimits_PerClient test that writes them via
tstest.Replace. It is not obvious why this has only recently started
being detected as a data race; likely some combination of gVisor
version bumps, Go toolchain scheduler changes, and additional
TCP-injecting subtests (e.g. 03461ea7f, 2026-01-30) increased
goroutine churn enough to hit the window.
Change both globals to atomic.Int32 and replace tstest.Replace (which
does non-atomic *target = old on cleanup) with explicit Store/Cleanup
pairs.
Fixes#19118
Change-Id: Id26ba6fbfb2e4ade319976db80af8e16c7c8778e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When built with the Tailscale Go toolchain, include the toolchain's
git revision in the version output. The non-JSON output shows the
first 10 hex digits:
go version: go1.26.2 (tailscale/go dfe2a5fd8e)
The JSON output includes the full hash as "tailscaleGoGitHash", or
omits the field when not using tsgo.
The toolchain rev is read via a separate sync.OnceValue rather than
piggybacking on getEmbeddedInfo, because that function discards all
data when VCS fields are absent (e.g. in test binaries), while the
tailscale.toolchain.rev setting is still present.
Also add a CI-only test verifying tailscaleToolchainRev is non-empty
when built with the tailscale_go build tag.
Fixes#19374
Change-Id: Ied0b16d7aead5471d8c614c30cba8b0dcf80c691
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Parallelize the SSH integration tests across OS targets and reduce
per-container overhead:
- CI: use GitHub Actions matrix strategy to run all 4 OS containers
(ubuntu:focal, ubuntu:jammy, ubuntu:noble, alpine:latest) in parallel
instead of sequentially (~4x wall-clock improvement)
- Makefile: run docker builds in parallel for local dev too
- Dockerfile: consolidate ~20 separate RUN commands into 5 (one per
test phase), eliminating Docker layer overhead. Combine test binary
invocations where no state mutation is needed between them. Fix a bug
where TestDoDropPrivileges was silently not being run (was passed as a
second positional arg to -test.run instead of using regex alternation).
- TestMain: replace tail -F + 2s sleep with synchronous log read,
eliminating 2s overhead per test binary invocation. Set debugTest once
in TestMain instead of redundantly in each test function.
- session.read(): close channel on EOF so non-shell tests return
immediately instead of waiting for the 1s silence timeout.
Updates #19244
Change-Id: I2cc8588964fbce0dd7b654fb94e7ff33440b8584
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I'm not sure how this file got into the repo without gofmt.
Maybe gofmt rules changed in some Go release?
Updates #cleanup
Change-Id: Ia8bd46e29f116f7fbfca11be80c8ef48699cd9f2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Verify that GODEBUG=gocachehash=1 output from ./tool/go includes the
git revision from go.toolchain.rev, ensuring that bumping the Tailscale
Go fork (without a Go version number change) properly invalidates the
build cache.
The test only runs in CI or when the current Go binary is the Tailscale
toolchain (GOROOT contains /.cache/tsgo/), so open source contributors
using stock Go aren't forced to download tsgo.
Fixestailscale/corp#36589
Change-Id: Ia98d3a3aa8c7fa67f9a0293066fa02a1997dcb95
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a --headless flag to the Host.app Run subcommand for running
macOS VMs without a GUI, enabling use from test frameworks.
Key changes:
- HostCli.swift: When --headless is set, run the VM via VMController
+ RunLoop.main.run() instead of NSApplicationMain. Using the
RunLoop (not dispatchMain) is required because VZ framework
callbacks depend on RunLoop sources.
- VMController.swift: Add headless parameter to createVirtualMachine
that configures a single socket-based NIC (no NAT NIC). This
matches the NIC configuration used when creating/saving VMs, so
saved state restoration works correctly. A NIC count mismatch
causes VZ to silently fail to execute guest code.
- TailMacConfigHelper.swift: Clean up socket network device logging.
- Config.swift: Move VM storage from ~/VM.bundle to
~/.cache/tailscale/vmtest/macos/.
- TailMac.swift: Fix dispatchMain→RunLoop.main.run() in the create
command (same VZ RunLoop requirement).
Updates #13038
Change-Id: Iea51c043aa92e8fc6257139b9f0e2e7677072fa2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add natlabapp.arm64 config and gokrazydeps.go for building a gokrazy
natlab appliance image targeting arm64 (Apple Silicon). This is the
arm64 counterpart to the existing natlabapp (amd64) used by vmtest.
The arm64 image uses github.com/gokrazy/kernel.arm64 and is built
with "make natlab-arm64" in the gokrazy directory.
Updates #13038
Change-Id: I0e1f8e5840083a5de5954f2cf46e3babec129d96
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Start using a common helper for tests to declare that they require root.
This is step 1. A later step will then make this helper track which tests were
skipped so a subsequent pass will run these test as root.
Updates tailscale/corp#40007
Change-Id: I4979e1def0fa3691d38c83f48c89aaa443e7f62e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
As a warm-up to making natlab support multiple operating systems,
start with an easy one (in that it's also Unixy and open source like
Linux) and add FreeBSD 15.0 as a VM OS option for the vmtest
integration test framework, and add TestSubnetRouterFreeBSD which
tests subnet routing through a FreeBSD VM (Gokrazy → FreeBSD →
Gokrazy).
Key changes:
- Add FreeBSD150 OSImage using the official FreeBSD 15.0
BASIC-CLOUDINIT cloud image (xz-compressed qcow2)
- Add GOOS()/IsFreeBSD() methods to OSImage for cross-compilation
and OS-specific behavior
- Handle xz-compressed image downloads in ensureImage
- Refactor compileBinaries into compileBinariesForOS to support
multiple GOOS targets (linux, freebsd), with binaries registered
at <goos>/<name> paths on the file server VIP
- Add FreeBSD-specific cloud-init (nuageinit) user-data generation:
string-form runcmd (nuageinit doesn't support YAML arrays),
fetch(1) instead of curl, FreeBSD sysctl names for IP forwarding,
mkdir /usr/local/bin, PATH setup for tta
- Skip network-config in cidata ISO for FreeBSD (DHCP via rc.conf)
Updates tailscale/tailscale#13038
Change-Id: Ibeb4f7d02659d5cd8e3a7c3a66ee7b1a92a0110d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add tstest/natlab/vmtest, a high-level framework for running multi-VM
integration tests with mixed OS types (gokrazy + Ubuntu/Debian cloud
images) connected via natlab's vnet virtual network.
The vmtest package provides:
- Env type that orchestrates vnet, QEMU processes, and agent connections
- OS image support (Gokrazy, Ubuntu2404, Debian12) with download/cache
- QEMU launch per OS type (microvm for gokrazy, q35+KVM for cloud)
- Cloud-init seed ISO generation with network-config for multi-NIC
- Cross-compilation of test binaries for cloud VMs
- Debug SSH NIC on cloud VMs for interactive debugging
- Test helpers: ApproveRoutes, HTTPGet, TailscalePing, DumpStatus,
WaitForPeerRoute, SSHExec
TTA enhancements (cmd/tta):
- Parameterize /up (accept-routes, advertise-routes, snat-subnet-routes)
- Add /set, /start-webserver, /http-get endpoints
- /http-get uses local.Client.UserDial for Tailscale-routed requests
- Fix /ping for non-gokrazy systems
TestSubnetRouter exercises a 3-VM subnet router scenario:
client (gokrazy) → subnet-router (Ubuntu, dual-NIC) → backend (gokrazy)
Verifies HTTP access to the backend webserver through the Tailscale
subnet route. Passes in ~30 seconds.
Updates tailscale/tailscale#13038
Change-Id: I165b64af241d37f5f5870e796a52502fc56146fa
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add misc/install-git-hooks.go and misc/git_hook/ to the OSS repo,
adapted from the corp repo. The primary motivation is Change-Id
generation in commit messages, which provides a persistent identifier
for a change across cherry-picks between branches.
The installer uses "git rev-parse --git-common-dir" instead of go-git
to find the hooks directory, avoiding a new direct dependency while
still supporting worktrees.
Hooks included:
- commit-msg: adds Change-Id trailer
- pre-commit: blocks NOCOMMIT / DO NOT SUBMIT markers
- pre-push: blocks local-directory replace directives in go.mod
- post-checkout: warns when the hook binary is outdated
Also update docs/commit-messages.md to reflect that Change-Id is no
longer optional in the OSS repo.
Updates tailscale/corp#39860
Change-Id: I09066b889118840c0ec6995cc03a9cf464740ffa
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
go.cmd used cmd.exe to invoke PowerShell, which mangled arguments:
cmd.exe treats ^ as an escape character (so -run "^$" became -run "$",
running all tests instead of none) and = signs also caused issues in
the PowerShell→cmd.exe argument passing layer.
Replace it with a tiny no_std Rust binary (19KB, 32-bit x86 for
universal Windows compat: x86/x64/ARM64) that directly invokes the
Tailscale Go toolchain via CreateProcessW. The raw command line from
GetCommandLineW is passed through to CreateProcessW with only argv[0]
replaced, so arguments are never parsed or re-escaped.
The binary also handles first-run toolchain download natively using
curl.exe and tar.exe (both ship with Windows 10+), so PowerShell is
no longer required for normal operation. The PowerShell fallback is
only used for the rare TS_USE_GOCROSS=1 path.
PowerShell prefers go.exe over go.cmd when resolving ./tool/go, so
this is a drop-in replacement.
With go.exe in place, the CI can use the natural -bench=. -benchtime=1x
-run="^$" flags directly.
Also removes tool/go-win.ps1 which is now unused.
Updates #19255
Change-Id: I80da23285b74796e7694b89cff29a9fa0eaa6281
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Multi-NIC support:
- Add nodeNIC type and node.extraNICs for secondary network interfaces
- Add netForMAC/macForNet to route packets to the correct network by MAC
- Update initFromConfig to allocate a MAC + LAN IP per network
- Fix handleEthernetFrameFromVM, ServeUnixConn to use netForMAC
- Fix MACOfIP, writeEth, WriteUDPPacketNoNAT, gVisor write path, and
createARPResponse to use macForNet (return the MAC actually on that
network, not the node's primary MAC)
- Fix createDHCPResponse for multi-NIC (correct client IP and subnet)
- Add nodeNICMac for secondary NIC MAC generation
- Add Node accessors: NumNICs, NICMac, Networks, LanIP
DHCP fixes:
- Include LeaseTime, SubnetMask, Router, DNS in DHCP Offer (not just
Ack). systemd-networkd requires these to accept an Offer.
- Fix DHCP response source IP: use gateway IP instead of echoing
the request's destination (which was 255.255.255.255 for discovers)
New VIPs:
- cloud-init.tailscale: serves per-node cloud-init meta-data, user-data,
and network-config for VMs booting with nocloud datasource
- files.tailscale: serves binary files (tta, tailscale, tailscaled)
registered via RegisterFile for cloud VM provisioning
- Add ControlServer() accessor for test control server
This is necessary for a three-VM natlab subnet router
integration test, coming later.
Updates #13038
Change-Id: I59f9f356bae9b5509c117265237983972dfdd5af
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When SetSubnetRoutes is called, also send updatePeerChanged to all
other connected nodes so they re-fetch their MapResponse and learn
about the updated AllowedIPs. Without this, peers never see new
subnet routes until they happen to reconnect to the control server.
Discovered while working on a three-VM natlab subnet router
integration test, coming later.
Updates #13038
Change-Id: I20e7a2fda994a8ab0e7a24240e6eae536f4f5f15
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add ExtraRootCAs *x509.CertPool to tsd.System and plumb it through
the control client, noise transport, DERP, and wgengine layers so
that platforms like Android can inject user-installed CA certificates
into Go's TLS verification.
tlsdial.Config now honors base.RootCAs as additional trusted roots,
tried after system roots and before the baked-in LetsEncrypt fallback.
SetConfigExpectedCert gets the same treatment for domain-fronted DERP.
The Android client will set sys.ExtraRootCAs with a pool built from
x509.SystemCertPool + user-installed certs obtained via the Android
KeyStore API, replacing the current SSL_CERT_DIR environment variable
approach.
Updates #8085
Change-Id: Iecce0fd140cd5aa0331b124e55a7045e24d8e0c2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Move the ipn/desktop blank import from cmd/tailscaled/tailscaled_windows.go
into feature/condregister/maybe_desktop_sessions.go, consistent with how
all other modular features are registered. tailscaled already imports
feature/condregister, so it still gets ipn/desktop on Windows.
Updates #12614
Change-Id: I92418c4bf0e67f0ab40542e47584762ac0ffa2b2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a new "ipnbus" build feature tag so the watch-ipn-bus LocalAPI
endpoint can be independently controlled, rather than being gated
behind HasDebug || HasServe. Minimal/embedded builds that omit both
debug and serve were getting 404s on watch-ipn-bus, breaking
"tailscale up --authkey=..." and other CLI flows that depend on
WatchIPNBus.
In the CLI, check buildfeatures.HasIPNBus before attempting to watch
the IPN bus in "tailscale up"/"tailscale login", and exit early with
an informational message when the feature is omitted.
Also add the missing NewCounterFunc stub to clientmetric/omit.go,
which caused compilation errors when building with
ts_omit_clientmetrics and netstack enabled.
Fixes#19240
Change-Id: I2e3c69a72fc50fa02542b91b8a54859618a463d1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When a recording upload fails mid-session, the recording goroutine
cancels the session context. This triggers two concurrent paths:
exec.CommandContext kills the process (causing cmd.Wait to return),
and killProcessOnContextDone tries to write the termination message
via exitOnce.Do. If cmd.Wait returns first, the main goroutine's
exitOnce.Do(func(){}) steals the once, and the termination message
is never written to the client.
Fix by waiting for killProcessOnContextDone to finish writing the
termination message (via <-ss.exitHandled) before claiming exitOnce,
when the context is already done.
Also fix the fallback path when launchProcess itself fails due to
context cancellation: use SSHTerminationMessage() with the correct
"\r\n\r\n" framing instead of fmt.Fprintf with the internal error
string.
Deflakes TestSSHRecordingCancelsSessionsOnUploadFailure, which was
failing consistently at a low rate due to the exitOnce race. After
this fix, flakestress passes with 8,668 runs, 0 failures.
Fixes#7707 (again. hopefully for good.)
Change-Id: I5ab911c71574db8d3f9d979fb839f273be51ecf9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Commit f905871fb moved host key generation from the ipnLocalBackend
interface (GetSSH_HostKeys) to the standalone getHostKeys function,
which requires either system host keys in /etc/ssh/ or a valid
TailscaleVarRoot to generate keys into. The testBackend returned ""
for TailscaleVarRoot, and the Docker test containers only install
openssh-client (no server host keys), so getHostKeys always failed.
When getHostKeys fails, HandleSSHConn returns the error but never
closes the TCP connection, so SSH clients hang forever waiting for
the server hello.
Fix by creating a temp directory in TestMain and returning it from
testBackend.TailscaleVarRoot().
Regression from f905871fb#18949 ("ipn/ipnlocal, feature/ssh: move SSH code
out of LocalBackend to feature").
I was apparently too impatient to wait for the test to complete
and didn't connect the dots: https://github.com/tailscale/tailscale/actions/runs/22930275950
We should make that test faster (#19244) for the patience issue, but
also fail more nicely if this happens in the future.
Updates #19244
Change-Id: If82393b8f35413b04174e6f7d09a1ee3a2125a6b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The cloner and viewer code generators didn't handle named types
with basic underlying types (map/slice) that have their own Clone
or View methods. For example, a type like:
type Map map[string]any
func (m Map) Clone() Map { ... }
func (m Map) View() MapView { ... }
When used as a struct field, the cloner would descend into the
underlying map[string]any and fail because it can't clone the any
(interface{}) value type. Similarly, the viewer would try to create
a MapFnOf view and fail.
Fix the cloner to check for a Clone method on the named type
before falling through to the underlying type handling.
Fix the viewer to check for a View method on named map/slice types,
so the type author can provide a purpose-built safe view that
doesn't leak raw any values. Named map/slice types without a View
method fall through to normal handling, which correctly rejects
types like map[string]any as unsupported.
Updates tailscale/corp#39502 (needed by tailscale/corp#39594)
Change-Id: Iaef0192a221e02b4b8e409c99ef8398090327744
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
To denoise log output, to make it easier to find real failures.
Updates #19252
Change-Id: Iae64a9278c70de24a236c39e3d181a509a512a0b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The -run "^$" flag was being mangled by cmd.exe's argument processing.
The ^ character is cmd.exe's escape character, so go.cmd's cmd.exe layer
eats it, turning -run "^$" into -run "$" which matches all test names.
This caused the benchmark job to run every test, leading to timeouts
and Go runtime crashes.
Use -run XXXXNothingXXXX instead, which avoids special characters
entirely.
Updates #19252
Change-Id: I888c124254dd2767a40b61bcd68dbc9b22ad35a1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The upload-client-metrics handler called metricCapture without
checking if it was nil or if the metrics slice was empty. Most
tests pass nil for metricCapture, so if a metrics upload races
in during the test, it panics.
Fixes#19252
Change-Id: Ib904d1fe6779067dc2a153d1680b8f50cba9c773
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The tailscale-online.target and tailscale-wait-online.service systemd
units were added in 30e12310f1 but never included in the release
packaging (tarballs, debs, rpms).
Updates #11504
Change-Id: I93e03e1330a7ff8facf845c7ca062ed2f0d35eaa
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Before sending a fix for #18991, this adds an integration test that
locks in that the proxymap WhoIs code works with two nodes running as
different users, with the second node running a localhost service and
able to use its local tailscaled to identify a Tailscale connection
from the other tailscaled.
Updates #18991
Change-Id: I6fbb0810204d77d2ac558f0cc786b73e3248d031
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add two small APIs to support out-of-tree projects to exchange custom
signaling messages over DERP without requiring disco protocol
extensions:
- OnDERPRecv callback on magicsock.Options / wgengine.Config: called for
every non-disco DERP packet before the peer map lookup, allowing callers
to intercept packets from unknown peers that would otherwise be dropped.
- SendDERPPacketTo method on magicsock.Conn: sends arbitrary bytes to a
node key via a DERP region, creating the connection if needed. Thin
wrapper around the existing internal sendAddr.
Also allow netstack.Start to accept a nil LocalBackend for use cases
that wire up TCP/UDP handlers directly without a full LocalBackend.
Updates tailscale/corp#24454
Change-Id: I99a523ef281625b8c0024a963f5f5bf5d8792c17
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Fix three independent flake sources, at least as debugged by Claude,
though empirically no longer flaking as it was before:
1. Poll for connection counter data instead of reading immediately.
The conncount callback fires asynchronously on received WireGuard
traffic, so after counts.Reset() there is no guarantee the counter
has been repopulated before checkStats reads it. Use tstest.WaitFor
with a 5s timeout to retry until a matching connection appears.
2. Replace the *2 symmetry assumption in global metric assertions.
metricSendUDP and friends are AggregateCounters that sum per-conn
expvars from both magicsock instances. The old assertion assumed
both instances had identical packet counts, which breaks under
asymmetric background WireGuard activity (handshake retries, etc).
The new assertGlobalMetricsMatchPerConn computes the actual sum of
both conns' expvars and compares against the AggregateCounter value.
3. Tolerate physical stats being 0 when user metrics are non-zero.
A rebind event replaces the socket mid-measurement, resetting the
physical connection counter while user metrics still reflect packets
processed before the rebind. Log instead of failing in this case.
Also move counts.Reset() after metric reads and reorder the reset
sequence (counts before metrics) to minimize the race window.
Fixestailscale/tailscale#13420
Change-Id: I7b090a4dc229a862c1a52161b3f2547ec1d1f23f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This makes tsnet apps not depend on x/crypto/ssh and locks that in with a test.
It also paves the wave for tsnet apps to opt-in to SSH support via a
blank feature import in the future.
Updates #12614
Change-Id: Ica85628f89c8f015413b074f5001b82b27c953a9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Two issues caused TestCollectPanic to flake:
1. ETXTBSY: The test exec'd the tailscaled binary directly without
going through StartDaemon/awaitTailscaledRunnable, so it lacked
the retry loop that other tests use to work around a mysterious
ETXTBSY on GitHub Actions.
2. Shared filch files: The test didn't pass --statedir or TS_LOGS_DIR,
so all parallel test instances wrote panic logs to the shared system
state directory (~/.local/share/tailscale). Concurrent runs would
clobber each other's filch log files, causing the second run to not
find the panic data from the first.
Fix both by adding awaitTailscaledRunnable before the first exec, and
passing --statedir and TS_LOGS_DIR to isolate each test's log files,
matching what StartDaemon does.
It now passes x/tools/cmd/stress.
Fixes#15865
Change-Id: If18b9acf8dbe9a986446a42c5d98de7ad8aae098
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Raw byte accessors for key types, mirroring existing patterns
(NodePublic.Raw32 and DiscoPublicFromRaw32 already exist).
NodePrivate.Raw32 returns the raw 32 bytes of a node private key.
DiscoPrivateFromRaw32 parses a 32-byte raw value as a DiscoPrivate.
Updates tailscale/corp#24454
Change-Id: Ibc08bed14ab359eddefbebd811c375b6365c7919
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We did so for Linux and macOS already, so also do so for Windows. We
only didn't already because originally we never produced binaries for
it (due to our corp repo not needing them), and later because we had
no ./tool/go wrapper. But we have both of those things now.
Updates #18884
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When a recording upload fails mid-session, killProcessOnContextDone
writes the termination message to ss.Stderr() and kills the process.
Meanwhile, run() takes the ss.ctx.Done() path and proceeds to
ss.Exit(), which tears down the SSH channel. The termination message
write races with the channel teardown, so the client sometimes never
receives it.
Fix by adding an exitHandled channel that killProcessOnContextDone
closes when done. run() now waits on this channel after ctx.Done()
fires, ensuring the termination message is fully written before
the SSH channel is torn down.
Fixes#7707
Change-Id: Ib60116c928d3af46d553a4186a72963c2c731e3e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This had gotten flaky with Go 1.26.
Use synctest + AllocsPerRun to make it fast and deterministic.
Updates #18682
Change-Id: If673d6ecd8c1177f59c1b9c0f3fca42309375dff
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Otherwise it gets confused on new(123) etc.
Updates #18682
Change-Id: I9e2e93ea24f2b952b2396dceaf094b4db64424b0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Fix its/it's, who's/whose, wether/whether, missing apostrophes
in contractions, and other misspellings across the codebase.
Updates #cleanup
Change-Id: I20453b81a7aceaa14ea2a551abba08a2e7f0a1d8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Go 1.26's url.Parser is stricter and made our tests elsewhere fail
with this scheme because when these listen addresses get shoved
into a URL, it can't parse back out.
I verified this makes tests elsewhere pass with Go 1.26.
Updates #18682
Change-Id: I04dd3cee591aa85a9417a0bbae2b6f699d8302fa
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This is a minimal hacky fix for a case where the portlist poller extension
could miss updates to NetMap's CollectServices bool.
Updates tailscale/corp#36813
Change-Id: I9b50de8ba8b09e4a44f9fbfe90c9df4d8ab4d586
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This makes Set.MarshalJSON produce deterministic output in many cases now.
We still need to do make it deterministic for non-ordered types.
Updates #18808
Change-Id: I7f341ec039c661a8e88d07d7f4dc0f15d5d4ab86
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
For paring back build tag variant CI runs' set of packages to test.
Updates tailscale/corp#28679
Change-Id: Iba46fd1f58c1eaee1f7888ef573bc8b14fa73208
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The old check was too aggressive and required TS_GO_NEXT=1 at runtime
as well, which is too strict and onerous.
This is a sanity check only (and an outdated one, at that); it's okay for
it to be slightly loose and permit two possible values. If either is working,
we're already way past the old bug that this was introduced to catch.
Updates tailscale/corp#36382
Change-Id: Ib9a62e10382cd889ba590c3539e6b8535c6b19fe
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This adds a new ControlKnob to make MagicDNS IPv6 registration
(telling systemd/etc) opt-out rather than opt-in.
Updates #15404
Change-Id: If008e1cb046b792c6aff7bb1d7c58638f7d650b1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
OnPolicyChange can observe a conn in activeConns before authentication
completes. The previous `c.info == nil` guard was itself a data race
against clientAuth writing c.info, and even when c.info appeared
non-nil, c.localUser could still be nil, causing a nil pointer
dereference at c.localUser.Username.
Add an authCompleted atomic.Bool to conn, stored true after all auth
fields are written in clientAuth. OnPolicyChange checks this atomic
instead of c.info, which provides the memory barrier guaranteeing all
prior writes are visible to the concurrent reader.
Updates tailscale/corp#36268 (fixes, but we might want to cherry-pick)
Co-authored-by: Gesa Stupperich <gesa@tailscale.com>
Change-Id: I4c69843541f5f9f04add9bf431e320c65a203a39
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The gocross-wrapper.sh bash script already checks TS_GO_NEXT (as of
a374cc344e) to select go.toolchain.next.rev over go.toolchain.rev,
but when TS_USE_GOCROSS=1 the Go binary itself was hardcoded to read
go.toolchain.rev. This makes gocross also respect the TS_GO_NEXT=1
environment variable.
Updates tailscale/corp#36382
Change-Id: I04bef25a34e7ed3ccb1bfdb33a3a1f896236c6ee
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This switches our gokrazy builds to use a new variant of cmd/gok called
opinionated about using monorepos: https://github.com/bradfitz/monogok
And with that, we can get rid of all the go.mod files and builddir forests
under gokrazy/**.
Updates #13038
Updates gokrazy/gokrazy#361
Change-Id: I9f18fbe59b8792286abc1e563d686ea9472c622d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This updates the TS_GO_NEXT=1 (testing) toolchain to Go 1.26.0
The default one is still Go 1.25.x.
Updates #18682
Change-Id: I99747798c166ce162ee9eee74baa9ff6744a62f6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Restore synchronous method calls from LocalBackend to magicsock.Conn
for node views, filter, and delta mutations. The eventbus delivery
introduced in 8e6f63cf1 was invalid for these updates because
subsequent operations in the same call chain depend on magicsock
already having the current state. The Synchronize/settleEventBus
workaround was fragile and kept requiring more workarounds and
introducing new mystery bugs.
Since eventbus was added, we've since learned more about when to use
eventbus, and this wasn't one of the cases.
We can take another swing at using eventbus for netmap changes in a
future change.
Fixes#16369
Updates #18575 (likely fixes)
Change-Id: I79057cc9259993368bb1e350ff0e073adf6b9a8f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add new "webbrowser" and "colorable" feature tags so that the
github.com/toqueteos/webbrowser and mattn/go-colorable packages
can be excluded from minbox builds.
Updates #12614
Change-Id: Iabd38b242f5a56aa10ef2050113785283f4e1fe8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We already had a featuretag for clientupdate, but the CLI wasn't using
it, making the "minbox" build (minimal combined tailscaled + CLI
build) larger than necessary.
Updates #12614
Change-Id: Idd7546c67dece7078f25b8f2ae9886f58d599002
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This provides a mechanism to block, waiting for Tailscale's IP to be
ready for a bind/listen, to gate the starting of other services.
It also adds a new --assert=[IP] option to "tailscale ip", for services
that want extra paranoia about what IP is in use, if they're worried about
having switched to the wrong tailnet prior to reboot or something.
Updates #3340
Updates #11504
... and many more, IIRC
Change-Id: I88ab19ac5fae58fd8c516065bab685e292395565
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
So callers can run testwrapper with -vet=off if they're already
running vet explicitly in a concurrent test job.
Updates tailscale/corp#28679
Change-Id: I74ad56e560076d187f5e3a7d7381e1dac89d860c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This is for a future test scheduler, so it can run potentially flaky
tests separately, doing all the non-flaky ones together in one batch.
Updates tailscale/corp#28679
Change-Id: Ic4a11f9bf394528ef75792fd622f17bc01a4ec8a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When TS_GO_NEXT=1 is set, update/use the
go.toolchain.next.{branch,rev} files instead.
This lets us do test deploys of Go release candidates on some
backends, without affecting all backends.
Updates tailscale/corp#36382
Change-Id: I00dbde87b219b720be5ea142325c4711f101a364
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We don't hold q.mu while running normal ExecQueue.Add funcs, so we
shouldn't in RunSync either. Otherwise code it calls can't shut down
the queue, as seen in #18502.
Updates #18052
Co-authored-by: Nick Khyl <nickk@tailscale.com>
Change-Id: Ic5e53440411eca5e9fabac7f4a68a9f6ef026de1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And fix up the TestAutoUpdateDefaults integration tests as they
weren't testing reality: the DefaultAutoUpdate is supposed to only be
relevant on the first MapResponse in the stream, but the tests weren't
testing that. They were instead injecting a 2nd+ MapResponse.
This changes the test control server to add a hook to modify the first
map response, and then makes the test control when the node goes up
and down to make new map responses.
Also, the test now runs on macOS where the auto-update feature being
disabled would've previously t.Skipped the whole test.
Updates #11502
Change-Id: If2319bd1f71e108b57d79fe500b2acedbc76e1a6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
(trying to get in smaller obvious chunks ahead of later PRs to make
them smaller)
Updates #17925
Change-Id: I184002001055790484e4792af8ffe2a9a2465b2e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
