The new logging in 2dd71e64ac is spammy at shutdown:
Receive func ReceiveIPv6 exiting with error: *net.OpError, read udp [::]:38869: raw-read udp6 [::]:38869: use of closed network connection
Receive func ReceiveIPv4 exiting with error: *net.OpError, read udp 0.0.0.0:36123: raw-read udp4 0.0.0.0:36123: use of closed network connection
Skip it if we're in the process of shutting down.
Updates #10976
Change-Id: I4f6d1c68465557eb9ffe335d43d740e499ba9786
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We were selectively uploading it, but we were still gathering it,
which can be a waste of CPU.
Also remove a bunch of complexity that I don't think matters anymore.
And add an envknob to force service collection off on a single node,
even if the tailnet policy permits it.
Fixes#13463
Change-Id: Ib6abe9e29d92df4ffa955225289f045eeeb279cf
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Of tests we wish we could easily add. One day.
Updates #13038
Change-Id: If44646f8d477674bbf2c9a6e58c3cd8f94a4e8df
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1eaad7d3de regressed some tests in another repo that were starting up
a control server on `http://127.0.0.1:nnn`. Because there was no https
running, and because of a bug in 1eaad7d3de (which ended up checking
the recently-dialed-control check twice in a single dial call), we
ended up forcing only the use of TLS dials in a test that only had
plaintext HTTP running.
Instead, plumb down support for explicitly disabling TLS fallbacks and
use it only when running in a test and using `http` scheme control
plane URLs to 127.0.0.1 or localhost.
This fixes the tests elsewhere.
Updates #13597
Change-Id: I97212ded21daf0bd510891a278078daec3eebaa6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I noticed while debugging a test failure elsewhere that our failure
logs (when verbosity is cranked up) were uselessly attributing dial
failures to failure to dial an invalid IP address (this IPv6 address
we didn't have), rather than showing me the actual IPv4 connection
failure.
Updates #13597 (tangentially)
Change-Id: I45ffbefbc7e25ebfb15768006413a705b941dae5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This pulls out the clock and forceNoise443 code into methods on the
Dialer as cleanup in its own commit to make a future change less
distracting.
Updates #13597
Change-Id: I7001e57fe7b508605930c5b141a061b6fb908733
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In prep for a future port 80 MITM fix, make the 'debug ts2021' command
retry once after a failure to give it a chance to pick a new strategy.
Updates #13597
Change-Id: Icb7bad60cbf0dbec78097df4a00e9795757bc8e4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
So it doesn't delete and re-pull when switching between branches.
Updates tailscale/corp#17686
Change-Id: Iffb989781db42fcd673c5f03dbd0ce95972ede0f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Update go.toolchain.rev for https://github.com/tailscale/go/pull/104 and
add a test that, when using the tailscale_go build tag, we use the
right Go toolchain.
We'll crank up the strictness in later commits.
Updates #13527
Change-Id: Ifb09a844858be2beb144a420e4e9dbdc5c03ae3a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In prep for upcoming flow tracking & mutex contention optimization
changes, this change refactors (subjectively simplifying) how the DERP
Server accounts for which peers have written to which other peers, to
be able to send PeerGoneReasonDisconnected messages to writes to
uncache their DRPO (DERP Return Path Optimization) routes.
Notably, this removes the Server.sentTo field which was guarded by
Server.mu and checked on all packet sends. Instead, the accounting is
moved to each sclient's sendLoop goroutine and now only needs to
acquire Server.mu for newly seen senders, the first time a peer sends
a packet to that sclient.
This change reduces the number of reasons to acquire Server.mu
per-packet from two to one. Removing the last one is the subject of an
upcoming change.
Updates #3560
Updates #150
Change-Id: Id226216d6629d61254b6bfd532887534ac38586c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And update a few callers as examples of motivation. (there are a
couple others, but these are the ones where it's prettier)
Updates #cleanup
Change-Id: Ic8c5cb7af0a59c6e790a599136b591ebe16d38eb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
73280595a8 for #2751 added a "clientSet" interface to
distinguish the two cases of a client being singly connected (the
common case) vs tolerating multiple connections from the client at
once. At the time (three years ago) it was kinda an experiment
and we didn't know whether it'd stop the reconnect floods we saw
from certain clients. It did.
So this promotes it to a be first-class thing a bit, removing the
interface. The old tests from 73280595a were invaluable in ensuring
correctness while writing this change (they failed a bunch).
But the real motivation for this change is that it'll permit a future
optimization to add flow tracking for stats & performance where we
don't contend on Server.mu for each packet sent via DERP. Instead,
each client can track its active flows and hold on to a *clientSet and
ask the clientSet per packet what the active client is via one atomic
load rather than a mutex. And if the atomic load returns nil, we'll
know we need to ask the server to see if they died and reconnected and
got a new clientSet. But that's all coming later.
Updates #3560
Change-Id: I9ccda3e5381226563b5ec171ceeacf5c210e1faf
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
It was scaring people. It's been pretty stable for quite some time now
and we're unlikely to change the API and break people at this point.
We might, but have been trying not to.
Fixestailscale/corp#22933
Change-Id: I0c3c79b57ccac979693c62ba320643a940ac947e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I should've bumped capver in 65fe0ba7b5 but forgot.
This lets us turn off the cryptokey routing change from control for
the affected panicky range of commits, based on capver.
Updates #13332
Updates tailscale/corp#20732
Change-Id: I32c17cfcb45b2369b2b560032330551d47a0ce0b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Both for Raspberry Pis, and for running natlab tests faster on Apple
Silicon Macs without emulating x86.
Not fully wired up yet.
Updates #1866
Updates #13038
Change-Id: I1552bf107069308f325f640773cc881ed735b5ab
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
No need to make callers specify the redundant IP version or
TTL/HopLimit or EthernetType in the common case. The mkPacket helper
can set those when unset.
And use the mkIPLayer in another place, simplifying some code.
And rename mkPacketErr to just mkPacket, then move mkPacket to
test-only code, as mustPacket.
Updates #13038
Change-Id: Ic216e44dda760c69ab9bfc509370040874a47d30
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And clean up some of the test helpers in the process.
Updates #13038
Change-Id: I3e2b5f7028a32d97af7f91941e59399a8e222b25
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I'd added this helper for tests, but then moved it to non-test code
and forgot some places to use it. This uses it in more places to
remove some boilerplate.
Updates #13038
Change-Id: Ic4dc339be1c47a55b71d806bab421097ee3d75ed
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This adds tests for DNS requests, and ignoring IPv6 packets on v4-only
networks.
No behavior changes. But some things are pulled out into functions.
And the mkPacket helpers previously just for tests are moved into
non-test code to be used elsewhere to reduce duplication, doing the
checksum stuff automatically.
Updates #13038
Change-Id: I4dd0b73c75b2b9567b4be3f05a2792999d83f6a3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This adds support for sending packets to 33:33:00:00:01 at IPv6
multicast address ff02::1 to send to all nodes.
Nothing in Tailscale depends on this (yet?), but it makes debugging in
VMs behind natlab easier (e.g. you can ping all nodes), and other
things might depend on this in the future.
Mostly I'm trying to flesh out the IPv6 support in natlab now that we
can write vnet tests.
Updates #13038
Change-Id: If590031fcf075690ca35c7b230a38c3e72e621eb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
To test how virtual machines connect to the natlab vnet code.
Updates #13038
Change-Id: Ia4fd4b0c1803580ee7d94cc9878d777ad4f24f82
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And refactor some of vnet.go for testability.
The only behavioral change (with a new test) is that ethernet
broadcasts no longer get sent back to the sender.
Updates #13038
Change-Id: Ic2e7e7d6d8805b7b7f2b5c52c2c5ba97101cef14
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The bad naming (which had only been half updated with the IPv6
changes) tripped me up in the earlier change.
Updates #13038
Change-Id: I65ce07c167e8219d35b87e1f4bf61aab4cac31ff
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The reason they weren't working was because the cmd/tta agent in the
guest was dialing out to the test and the vnet couldn't map its global
unicast IPv6 address to a node as it was just using a
map[netip.Addr]*node and blindly trusting the *node was
populated. Instead, it was nil, so the agent connection fetching
didn't work for its RoundTripper and the test could never drive the
node. That map worked for IPv4 but for IPv6 we need to use the method
that takes into account the node's IPv6 SLAAC address. Most call sites
had been converted but I'd missed that one.
Also clean up some debug, and prohibit nodes' link-local unicast
addresses from dialing 2000::/3 directly for now. We can allow that to
be configured opt-in later (some sort of IPv6 NAT mode. Whatever it's
called.) That mode was working on accident, but was confusing: Linux
would do source address selection from link local for the first few
seconds and then after SLAAC and DAD, switch to using the global
unicast source address. Be consistent for now and force it to use the
global unicast.
Updates #13038
Change-Id: I85e973aaa38b43c14611943ff45c7c825ee9200a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Really we need to fix logpolicy + bootstrapDNS to not be so aggressive,
but this is a quick workaround meanwhile.
Without this, tailscaled starts immediately while IPv6 DAD is
happening for a couple seconds and logpolicy freaks out without the
network available and starts spamming stderr about bootstrap DNS
options. But we see that regularly anyway from people whose wifi is
down. So we need to fix the general case. This is not that fix.
Updates #13038
Change-Id: Iba7e536d08e59d34abded1d279f88fdc9c46d94d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
There were a few places it could get wedged (notably the dial without
a timeout).
And add a knob for verbose debug logs.
And keep two idle connections always.
Updates #13038
Change-Id: I952ad182d7111481d97a83c12aa2ff4bfdc55fe8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Move all the UDP handling to its own func to remove a bunch of "if
isUDP" checks in a bunch of blocks.
Updates #13038
Change-Id: If71d71b49e57651d15bd307a2233c43751cc8639
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I didn't actually see this, but added this while debugging something
and figured it'd be good to keep.
Updates #13038
Change-Id: I67934c8a329e0233f79c3b08516fd6bad6bfe22a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And sprinkle some more docs around.
Updates #13038
Change-Id: Ia2dcf567b68170481cc2094d64b085c6b94a778a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
All the magic service names with virtual IPs will need IPv6 variants.
Pull this out in prep.
Updates #13038
Change-Id: I53b5eebd0679f9fa43dc0674805049258c83a0de
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
So we don't log about them when verbose logging is enabled.
Updates #13038
Change-Id: I925bc3a23e6c93d60dd4fb4bf6a4fdc5a326de95
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The natlab Test Agent (tta) still had its old log streaming hack in
place where it dialed out to anything on TCP port 124 and those logs
were streamed to the host running the tests. But we'd since added gokrazy
syslog streaming support, which made that redundant.
So remove all the port 124 stuff. And then make sure we log to stderr
so gokrazy logs it to syslog.
Also, keep the first 1MB of logs in memory in tta too, exported via
localhost:8034/logs for interactive debugging. That was very useful
during debugging when I added IPv6 support. (which is coming in future
PRs)
Updates #13038
Change-Id: Ieed904a704410b9031d5fd5f014a73412348fa7f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Otherwise you get "Access denied: watch IPN bus access denied, must
set ipn.NotifyNoPrivateKeys when not running as admin/root or
operator".
This lets a non-operator at least start the app and see the status, even
if they can't change everything. (the web UI is unaffected by operator)
A future change can add a LocalAPI call to check permissions and guide
people through adding a user as an operator (perhaps the web client
can do that?)
Updates #1708
Change-Id: I699e035a251b4ebe14385102d5e7a2993424c4b7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And convert a few callers as an example, but nowhere near all.
Updates #12912
Change-Id: I5eaa12a29a6cd03b58d6f1072bd27bc0467852f2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In prep for updating to new staticcheck required for Go 1.23.
Updates #12912
Change-Id: If77892a023b79c6fa798f936fc80428fd4ce0673
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
To avoid dig vs nslookup vs $X availability issues between
OSes/distros. And to be in Go, to match the resolver we use.
Updates #13038
Change-Id: Ib7e5c351ed36b5470a42cbc230b8f27eed9a1bf8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
... rather than abusing the generic tsapp.
Per discussion in https://github.com/gokrazy/gokrazy/pull/275
It also means we can remove stuff we don't need, like ntp or randomd.
Updates #13038
Change-Id: Iccf579c354bd3b5025d05fa1128e32f1d5bde4e4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
It's too new to be supported in Debian bookworm so just remove it.
It doesn't seem to matter or help speed anything up.
Updates #13038
Change-Id: I39077ba8032bebecd75209552b88f1842c843c33
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
84adfa1ba3 made MAC addresses 1-based too, but didn't adjust this IP address
calculation which was based on the MAC address
Updates #13038
Change-Id: Idc112b303b0b85f41fe51fd61ce1c0d8a3f0f57e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The heartbeat package does nothing if not configured anyway, so don't
even put it in the image and pay the cost of it running.
Updates #13038
Updates #1866
Change-Id: Id22c0fb1f8395ad21ab0e0350973d31730e8d39f
The change in b7e48058c8 was too loose; it also captured the CLI
being run as a child process under cmd/tta.
Updates #13038
Updates #1866
Change-Id: Id410b87132938dd38ed4dd3959473c5d0d242ff5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
These three packages aren't in gokrazy/tsapp/config.json but
used to be. Unfortunately, that meant that were being included
in the resulting image. Apparently `gok` doesn't delete them or
warn about them being present on disk when they're moved from
the config file.
Updates #13038
Updates #1866
Change-Id: I54918a9e3286ea755b11dde5e9efdd433b8f8fb8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We had a mix of 0-based and 1-based nodes and MACs in logs.
Updates #13038
Change-Id: I36d1b00f7f94b37b4ae2cd439bcdc5dbee6eda4d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Using https://github.com/gokrazy/gokrazy/pull/275
This is much lower latency than logcatcher, which is higher latency
and chunkier. And this is better than getting it via 'tailscale debug
daemon-logs', which misses early interesting logs.
Updates #13038
Change-Id: I499ec254c003a9494c0e9910f9c650c8ac44ef33
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In particular, tests showing that #3824 works. But that test doesn't
actually work yet; it only gets a DERP connection. (why?)
Updates #13038
Change-Id: Ie1fd1b6a38d4e90fae7e72a0b9a142a95f0b2e8f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Similar to UseSocketOnly, but pulled out separately in case
people are doing unknown weird things.
Updates #13038
Change-Id: I7478e5cb9794439b947440b831caa798941845ea
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
To test local connections.
Updates #13038
Change-Id: I575dcab31ca812edf7d04fa126772611cf89b9a7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
And don't make guests under vnet/natlab upload to logcatcher,
as there won't be a valid cert anyway.
Updates #13038
Change-Id: Ie1ce0139788036b8ecc1804549a9b5d326c5fef5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
It seems some security software or macOS itself might be MITMing TLS
(for ScreenTime?), so don't warn unless it fails x509 validation
against system roots.
Updates #3198
Change-Id: I6ea381b5bb6385b3d51da4a1468c0d803236b7bf
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
All wasi* are GOARCH wasm, so check that instead.
Updates #12732
Change-Id: Id3cc346295c1641bcf80a6c5eb1ad65488509656
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
... and then do approximately nothing with that information, other
than a big TODO. This is mostly me relearning this code and leaving
breadcrumbs for others in the future.
Updates #12724
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
