Add server-side per-client bandwidth enforcement using TCP backpressure.
When configured, the server calls WaitN after reading each DERP frame,
which delays the next read, fills the TCP receive buffer, shrinks
the TCP window, and naturally throttles the sender — no packets are dropped.
- Rate limiting is on the receive (inbound) side, which is what an abusive
client controls
- Mesh peers are exempt since they are trusted infrastructure
- The burst size is at least MaxPacketSize (64KB) to ensure a
single max-size frame can always be processed
Also refactors sclient to store a context.Context directly instead of a
done channel, which simplifies the rate limiter's WaitN call.
Flags added to cmd/derper:
--per-client-rate-limit (bytes/sec, default 0 = unlimited)
--per-client-rate-burst (bytes, default 0 = 2x rate limit)
Example for 10Mbps: --per-client-rate-limit=1250000
Updates #38509
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Move the ipn/desktop blank import from cmd/tailscaled/tailscaled_windows.go
into feature/condregister/maybe_desktop_sessions.go, consistent with how
all other modular features are registered. tailscaled already imports
feature/condregister, so it still gets ipn/desktop on Windows.
Updates #12614
Change-Id: I92418c4bf0e67f0ab40542e47584762ac0ffa2b2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a new "ipnbus" build feature tag so the watch-ipn-bus LocalAPI
endpoint can be independently controlled, rather than being gated
behind HasDebug || HasServe. Minimal/embedded builds that omit both
debug and serve were getting 404s on watch-ipn-bus, breaking
"tailscale up --authkey=..." and other CLI flows that depend on
WatchIPNBus.
In the CLI, check buildfeatures.HasIPNBus before attempting to watch
the IPN bus in "tailscale up"/"tailscale login", and exit early with
an informational message when the feature is omitted.
Also add the missing NewCounterFunc stub to clientmetric/omit.go,
which caused compilation errors when building with
ts_omit_clientmetrics and netstack enabled.
Fixes#19240
Change-Id: I2e3c69a72fc50fa02542b91b8a54859618a463d1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Brings in a newer version of Gliderlabs SSH with added socket forwarding support.
Fixes#12409Fixes#5295
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
The cloner and viewer code generators didn't handle named types
with basic underlying types (map/slice) that have their own Clone
or View methods. For example, a type like:
type Map map[string]any
func (m Map) Clone() Map { ... }
func (m Map) View() MapView { ... }
When used as a struct field, the cloner would descend into the
underlying map[string]any and fail because it can't clone the any
(interface{}) value type. Similarly, the viewer would try to create
a MapFnOf view and fail.
Fix the cloner to check for a Clone method on the named type
before falling through to the underlying type handling.
Fix the viewer to check for a View method on named map/slice types,
so the type author can provide a purpose-built safe view that
doesn't leak raw any values. Named map/slice types without a View
method fall through to normal handling, which correctly rejects
types like map[string]any as unsupported.
Updates tailscale/corp#39502 (needed by tailscale/corp#39594)
Change-Id: Iaef0192a221e02b4b8e409c99ef8398090327744
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This is a follow-up to #19117, adding a debug CLI command allowing the operator
to explicitly discard cached netmap data, as a safety and recovery measure.
Updates #12639
Change-Id: I5c3c47c0204754b9c8e526a4ff8f69d6974db6d0
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
* cmd/k8s-operator/e2e: add L7 HA ingress test
Change-Id: Ic017e4a7e3affbc3e2a87b9b6b9c38afd65f32ed
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
* cmd/k8s-operator: add further E2E tests for Ingress (#34833)
This change adds E2E tests for L3 HA Ingress and L7 Ingress (Standalone and
HA). Updates the existing L3 Ingress test to use the Service's Magic DNS
name to test connectivity.
Also refactors test setup to set TS_DEBUG_ACME_DIRECTORY_URL only for tests
running against devcontrol, and updates the Kind node image from v1.30.0 to
v1.35.0.
Fixestailscale/corp#34833
Signed-off-by: Becky Pauley <becky@tailscale.com>
---------
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Signed-off-by: Becky Pauley <becky@tailscale.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
In #10057, @seigel pointed out an inconsistency in the help text for
`exit-node list` and `set --exit-node`:
1. Use `tailscale exit-node list`, which has a column titled "hostname"
and tells you that you can use a hostname with `set --exit-node`:
```console
$ tailscale exit-node list
IP HOSTNAME COUNTRY CITY STATUS
100.98.193.6 linode-vps.tailfa84dd.ts.net - - -
[…]
100.93.242.75 ua-iev-wg-001.mullvad.ts.net Ukraine Kyiv -
# To view the complete list of exit nodes for a country, use `tailscale exit-node list --filter=` followed by the country name.
# To use an exit node, use `tailscale set --exit-node=` followed by the hostname or IP.
# To have Tailscale suggest an exit node, use `tailscale exit-node suggest`.
```
(This is the same format hostnames are presented in the admin
console.)
2. Try copy/pasting a hostname into `set --exit-node`:
```console
$ tailscale set --exit-node=linode-vps.tailfa84dd.ts.net
invalid value "linode-vps.tailfa84dd.ts.net" for --exit-node; must be IP or unique node name
```
3. Note that the command allows some hostnames, if they're from nodes
in a different tailnet:
```console
$ tailscale set --exit-node= ua-iev-wg-001.mullvad.ts.net
$ echo $?
0
```
This patch addresses the inconsistency in two ways:
1. Allow using `tailscale set --exit-node=` with an FQDN that's missing
the trailing dot, matching the formatting used in `exit-node list`
and the admin console.
2. Make the description of valid exit nodes consistent across commands
("hostname or IP").
Updates #10057
Change-Id: If5d74f950cc1a9cc4b0ebc0c2f2d70689ffe4d73
Signed-off-by: Alex Chan <alexc@tailscale.com>
This avoids putting "DisablementSecrets" in the JSON output from
`tailscale lock log`, which is potentially scary to somebody who doesn't
understand the distinction.
AUMs are stored and transmitted in CBOR-encoded format, which uses an
integer rather than a string key, so this doesn't break already-created
TKAs.
Fixes#19189
Change-Id: I15b4e81a7cef724a450bafcfa0b938da223c78c9
Signed-off-by: Alex Chan <alexc@tailscale.com>
Previously, running `add/remove/revoke-keys` without passing any keys
would fail with an unhelpful error:
```console
$ tailscale lock revoke-keys
generation of recovery AUM failed: sending generate-recovery-aum: 500 Internal Server Error: no provided key is currently trusted
```
or
```console
$ tailscale lock revoke-keys
generation of recovery AUM failed: sending generate-recovery-aum: 500 Internal Server Error: network-lock is not active
```
Now they fail with a more useful error:
```console
$ tailscale lock revoke-keys
missing argument, expected one or more tailnet lock keys
```
Fixes#19130
Change-Id: I9d81fe2f5b92a335854e71cbc6928e7e77e537e3
Signed-off-by: Alex Chan <alexc@tailscale.com>
* ipn/ipnlocal: warn incompatibility between no-snat-routes and exitnode
This commit adds a warning to health check when the --snat-subnet-routes=false flag for subnet router is
set alone side --advertise-exit-node=true. These two would conflict with each other and result internet-bound
traffic from peers using this exit node no masqueraded to the node's source IP and fail to route return
packets back. The described combination is not valid until we figure out a way to separate exitnode masquerade rule and skip it for subnet routes.
Updates #18725
Signed-off-by: KevinLiang10 <37811973+KevinLiang10@users.noreply.github.com>
* use date instead of for now to clarify effectivness
Signed-off-by: KevinLiang10 <37811973+KevinLiang10@users.noreply.github.com>
---------
Signed-off-by: KevinLiang10 <37811973+KevinLiang10@users.noreply.github.com>
By polling RTM_GETSTATS via netlink. RTM_GETSTATS is a relatively
efficient and targeted (single device) polling method available since
Linux v4.7.
The tundevstats "feature" can be extended to other platforms in the
future, and it's trivial to add new rtnl_link_stats64 counters on
Linux.
Updates tailscale/corp#38181
Signed-off-by: Jordan Whited <jordan@tailscale.com>
Replace byte-at-a-time ReadByte loops with Peek+Discard in the DERP
read path. Peek returns a slice into bufio's internal buffer without
allocating, and Discard advances the read pointer without copying.
Introduce util/bufiox with a BufferedReader interface and ReadFull
helper that uses Peek+copy+Discard as an allocation-free alternative
to io.ReadFull.
- derp.ReadFrameHeader: replace 5× ReadByte with Peek(5)+Discard(5),
reading the frame type and length directly from the peeked slice.
Remove now-unused readUint32 helper.
name old ns/op new ns/op speedup
ReadFrameHeader-8 24.2 12.4 ~2x
(0 allocs/op in both)
- key.NodePublic.ReadRawWithoutAllocating: replace 32× ReadByte with
bufiox.ReadFull. Addresses the "Dear future" comment about switching
away from byte-at-a-time reads once a non-escaping alternative exists.
name old ns/op new ns/op speedup
NodeReadRawWithoutAllocating-8 140 43.6 ~3.2x
(0 allocs/op in both)
- derpserver.handleFramePing: replace io.ReadFull with bufiox.ReadFull.
Updates tailscale/corp#38509
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
When we don't care about the payload value and are just checking whether
a set contains an IP/prefix, we can use `bart.Lite` for the same lookup
times but a lower memory footprint.
Fixes#19075
Change-Id: Ia709e8b718666cc61ea56eac1066467ae0b6e86c
Signed-off-by: Alex Chan <alexc@tailscale.com>
Rather than printing `unknown subcommand: drive` for any Taildrive
commands run in the macOS GUI, print an error message directing the user
to the GUI client and the docs page.
Updates #17210Fixes#18823
Change-Id: I6435007b5911baee79274b56e3ee101e6bb6d809
Signed-off-by: Alex Chan <alexc@tailscale.com>
When a client starts up without being able to connect to control, it
sends its discoKey to other nodes it wants to communicate with over
TSMP. This disco key will be a newer key than the one control knows
about.
If the client that can connect to control gets a full netmap, ensure
that the disco key for the node not connected to control is not
overwritten with the stale key control knows about.
This is implemented through keeping track of mapSession and use that for
the discokey injection if it is available. This ensures that we are not
constantly resetting the wireguard connection when getting the wrong
keys from control.
This is implemented as:
- If the key is received via TSMP:
- Set lastSeen for the peer to now()
- Set online for the peer to false
- When processing new keys, only accept keys where either:
- Peer is online
- lastSeen is newer than existing last seen
If mapSession is not available, as in we are not yet connected to
control, punt down the disco key injection to magicsock.
Ideally, we will want to have mapSession be long lived at some point in
the near future so we only need to inject keys in one location and then
also use that for testing and loading the cache, but that is a yak for
another PR.
Updates #12639
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Currently IP forwarding health check is done on sending MapRequests.
Move ip forwarding to the health service to gain the benefits
of the health tracker and perodic monitoring out of band from
the MapRequest path. ipnlocal now provides a closure to
the health service to provide the check if forwarding is broken.
Removed `skipIPForwardingCheck` from controlclient/direct.go,
it wasn't being used as the comments describe it, that check
has moved to ipnlocal for the closure to the health tracker.
Updates #18976
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Introduce a datapathHandler that implements hooks that will
receive packets from the tstun.Wrapper. This commit does not wire
those up just yet.
Perform DNAT from Magic IP to Transit IP on outbound flows on clients,
and reverse SNAT in the reverse direction.
Perform DNAT from Transit IP to final destination IP on outbound flows
on connectors, and reverse SNAT in the reverse direction.
Introduce FlowTable to cache validated flows by 5-tuple for fast lookups
after the first packet.
Flow expiration is not covered, and is intended as future work before
the feature is officially released.
Fixestailscale/corp#34249Fixestailscale/corp#35995
Co-authored-by: Fran Bull <fran@tailscale.com>
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
The e2e ingress test was very occasionally flaky. On looking at operator
logs from one failure, you can see the default ProxyClass was not ready
before the first reconcile loop for the exposed Service. The ProxyClass
became ready soon after, but no additional reconciles were triggered for
the exposed Service because we only triggered reconciles for Services
that explicitly named their ProxyClass.
This change adds additional list API calls for when it's the default
ProxyClass that's been updated in order to catch Services that use it by
default. It also adds indexes for the fields we need to search on to
ensure the list is efficient.
Fixestailscale/corp#37533
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Adds logic for containerboot to signal that it can't auth, so the
operator can reissue a new auth key. This only applies when running with
a config file and with a kube state store.
If the operator sees reissue_authkey in a state Secret, it will create a
new auth key iff the config has no auth key or its auth key matches the
value of reissue_authkey from the state Secret. This is to ensure we
don't reissue auth keys in a tight loop if the proxy is slow to start or
failing for some other reason. The reissue logic also uses a burstable
rate limiter to ensure there's no way a terminally misconfigured
or buggy operator can automatically generate new auth keys in a tight loop.
Additional implementation details (ChaosInTheCRD):
- Added `ipn.NotifyInitialHealthState` to ipn watcher, to ensure that
`n.Health` is populated when notify's are returned.
- on auth failure, containerboot:
- Disconnects from control server
- Sets reissue_authkey marker in state Secret with the failing key
- Polls config file for new auth key (10 minute timeout)
- Restarts after receiving new key to apply it
- modified operator's reissue logic slightly:
- Deletes old device from tailnet before creating new key
- Rate limiting: 1 key per 30s with initial burst equal to replica count
- In-flight tracking (authKeyReissuing map) prevents duplicate API calls
across reconcile loops
Updates #14080
Change-Id: I6982f8e741932a6891f2f48a2936f7f6a455317f
(cherry picked from commit 969927c47c3d4de05e90f5b26a6d8d931c5ceed4)
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
This makes tsnet apps not depend on x/crypto/ssh and locks that in with a test.
It also paves the wave for tsnet apps to opt-in to SSH support via a
blank feature import in the future.
Updates #12614
Change-Id: Ica85628f89c8f015413b074f5001b82b27c953a9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
* cmd/k8s-operator: use correct tailnet client for L7 & L3 ingresses
This commit fixes a bug when using multi-tailnet within the operator
to spin up L7 & L3 ingresses where the client used to create the
tailscale services was not switching depending on the tailnet used
by the proxygroup backing the service/ingress.
Updates: https://github.com/tailscale/corp/issues/34561
Signed-off-by: David Bond <davidsbond93@gmail.com>
* cmd/k8s-operator: adding server url to proxygroups when a custom tailnet has been specified
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
(cherry picked from commit 3b21ac5504e713e32dfcd43d9ee21e7e712ac200)
---------
Signed-off-by: David Bond <davidsbond93@gmail.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
After we intercept a DNS response and assign magic and transit addresses
we must communicate the assignment to our connector so that it can
direct traffic when it arrives.
Use the recently added peerapi endpoint to send the addresses.
Updates tailscale/corp#34258
Signed-off-by: Fran Bull <fran@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Fix its/it's, who's/whose, wether/whether, missing apostrophes
in contractions, and other misspellings across the codebase.
Updates #cleanup
Change-Id: I20453b81a7aceaa14ea2a551abba08a2e7f0a1d8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
considerable latency was seen when using k8s-proxy with ProxyGroup
in the kubernetes operator. Switching to L4 TCPForward solves this.
Fixes tailscale#18171
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds `--json` output mode to dns debug commands.
It defines structs for the data that is returned from:
`tailscale dns status` and `tailscale dns query <DOMAIN>` and
populates that as it runs the diagnostics.
When all the information is collected, it is serialised to JSON
or string built into an output and returned to the user.
The structs are defined and exported to golang consumers of this command
can use them for unmarshalling.
Updates #13326
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Remove the TS_EXPERIMENTAL_KUBE_API_EVENTS env var from the operator and its
helm chart. This has already been marked as deprecated, and has been
scheduled to be removed in release 1.96.
Add a check in helm chart to fail if the removed variable is set to true,
prompting users to move to ACLs instead.
Fixes: #18875
Signed-off-by: Becky Pauley <becky@tailscale.com>
PR #18860 adds firewall rules in the mangle table to save outbound packet
marks to conntrack and restore them on reply packets before the routing
decision. When reply packets have their marks restored, the kernel uses
the correct routing table (based on the mark) and the packets pass the
rp_filter check.
This makes the risk check and reverse path filtering warnings unnecessary.
Updates #3310Fixestailscale/corp#37846
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Using the new wait command from #18574 provide a tailscale-online.target
that has a similar usage model to the conventional
`network-online.target`.
Updates #3340
Updates #11504
Signed-off-by: James Tucker <james@tailscale.com>
Adds freedesktop as an option for installing autostart desktop files for
starting the systray application.
Fixes#18766
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
* cmd/k8s-operator/deploy/examples
Adds exitnode.yaml to k8s-operator
Fixes#18086
Signed-off-by: Christopher Mosetick <office@cpm.is>
* cmd/k8s-operator/deploy/examples: update connector and add exitnode examples
- Remove exitNode: true from connector.yaml to keep it focused as a subnet router example
- Update connector.yaml header comment to remove exit node reference and add pointer hint to exitnode.yaml
- Clarify exitnode.yaml comments to accurately describe separate Connector deployment pattern
Fixes#18086
Signed-off-by: Christopher Mosetick <office@cpm.is>
* Update cmd/k8s-operator/deploy/examples/exitnode.yaml
Co-authored-by: David Bond <davidsbond@users.noreply.github.com>
Signed-off-by: Chris Mosetick <cmosetick@gmail.com>
* Update cmd/k8s-operator/deploy/examples/exitnode.yaml
Co-authored-by: David Bond <davidsbond@users.noreply.github.com>
Signed-off-by: Chris Mosetick <cmosetick@gmail.com>
* Update cmd/k8s-operator/deploy/examples/exitnode.yaml
Co-authored-by: David Bond <davidsbond@users.noreply.github.com>
Signed-off-by: Chris Mosetick <cmosetick@gmail.com>
* Update cmd/k8s-operator/deploy/examples/exitnode.yaml
Co-authored-by: David Bond <davidsbond@users.noreply.github.com>
Signed-off-by: Chris Mosetick <cmosetick@gmail.com>
---------
Signed-off-by: Christopher Mosetick <office@cpm.is>
Signed-off-by: Chris Mosetick <cmosetick@gmail.com>
Co-authored-by: David Bond <davidsbond@users.noreply.github.com>
* cmd/containerboot,kube/services: support the ability to automatically advertise services on startup
Updates #17769
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/containerboot: don't assume we want to use kube state store if in kubernetes
Fixes#8188
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
The new version of app connector (conn25) needs to read DNS responses
for domains it is interested in and store and swap out IP addresses.
Add a hook to dns manager to enable this.
Give the conn25 updated netmaps so that it knows when to assign
connecting addresses and from what pool.
Assign an address when we see a DNS response for a domain we are
interested in, but don't do anything with the address yet.
Updates tailscale/corp#34252
Signed-off-by: Fran Bull <fran@tailscale.com>
Add more explicit `--bind-address` and `--bind-port` flags to the `tailscale netcheck` CLI to give users control over UDP probes' source IP and UDP port.
This was already supported in a less documented manner via the` TS_DEBUG_NETCHECK_UDP_BIND` environment variable. The environment variable reference is preserved and used as a fallback value in the absence of these new CLI flags.
Updates tailscale/corp#36833
Signed-off-by: Amal Bansode <amal@tailscale.com>
--auth is already best-effort, but we saw some CI failures due to
failing to fetch stats when cigocached was overwhelmed recently. Make
sure it fails more gracefully in the absence of cigocached like the rest
of cigocacher already does.
Updates tailscale/corp#37059
Change-Id: I0703b30b1c5a7f8c649879a87e6bcd2278610208
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Stop stripping the "*." prefix from wildcard domains when used
as storage keys. Instead, replace "*" with "wildcard_" only at
the filesystem boundary in certFile and keyFile. This prevents
wildcard and non-wildcard certs from colliding in storage.
Updates #1196
Updates #7081
Signed-off-by: Fernando Serboncini <fserb@tailscale.com>
This commit implements a reconciler for the new `ProxyGroupPolicy`
custom resource. When created, all `ProxyGroupPolicy` resources
within the same namespace are merged into two `ValidatingAdmissionPolicy`
resources, one for egress and one for ingress.
These policies use CEL expressions to limit the usage of the
"tailscale.com/proxy-group" annotation on `Service` and `Ingress`
resources on create & update.
Included here is also a new e2e test that ensures that resources that
violate the policy return an error on creation, and that once the
policy is changed to allow them they can be created.
Closes: https://github.com/tailscale/corp/issues/36830
Signed-off-by: David Bond <davidsbond93@gmail.com>
Mike O'Driscoll
Brad Fitzpatrick
Kristoffer Dalby
M. J. Fromberger
Naman Sood
BeckyPauley
Alex Chan
KevinLiang10
Jordan Whited
Claus Lensbøl
Michael Ben-Ami
Tom Proctor
David Bond
Fran Bull
Raj Singh
James Tucker
Chris Mosetick
Tom Meadows
Amal Bansode
dependabot[bot]
Fernando Serboncini