Author SHA1 Message Date
codingetandClaude 6893723ccf fix(wasm): correct ICMP case in ping type error message
The constant tailcfg.PingICMP is "ICMP" not "icmp"; the error message
was listing the wrong string, causing user confusion about valid values.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 18:01:34 +00:00
codingetandClaude c2ddadca72 fix(wasm): validate ping type early; fallback DNS resolver for exit node
Add a switch guard before the 30-second context in ping() so that invalid
ping type strings (e.g. "disco" vs "Disco") reject immediately with a clear
error rather than silently timing out because userspaceEngine.Ping has no
default case.

For queryDNS(), detect SERVFAIL responses returned with an empty resolver
list (the typical state when an exit node is active but the DNS manager
forwarder has no configured upstreams) and fall back to querying 8.8.8.8
via the dialer — which honours exit-node routing — for A/AAAA record types.
Fall further back to the browser's native resolver if UserDial fails.

Also accept bare IP addresses in whoIs() (in addition to ip:port) so
callers don't need to fabricate a port when they only have a peer IP.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 18:01:34 +00:00
codingetandClaude 453261aef0 feat(tsconnect): add peerAPIURL to netmap and localAPI in-process bridge
Include the PeerAPI base URL (http://ip:port) in every node entry of the
notifyNetMap payload — for self via LocalBackend.GetPeerAPIPort, for peers
by reading the PeerAPI4/PeerAPI6 Services entries in their Hostinfo. The URL
mirrors the address-family preference used by peerAPIBase (prefer IPv4).

Add a localAPI(method, path, body?) WASM binding that dispatches in-process
HTTP requests directly to a LocalAPI handler with full read/write/cert
permissions, returning {status, body}. Enables TypeScript callers to access
any LocalAPI endpoint (ACL policy, Taildrive shares, etc.) without network
setup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 18:01:34 +00:00
codingetandClaude bd124abc3c feat(tsconnect): add whoIs, queryDNS, ping, suggestExitNode WASM bindings
Expose four LocalBackend capabilities to JavaScript:
- whoIs(addrPort, proto?): resolves a connecting ip:port to a tailnet node
  and user profile; returns null for unknown peers
- queryDNS(name, type?): queries the tailnet DNS resolver (MagicDNS +
  upstream); parses A/AAAA/CNAME/TXT answers into strings
- ping(ip, type?, size?): pings a tailnet peer (TSMP, disco, ICMP, peerapi)
  with a 30 s context timeout; returns latency and path details
- suggestExitNode(): asks the coordination server for the best exit node

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude 9fd2f3bbf4 feat(tsconnect): add getCert, listenTLS, setFunnel + fix TLS cert for WASM
Enable ACME TLS certificates on js/wasm by dropping the !js build tag from
cert.go and routing storage through the state store. Add getCert, listenTLS,
and setFunnel WASM bindings with a combinedTLSListener that merges Funnel
ingress and direct tailnet connections. Notify the control plane immediately
after serve config changes to accelerate Funnel DNS provisioning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude a6b286b414 fix(tsconnect): pin types to avoid monorepo @types pollution
Replace skipLibCheck with an explicit types list so TypeScript and
dts-bundle-generator only auto-include @types/golang-wasm-exec and
@types/qrcode, preventing @types/eslint-scope and @types/ws from
leaking in from a parent node_modules when built inside a monorepo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude bc9884ce69 fix(tsconnect): skipLibCheck to avoid monorepo @types conflicts
When tsconnect is built inside a JS monorepo, TypeScript walks up the
directory tree and auto-discovers @types/eslint-scope and @types/ws
from the root node_modules, causing spurious type errors unrelated to
tsconnect itself. skipLibCheck suppresses these.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude 3f52ae7be2 fix(tsconnect): lowercase name/size in waitingFiles JSON
apitype.WaitingFile has no json tags so it serialised as {Name, Size}.
Introduce a local jsWaitingFile struct with json:"name" / json:"size"
so the JS side receives idiomatic camelCase property names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude fbc7982e01 fix(taildrop): restore incoming file progress notifications
The io.Copy in PutFile was writing directly to wc, bypassing the
incomingFile wrapper whose Write method increments f.copied and fires
a throttled sendFileNotify on progress. As a result, notifyIncomingFiles
on the JS side only ever fired once (on completion) with received=0,
making progress UI impossible. The original inFile wrapping was lost
during the Android SAF refactor.

Also surface the PartialFile.Done flag through jsIncomingFile so JS can
distinguish the final "transfer complete" notification from in-progress
updates.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude c4a2eb3451 fix(tsconnect): guard nil n.Prefs in notify callback
n.Prefs is *PrefsView (a pointer), so calling n.Prefs.Valid() on a
Notify where Prefs is nil auto-dereferenced nil and panicked. The
callback's defer recover() swallowed the panic, which meant every
Notify without Prefs (Health-only, FilesWaiting, IncomingFiles,
OutgoingFiles, etc.) never reached the file-related JS calls.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude 705eebe5fc feat(tsconnect): add outgoing file transfer progress notifications
- Export UpdateOutgoingFiles on taildrop.Extension so it can be called
  from outside the package (wasm bridge, package main).
- Wrap sendFile's PUT body with progresstracking.NewReader so bytes-sent
  is sampled roughly once per second during transfer.
- Create an OutgoingFile entry (with UUID, peer ID, name, declared size)
  before the PUT and call UpdateOutgoingFiles on each progress tick and
  on completion (setting Finished/Succeeded). This flows into the IPN
  notify stream as OutgoingFiles notifications.
- Add jsOutgoingFile struct and wire n.OutgoingFiles into a new
  notifyOutgoingFiles callback in run(), mirroring notifyIncomingFiles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude 4ef06f2498 feat(tsconnect): add notifyFilesWaiting and notifyIncomingFiles callbacks
Wire two new callbacks into the IPN notify stream:

- notifyFilesWaiting: fires when a completed inbound transfer is staged
  and ready to retrieve via waitingFiles(). Triggered by n.FilesWaiting
  in the notify stream.
- notifyIncomingFiles: fires with a JSON snapshot of in-progress inbound
  transfers whenever progress changes (roughly once per second while
  active, plus once at completion). The jsIncomingFile struct carries
  name, started (Unix ms), declaredSize, and received bytes. An empty
  array indicates all active transfers have finished.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude bdfcc55797 feat(taildrop): fix DirectFileMode, void callbacks, and empty WaitingFiles
- Add SetStagedFileOps to Extension: sets fileOps without enabling
  DirectFileMode, so WASM clients use staged retrieval (WaitingFiles,
  OpenFile, DeleteFile) instead of direct-write mode.
- Add directFileOps bool field: SetFileOps (Android SAF) sets it true;
  SetStagedFileOps (WASM JS) leaves it false. onChangeProfile now uses
  `fops != nil && e.directFileOps` to determine DirectFileMode.
- Add jsCallVoid to jsFileOps: void ops (openWriter, write, closeWriter,
  remove) now use cb(err?: string) instead of cb(null, err: string).
- Fix waitingFiles() returning JSON null when no files are waiting:
  normalise nil slice to empty slice before marshalling.
- Update wireTaildropFileOps to call SetStagedFileOps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codingetandClaude 2ddaf2f5aa feat(tsconnect): expose exit node selection to JS
Add exit node support to the wasm JS bridge:

- Include `exitNodeOption` and `stableNodeID` on each peer in the
  notifyNetMap payload so callers can identify which peers are exit
  nodes and reference them by stable ID.
- Call `notifyExitNode(stableNodeID)` whenever prefs change, so
  callers can track which exit node (if any) is currently active.
- Expose `setExitNode(stableNodeID)` — sets ExitNodeID via EditPrefs.
- Expose `setExitNodeEnabled(enabled)` — toggles the last-used exit
  node on/off via SetUseExitNodeEnabled.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:56:31 +00:00
codinget 8357137a59 feat(tsconnect): add TCP listening to ipn.listen
Extend ipn.listen to also accept "tcp"/"tcp4"/"tcp6" and return a
TCPListener bound to a netstack gonet.TCPListener. The listener
exposes accept/close/addr like a Go net.Listener and additionally
implements Symbol.asyncIterator so JS callers can write:

  for await (const conn of listener) { ... }

The async iterator returns done when the listener is closed (via
errors.Is(net.ErrClosed)) and rejects on any other accept error.
Symbol-keyed properties are set via Reflect.set since syscall/js
only exposes string-keyed Set.
2026-05-18 01:18:09 +00:00
codinget 4acd937b0f feat(tsconnect): expose dialTLS to JS
Add ipn.dialTLS(addr, opts?) which dials a TCP connection through
the Tailscale dialer and performs a TLS handshake on top, returning
a JS Conn just like ipn.dial.

WASM has no system root pool, so verification defaults to the
baked-in LetsEncrypt ISRG roots already linked via net/bakedroots.
That covers any tailnet HTTPS endpoint provisioned via
`tailscale cert`. Callers can override with opts.caCerts (PEM) or
bypass entirely with opts.insecureSkipVerify, and override SNI with
opts.serverName.

Marginal binary cost is ~10 KiB on top of the existing ~31.6 MiB
wasm: crypto/tls and the x509 verification path are already pulled
in by control/controlclient and net/tlsdial.
2026-05-18 01:18:09 +00:00
codinget 301137edc4 feat(tsconnect): expose dial, listen and listenICMP to JS
Wire up the userspace networking primitives to the JS bridge so
browser callers can initiate outbound and receive inbound traffic
over the Tailscale network:

- ipn.dial(network, addr) wraps a tsdial UserDial into a JS Conn
  with read/write/close/localAddr/remoteAddr.
- ipn.listen(network, addr) wraps a netstack ListenPacket into a
  JS PacketConn with readFrom/writeTo/close/localAddr.
- ipn.listenICMP("icmp4"|"icmp6"|"icmp") creates a raw ICMP
  endpoint on the underlying gVisor stack and wraps it as a
  PacketConn for sending/receiving ping traffic.

To support listenICMP, netstack.Impl gains a Stack() accessor that
returns the underlying *stack.Stack so jsIPN can call NewEndpoint
with icmp.ProtocolNumber4/6.

Binary I/O uses js.CopyBytesToGo / js.CopyBytesToJS to move bytes
across the syscall/js boundary without base64 round-trips.
2026-05-18 01:18:09 +00:00
codinget dec913b1e3 fix(tsconnect): drop nethttpomithttp2 build tag
After 1d93bdce2 ("control/controlclient: remove x/net/http2, use
net/http"), the noise control client uses net/http's Transport with
Protocols.SetUnencryptedHTTP2(true). The nethttpomithttp2 build tag
strips the bundled HTTP/2 implementation from net/http, so at runtime
the control client fails the first register request with "http:
Transport does not support unencrypted HTTP/2" and the wasm never
connects.

Drop the tag so the bundled HTTP/2 ships in the wasm binary.
2026-05-18 01:18:09 +00:00
775 changed files with 10879 additions and 64931 deletions
+2 -60
View File
@@ -1,60 +1,2 @@
go.mod filter=go-mod eol=lf text
*.go diff=golang eol=lf text
*.adml eol=lf text
*.admx eol=lf text
*.bash eol=lf text
*.c eol=lf text
*.cgi eol=lf text
*.conf eol=lf text
*.css eol=lf text
*.csv eol=lf text
*.desktop eol=lf text
*.fish eol=lf text
*.gitattributes eol=lf text
*.gitignore eol=lf text
*.gitkeep eol=lf text
*.go eol=lf text
*.h eol=lf text
*.helmignore eol=lf text
*.htaccess eol=lf text
*.html eol=lf text
*.hujson eol=lf text
*.in eol=lf text
*.init eol=lf text
*.js eol=lf text
*.json eol=lf text
*.lock eol=lf text
*.lua eol=lf text
*.md eol=lf text
*.mod eol=lf text
*.nix eol=lf text
*.openrc eol=lf text
*.pbxproj eol=lf text
*.pem eol=lf text
*.plg eol=lf text
*.plist eol=lf text
*.rc eol=lf text
*.resolved eol=lf text
*.rev eol=lf text
*.rs eol=lf text
*.sc eol=lf text
*.service eol=lf text
*.sh eol=lf text
*.socket eol=lf text
*.stignore eol=lf text
*.sum eol=lf text
*.svg eol=lf text
*.swift eol=lf text
*.tmpl eol=lf text
*.toml eol=lf text
*.ts eol=lf text
*.tsx eol=lf text
*.txt eol=lf text
*.version eol=lf text
*.xcscheme eol=lf text
*.xcsettings eol=lf text
*.xib eol=lf text
*.xml eol=lf text
*.yaml eol=lf text
*.yml eol=lf text
*.zsh eol=lf text
go.mod filter=go-mod
*.go diff=golang
+1 -1
View File
@@ -49,7 +49,7 @@ jobs:
# Install a more recent Go that understands modern go.mod content.
- name: Install Go
uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # zizmor: ignore[cache-poisoning] v6.3.0
uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
with:
go-version-file: go.mod
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install govulncheck
run: ./tool/go install golang.org/x/vuln/cmd/govulncheck@0782b76014f15f24e22a438f30f308df42899ba1 # 1.3.0
run: ./tool/go install golang.org/x/vuln/cmd/govulncheck@latest
- name: Scan source code for known vulnerabilities
run: PATH=$PWD/tool/:$PATH "$(./tool/go env GOPATH)/bin/govulncheck" -test ./...
+1 -1
View File
@@ -69,7 +69,7 @@ jobs:
- { image: "fedora:latest", deps: "curl", version: "1.80.0" }
runs-on: ubuntu-latest
container:
image: ${{ matrix.image }} # zizmor: ignore[unpinned-images]
image: ${{ matrix.image }}
options: --user root
steps:
- name: install dependencies (pacman)
+6 -6
View File
@@ -102,15 +102,15 @@ jobs:
# single-test-per-matrix-job model. They stay runnable locally.
run: |
set -euo pipefail
exclude='^(TestGrid|TestVnetPerf.*)$'
exclude='^(TestGrid)$'
tmp=$(mktemp)
for pkg_dir in tstest/natlab/vmtest tstest/integration/nat; do
pkg="./${pkg_dir}/"
for f in "${pkg_dir}"/*_test.go; do
[ -e "$f" ] || continue
{ grep -hE '^func Test[A-Z][A-Za-z0-9_]*\(t \*testing\.T\)' "$f" || true; } \
grep -hE '^func Test[A-Z][A-Za-z0-9_]*\(t \*testing\.T\)' "$f" \
| sed -E 's/^func (Test[A-Za-z0-9_]+).*/\1/' \
| { grep -vE "$exclude" || true; } \
| grep -vE "$exclude" \
| while read -r t; do
jq -nc --arg pkg "$pkg" --arg test "$t" \
'{pkg: $pkg, test: $test}' >> "$tmp"
@@ -165,13 +165,13 @@ jobs:
key: natlab-gokrazy-${{ github.sha }}
# The gokrazy-based tests boot the kernel directly from
# vmlinuz that ships in the gokrazy/kernel.amd64 module.
# vmlinuz that ships in the tailscale/gokrazy-kernel module.
# Tests look it up under GOMODCACHE via findKernelPath, so the
# module has to be present even though no Go source imports it
# in the test package itself.
- name: Download kernel.amd64 module
- name: Download gokrazy-kernel module
run: |
./tool/go mod download github.com/gokrazy/kernel.amd64
./tool/go mod download github.com/tailscale/gokrazy-kernel
- name: Run ${{ matrix.test }}
# Per-test timeout is well above the few-minute typical runtime
-45
View File
@@ -1,45 +0,0 @@
name: policybot-test
env:
HOME: ${{ github.workspace }}
GOMODCACHE: ${{ github.workspace }}/gomodcache
CMD_GO_USE_GIT_HASH: "true"
on:
push:
branches:
- main
- "release-branch/*"
paths:
- .github/workflows/policybot-test.yml
- .policy.yml
- .policy-tests.yml
- go.mod
pull_request:
paths:
- .github/workflows/policybot-test.yml
- .policy.yml
- .policy-tests.yml
- go.mod
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
policybot-test:
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Check out code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: src
# The version of github.com/tailscale/policybottest used here is
# pinned by go.mod via internal/tooldeps/tooldeps.go; bump it with
# "go get github.com/tailscale/policybottest@<sha> && go mod tidy".
- name: Run policy tests
working-directory: src
run: ./tool/go run github.com/tailscale/policybottest -policy .policy.yml -tests .policy-tests.yml
@@ -2,7 +2,7 @@ name: request-dataplane-review
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
types: [ opened, synchronize, reopened, ready_for_review ]
paths:
- ".github/workflows/request-dataplane-review.yml"
- "**/*derp*"
@@ -15,6 +15,8 @@ jobs:
name: Request Dataplane Review
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Get access token
uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
id: generate-token
@@ -22,8 +24,6 @@ jobs:
# Get token for app: https://github.com/apps/change-visibility-bot
app-id: ${{ secrets.VISIBILITY_BOT_APP_ID }}
private-key: ${{ secrets.VISIBILITY_BOT_APP_PRIVATE_KEY }}
# Limit the token to only requesting reviewers on pull requests.
permission-pull-requests: write
- name: Add reviewers
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
-39
View File
@@ -1,39 +0,0 @@
name: request-k8s-review
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- ".github/workflows/request-k8s-review.yml"
- "k8s-operator/**"
- "kube/**"
- "cmd/k8s-operator/**"
- "cmd/k8s-proxy/**"
- "cmd/k8s-nameserver/**"
- "cmd/containerboot/**"
- "cmd/sync-containers/**"
- "ipn/store/kubestore/**"
- "docs/k8s/**"
- "!**/depaware.txt"
jobs:
request-k8s-review:
if: github.event.pull_request.draft == false
name: Request K8s Review
runs-on: ubuntu-latest
steps:
- name: Get access token
uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
id: generate-token
with:
# Get token for app: https://github.com/apps/change-visibility-bot
app-id: ${{ secrets.VISIBILITY_BOT_APP_ID }}
private-key: ${{ secrets.VISIBILITY_BOT_APP_PRIVATE_KEY }}
# Limit the token to only requesting reviewers on pull requests.
permission-pull-requests: write
- name: Add reviewers
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
url: ${{ github.event.pull_request.html_url }}
run: |
gh pr edit "$url" --add-reviewer tailscale/k8s-devs
+31 -13
View File
@@ -70,7 +70,7 @@ jobs:
run: go mod download
- name: Cache Go modules
if: steps.check-cache.outputs.cache-hit != 'true'
uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # zizmor: ignore[cache-poisoning] v5.0.4
uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
path: gomodcache # relative to workspace; see env note at top of file
key: ${{ steps.hash.outputs.key }}
@@ -183,7 +183,7 @@ jobs:
TS_TEST_SHARD: ${{ matrix.shard }}
- name: bench all
working-directory: src
run: ./tool/go test ${{matrix.buildflags}} -bench=. -benchtime=1x -run='^$' $(for x in $(git grep -l '^func Benchmark' | xargs dirname | sort | uniq); do echo "./$x"; done)
run: ./tool/go test ${{matrix.buildflags}} -bench=. -benchtime=1x -run=^$ $(for x in $(git grep -l "^func Benchmark" | xargs dirname | sort | uniq); do echo "./$x"; done)
env:
GOARCH: ${{ matrix.goarch }}
- name: check that no tracked files changed
@@ -261,7 +261,6 @@ jobs:
cigocached-host: ${{ vars.CIGOCACHED_AZURE_HOST }}
- name: test
shell: bash
if: matrix.key != 'win-bench' # skip on bench builder
working-directory: src
run: ./tool/go run ./cmd/testwrapper sharded:${{ matrix.shard }}
@@ -269,10 +268,9 @@ jobs:
NOPWSHDEBUG: "true" # to quiet tool/gocross/gocross-wrapper.ps1 in CI
- name: bench all
shell: bash
if: matrix.key == 'win-bench'
working-directory: src
run: ./tool/go test -bench=. -benchtime=1x -run='^$' $(for x in $(git grep -l '^func Benchmark' | xargs dirname | sort | uniq); do echo "./$x"; done)
run: ./tool/go test ./... -bench=. -benchtime=1x -run="^$"
env:
NOPWSHDEBUG: "true" # to quiet tool/gocross/gocross-wrapper.ps1 in CI
@@ -345,7 +343,7 @@ jobs:
needs: gomod-cache
runs-on: ubuntu-24.04
container:
image: golang:latest # zizmor: ignore[unpinned-images]
image: golang:latest
options: --privileged
steps:
- name: checkout
@@ -365,6 +363,30 @@ jobs:
working-directory: src
run: ./tool/go test $(./tool/go run ./tool/listpkgs --has-root-tests)
vm:
needs: gomod-cache
runs-on: ["self-hosted", "linux", "vm"]
# VM tests run with some privileges, don't let them run on 3p PRs.
if: github.repository == 'tailscale/tailscale'
steps:
- name: checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: src
- name: Restore Go module cache
uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
path: gomodcache
key: ${{ needs.gomod-cache.outputs.cache-key }}
enableCrossOsArchive: true
- name: Run VM tests
working-directory: src
run: ./tool/go test ./tstest/integration/vms -v -no-s3 -run-vm-tests -run=TestRunUbuntu2404
env:
HOME: "/var/lib/ghrunner/home"
TMPDIR: "/tmp"
XDG_CACHE_HOME: "/var/lib/ghrunner/cache"
cross: # cross-compile checks, build only.
needs: gomod-cache
strategy:
@@ -620,13 +642,6 @@ jobs:
run: |
./tool/go run ./cmd/tsconnect --fast-compression build
./tool/go run ./cmd/tsconnect --fast-compression build-pkg
- name: verify Google Chrome is available
run: |
which google-chrome
google-chrome --version
- name: tsconnect js/wasm headless-browser tests
working-directory: src
run: ./tool/go test ./tstest/integration/jswasmtest/ -v -timeout 180s --run-headless-browser-tests
- name: Tidy cache
working-directory: src
shell: bash
@@ -888,6 +903,7 @@ jobs:
- test
- windows
- macos
- vm
- cross
- ios
- wasm
@@ -933,6 +949,7 @@ jobs:
- test
- windows
- macos
- vm
- cross
- ios
- wasm
@@ -982,6 +999,7 @@ jobs:
- test
- windows
- macos
- vm
- wasm
- fuzz
- race-root-integration
-3
View File
@@ -33,9 +33,6 @@ jobs:
# Get token for app: https://github.com/apps/tailscale-code-updater
app-id: ${{ secrets.CODE_UPDATER_APP_ID }}
private-key: ${{ secrets.CODE_UPDATER_APP_PRIVATE_KEY }}
# Limit the token to only pushing a branch and opening a pull request.
permission-contents: write
permission-pull-requests: write
- name: Send pull request
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0 #v8.1.0
@@ -29,9 +29,6 @@ jobs:
# Get token for app: https://github.com/apps/tailscale-code-updater
app-id: ${{ secrets.CODE_UPDATER_APP_ID }}
private-key: ${{ secrets.CODE_UPDATER_APP_PRIVATE_KEY }}
# Limit the token to only pushing a branch and opening a pull request.
permission-contents: write
permission-pull-requests: write
- name: Send pull request
id: pull-request
+2 -4
View File
@@ -14,17 +14,15 @@ on:
- main
- "release-branch/*"
paths:
- .github/workflows/vet.yml
- "**.go"
pull_request:
paths:
- .github/workflows/vet.yml
- "**.go"
jobs:
vet:
runs-on: ubuntu-24.04
timeout-minutes: 10
runs-on: [ self-hosted, linux ]
timeout-minutes: 5
steps:
- name: Check out code
-32
View File
@@ -1,32 +0,0 @@
name: security lint GitHub Actions with zizmor
on:
push:
branches: ["main"]
paths:
- ".github/workflows/**"
pull_request:
branches: ["**"]
paths:
- ".github/workflows/**"
permissions: {}
jobs:
zizmor:
runs-on: ubuntu-latest
permissions:
contents: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@5f14fd08f7cf1cb1609c1e344975f152c7ee938d # v0.5.6
with:
min-severity: high
advanced-security: false
annotations: true
-2
View File
@@ -58,5 +58,3 @@ client/web/build/assets
# Ignore syncthing state directory.
/.stfolder
fbstatus
gafpush
-258
View File
@@ -1,258 +0,0 @@
# Tests for testdata/tailscale.com.policy.yml.
#
# Run from the parent directory with:
# go run . -policy testdata/tailscale.com.policy.yml \
# -tests testdata/tailscale.com.policy-test.yml
teams:
# Members are picked solely so tests below can refer to them.
# `alice` and `bob` are control-protocol-owners *and* dev. `carol`,
# `dave`, and `eve` are dev only. `outsider` is in nothing.
tailscale/control-protocol-owners:
- alice
- bob
tailscale/dev:
- alice
- bob
- carol
- dave
- eve
tests:
# ------------------------------------------------------------------
# Baseline: every PR (touching tailcfg/ or not) requires a +1 from a
# tailscale/dev member.
# ------------------------------------------------------------------
- name: non-tailcfg PR with no review is pending
pull_request:
author: carol
changed_files:
- README.md
- cmd/tailscale/main.go
expect:
status: pending
rules:
"tailcfg changes approved by control-protocol-owners": skipped
"tailcfg changes overridden by another tailscale/dev": skipped
"any tailscale/dev review": pending
- name: non-tailcfg PR with dev review is approved
pull_request:
author: carol
changed_files:
- README.md
reviews:
- user: dave
state: approved
expect:
status: approved
rules:
"any tailscale/dev review": approved
- name: non-tailcfg PR with thumbs-up comment from dev is approved
pull_request:
author: carol
changed_files:
- README.md
comments:
- user: dave
body: ":+1:"
expect:
status: approved
rules:
"any tailscale/dev review": approved
- name: non-tailcfg PR with non-dev review is pending
pull_request:
author: carol
changed_files:
- README.md
reviews:
- user: outsider
state: approved
expect:
status: pending
rules:
"any tailscale/dev review": pending
- name: PR with no files at all is pending without a dev review
pull_request:
author: carol
changed_files: []
expect:
status: pending
rules:
"any tailscale/dev review": pending
# ------------------------------------------------------------------
# tailcfg/ requires BOTH a control-protocol-owners review (or an
# override comment) AND a dev +1. Owners are in tailscale/dev too,
# so their single review satisfies both gates.
# ------------------------------------------------------------------
- name: tailcfg change without anything is pending
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
expect:
status: pending
rules:
"tailcfg changes approved by control-protocol-owners": pending
"tailcfg changes overridden by another tailscale/dev": pending
"any tailscale/dev review": pending
- name: a single owner review approves both the owner rule and the dev rule
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
reviews:
- user: alice
state: approved
expect:
status: approved
rules:
"tailcfg changes approved by control-protocol-owners": approved
"any tailscale/dev review": approved
- name: owner cannot self-approve their own tailcfg PR
pull_request:
author: alice
changed_files:
- tailcfg/tailcfg.go
reviews:
- user: alice
state: approved
expect:
status: pending
- name: non-owner dev review alone leaves the tailcfg gate pending
pull_request:
author: alice
changed_files:
- tailcfg/tailcfg.go
reviews:
- user: carol
state: approved
expect:
status: pending
rules:
"tailcfg changes approved by control-protocol-owners": pending
"tailcfg changes overridden by another tailscale/dev": pending
"any tailscale/dev review": approved
# ------------------------------------------------------------------
# policybot-override flow on a tailcfg/ PR.
# ------------------------------------------------------------------
# An override comment by itself only satisfies the override rule.
# The baseline dev-review rule still needs a separate +1.
- name: override comment alone leaves baseline dev review pending
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: dave
body: "policybot-override: emergency rollback"
expect:
status: pending
rules:
"tailcfg changes overridden by another tailscale/dev": approved
"any tailscale/dev review": pending
- name: override comment plus a separate dev review approves
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: dave
body: "policybot-override: emergency rollback"
reviews:
- user: eve
state: approved
expect:
status: approved
# One dev doing BOTH (post the override AND a regular +1) works too.
- name: same dev posts both the override and a +1 review
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: dave
body: "policybot-override: shipping a typo fix"
reviews:
- user: dave
state: approved
expect:
status: approved
- name: override comment from the author does not approve the override
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: carol
body: "policybot-override: please let me ship this"
expect:
status: pending
rules:
"tailcfg changes overridden by another tailscale/dev": pending
- name: override comment from a non-dev does not approve the override
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: outsider
body: "policybot-override: I am not on the dev team"
expect:
rules:
"tailcfg changes overridden by another tailscale/dev": pending
- name: empty-reason policybot-override does not approve
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: dave
body: "policybot-override:"
expect:
rules:
"tailcfg changes overridden by another tailscale/dev": pending
# Defaults regression guards: a normal review approval from a dev
# member must NOT silently approve the override rule. Same for a
# plain :+1: comment. Without the explicit `methods: github_review:
# false` / `comments: []` on the override rule, both would.
- name: a regular dev review does not silently satisfy the override rule
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
reviews:
- user: dave
state: approved
expect:
rules:
"tailcfg changes overridden by another tailscale/dev": pending
- name: a thumbs-up comment from a dev does not silently satisfy the override rule
pull_request:
author: carol
changed_files:
- tailcfg/tailcfg.go
comments:
- user: dave
body: ":+1:"
expect:
rules:
"tailcfg changes overridden by another tailscale/dev": pending
-84
View File
@@ -1,84 +0,0 @@
# Approval policy for this repository, enforced by policy-bot
# (https://github.com/palantir/policy-bot) running at
# https://policybot.corp.ts.net.
#
# This file replaces the role GitHub's CODEOWNERS played: when a pull
# request touches a path covered by a rule below, policy-bot posts a
# status check that blocks merging until the required reviewers approve.
#
# Policy and rule syntax reference:
# https://github.com/palantir/policy-bot/blob/develop/README.md
# Example policy files (team-approval, disapproval, remote, etc.):
# https://github.com/palantir/policy-bot/tree/develop/config/policy-examples
#
# Do not add to this policy without wide discussion.
# See https://github.com/tailscale/corp/issues/13972.
policy:
approval:
# tailcfg/ has an extra gate: either a control-protocol-owners
# review or an explicit policybot-override: comment from a dev.
# The block is skipped on PRs that don't touch tailcfg/.
- or:
- tailcfg changes approved by control-protocol-owners
- tailcfg changes overridden by another tailscale/dev
# And every PR, regardless of files, needs a +1 from a dev.
# This rule also keeps the policy from collapsing into "all rules
# skipped" (which policy-bot treats as a failure) on non-tailcfg
# PRs.
- any tailscale/dev review
approval_rules:
- name: tailcfg changes approved by control-protocol-owners
if:
changed_files:
paths:
- "^tailcfg/"
requires:
count: 1
teams:
- "tailscale/control-protocol-owners"
- name: tailcfg changes overridden by another tailscale/dev
description: |
Any member of @tailscale/dev (other than the PR author) can
override the control-protocol-owners requirement by leaving a
comment of the form
policybot-override: <reason>
on the pull request. The reason can be anything but should
explain why the override is appropriate; it stays in the PR
conversation as a record. The override comment also counts as
that developer's approval.
if:
changed_files:
paths:
- "^tailcfg/"
requires:
count: 1
teams:
- "tailscale/dev"
options:
methods:
# Explicitly turn off the defaults (github_review: true,
# comments: [":+1:", "👍"]) so the ONLY way to satisfy this
# rule is a "policybot-override:" comment. Otherwise a normal
# review approval or thumbs-up from any tailscale/dev member
# would silently pass the rule.
github_review: false
comments: []
comment_patterns:
- '^policybot-override: \S.*'
- name: any tailscale/dev review
description: |
Every PR needs at least one approval from a member of
@tailscale/dev. policy-bot's default approval methods count
a GitHub review approval, a ":+1:" comment, or a "👍"
comment as approval. The PR author cannot approve their
own PR.
requires:
count: 1
teams:
- "tailscale/dev"
+1 -7
View File
@@ -1,7 +1 @@
# This repository does NOT use GitHub's CODEOWNERS for review enforcement.
# Approval policies live in .policy.yml at the repository root and are
# enforced by policy-bot (https://github.com/palantir/policy-bot).
#
# To change required reviewers for a path, edit .policy.yml.
#
# See https://github.com/tailscale/corp/issues/13972.
/tailcfg/ @tailscale/control-protocol-owners
-49
View File
@@ -148,55 +148,6 @@ sshintegrationtest: ## Run the SSH integration tests in various Docker container
generate: ## Generate code
./tool/go generate ./...
.PHONY: tsapp-build-and-flash-pi
tsapp-build-and-flash-pi: ## Build a tsapp-pi.arm64 GAF from HEAD and flash a local SD card (macOS auto-detects the disk; pass DISK=/dev/sdX on Linux)
cd gokrazy && ../tool/go run build.go --gaf --app=tsapp-pi.arm64
./tool/go run --exec=sudo ./cmd/tailscale configure flash-appliance \
--variant=pi-arm64 \
--gaf=gokrazy/tsapp-pi.arm64.gaf \
$(if $(DISK),--disk=$(DISK)) \
$(if $(wildcard $(HOME)/.ssh/id_ed25519.pub),--add-ssh-authorized-keys=$(HOME)/.ssh/id_ed25519.pub)
.PHONY: tsapp-qemu-pi
tsapp-qemu-pi: ## Build tsapp-pi.arm64 and boot it under qemu-system-aarch64 with a framebuffer GUI window and working network (requires mtools, dtc, qemu-efi-aarch64)
cd gokrazy && ../tool/go run build.go --build --app=tsapp-pi.arm64
# Extract the kernel from the FAT boot partition for direct -kernel boot.
rm -f gokrazy/tsapp-pi.arm64.vmlinuz
mcopy -i gokrazy/tsapp-pi.arm64.img@@4194304 ::vmlinuz gokrazy/tsapp-pi.arm64.vmlinuz
# Use the "virt" machine (not raspi3b) because it provides working
# PCI e1000 networking and, with UEFI firmware, an EFI framebuffer
# via the ramfb device. The raspi3b machine's USB NIC emulation is
# too broken for DHCP and its SoC watchdog reboots the guest.
#
# Find the UEFI firmware. Common paths:
# Debian/Ubuntu: /usr/share/qemu-efi-aarch64/QEMU_EFI.fd
# Homebrew: /opt/homebrew/share/qemu/edk2-aarch64-code.fd
# Fedora: /usr/share/edk2/aarch64/QEMU_EFI.fd
QEMU_EFI=$$(for f in \
/usr/share/qemu-efi-aarch64/QEMU_EFI.fd \
/opt/homebrew/share/qemu/edk2-aarch64-code.fd \
/usr/share/edk2/aarch64/QEMU_EFI.fd \
$$(dirname $$(which qemu-system-aarch64))/../share/qemu/edk2-aarch64-code.fd; do \
[ -f "$$f" ] && echo "$$f" && break; \
done) && \
[ -n "$$QEMU_EFI" ] || { echo "error: cannot find QEMU EFI firmware (install qemu-efi-aarch64)"; exit 1; } && \
qemu-system-aarch64 \
-M virt -cpu cortex-a53 -m 1G \
-bios "$$QEMU_EFI" \
-device ramfb \
-device e1000,netdev=net0 -netdev user,id=net0 \
-kernel gokrazy/tsapp-pi.arm64.vmlinuz \
-append "console=ttyAMA0,115200 nowatchdog gokrazy.log_to_serial=1 root=PARTUUID=60c24cc1-f3f9-427a-8199-dd02023b0001/PARTNROFF=1 ro init=/gokrazy/init rootwait" \
-drive file=gokrazy/tsapp-pi.arm64.img,format=raw,if=none,id=disk0 \
-device virtio-blk-device,drive=disk0 \
-serial mon:stdio
.PHONY: tsapp-push-pi
tsapp-push-pi: ## Build a tsapp-pi.arm64 GAF from HEAD and push it to a running Pi over the network (pass PI=<ip>)
@[ -n "$(PI)" ] || { echo "usage: make tsapp-push-pi PI=<ip-address>"; exit 1; }
cd gokrazy && ../tool/go run build.go --gaf --app=tsapp-pi.arm64
./tool/go run ./gokrazy/gafpush --gaf=gokrazy/tsapp-pi.arm64.gaf --pi=$(PI)
.PHONY: pin-github-actions
pin-github-actions:
./tool/go tool github.com/stacklok/frizbee actions .github/workflows
+1 -1
View File
@@ -1 +1 @@
1.103.0
1.99.0
+57 -19
View File
@@ -5,14 +5,13 @@ package appc
import (
"cmp"
"fmt"
"slices"
"strings"
"tailscale.com/ipn/ipnext"
"tailscale.com/tailcfg"
"tailscale.com/types/appctype"
"tailscale.com/types/dnstype"
"tailscale.com/util/mak"
"tailscale.com/util/set"
)
@@ -55,32 +54,71 @@ func PickConnector(nb ipnext.NodeBackend, app appctype.Conn25Attr) []tailcfg.Nod
return matches
}
// DNSAddrScheme is the custom URI scheme used for conn25-managed split DNS
// entries to determine the destination at query time rather than configuration
// time.
const DNSAddrScheme = "tailscale-app"
func AppDNSRoutes(hasCap func(c tailcfg.NodeCapability) bool, self tailcfg.NodeView) map[string][]*dnstype.Resolver {
// PickSplitDNSPeers looks at the netmap peers capabilities and finds which peers
// want to be connectors for which domains.
func PickSplitDNSPeers(hasCap func(c tailcfg.NodeCapability) bool, self tailcfg.NodeView, peers map[tailcfg.NodeID]tailcfg.NodeView, isSelfEligibleConnector bool) map[string][]tailcfg.NodeView {
var m map[string][]tailcfg.NodeView
if !hasCap(AppConnectorsExperimentalAttrName) {
return nil
return m
}
apps, err := tailcfg.UnmarshalNodeCapViewJSON[appctype.AppConnectorAttr](self.CapMap(), AppConnectorsExperimentalAttrName)
if err != nil {
return nil
return m
}
appNamesByDomain := map[string]string{}
// We strip the leading *. from any domains because the OS treats all domains
// that we pass to it as wildcard domains, and the OS would treat the * character
// as a literal domain component instead of treating it as a wildcard.
// We also use a Set to deduplicate the domains we pass to the OS in case removing
// the *. prefix resulted in duplicate entries.
tagToDomain := make(map[string]set.Set[string])
selfTags := set.SetOf(self.Tags().AsSlice())
selfRoutedDomains := set.Set[string]{}
for _, app := range apps {
domains := make(set.Set[string])
for _, domain := range app.Domains {
domain, _ = strings.CutPrefix(domain, "*.")
domain = strings.ToLower(domain)
// in the case of multiple apps specifying the same domain (which is misconfiguration
// that should be validated at point of input) last write wins.
appNamesByDomain[domain] = app.Name
domains.Add(strings.ToLower(strings.TrimPrefix(domain, "*.")))
}
for _, tag := range app.Connectors {
if tagToDomain[tag] == nil {
tagToDomain[tag] = set.Set[string]{}
}
tagToDomain[tag].AddSet(domains)
if isSelfEligibleConnector && selfTags.Contains(tag) {
selfRoutedDomains.AddSet(domains)
}
}
}
m := make(map[string][]*dnstype.Resolver, len(appNamesByDomain))
for domain, appName := range appNamesByDomain {
m[domain] = []*dnstype.Resolver{{Addr: fmt.Sprintf("%s:%s", DNSAddrScheme, appName), UseWithExitNode: true}}
// NodeIDs are Comparable, and we have a map of NodeID to NodeView anyway, so
// use a Set of NodeIDs to deduplicate, and populate into a []NodeView later.
var work map[string]set.Set[tailcfg.NodeID]
for _, peer := range peers {
if !isPeerEligibleConnector(peer) {
continue
}
for _, t := range peer.Tags().All() {
domains := tagToDomain[t]
for domain := range domains {
if selfRoutedDomains.Contains(domain) {
continue
}
if work[domain] == nil {
mak.Set(&work, domain, set.Set[tailcfg.NodeID]{})
}
work[domain].Add(peer.ID())
}
}
}
// Populate m. Make a []tailcfg.NodeView from []tailcfg.NodeID using the peers map.
// And sort it to our preference.
for domain, ids := range work {
nodes := make([]tailcfg.NodeView, 0, ids.Len())
for id := range ids {
nodes = append(nodes, peers[id])
}
sortByPreference(nodes)
mak.Set(&m, domain, nodes)
}
return m
}
+174 -65
View File
@@ -5,18 +5,17 @@ package appc
import (
"encoding/json"
"fmt"
"reflect"
"testing"
"github.com/google/go-cmp/cmp"
"tailscale.com/ipn/ipnext"
"tailscale.com/tailcfg"
"tailscale.com/types/appctype"
"tailscale.com/types/dnstype"
"tailscale.com/types/opt"
)
func TestAppDNSRoutes(t *testing.T) {
func TestPickSplitDNSPeers(t *testing.T) {
getBytesForAttr := func(name string, domains []string, tags []string) []byte {
attr := appctype.AppConnectorAttr{
Name: name,
@@ -36,102 +35,206 @@ func TestAppDNSRoutes(t *testing.T) {
appFiveBytes := getBytesForAttr("app5", []string{"*.example.com", "example.com"}, []string{"tag:one"})
appSixBytes := getBytesForAttr("app6", []string{"*.Example.com", "EXAMPLE.com", "EXAMPLE.COM"}, []string{"tag:one"})
resolver := func(appName string) []*dnstype.Resolver {
return []*dnstype.Resolver{{Addr: fmt.Sprintf("%s:%s", DNSAddrScheme, appName), UseWithExitNode: true}}
makeNodeView := func(id tailcfg.NodeID, name string, tags []string) tailcfg.NodeView {
return (&tailcfg.Node{
ID: id,
Name: name,
Tags: tags,
Hostinfo: (&tailcfg.Hostinfo{AppConnector: opt.NewBool(true)}).View(),
}).View()
}
nvp1 := makeNodeView(1, "p1", []string{"tag:one"})
nvp2 := makeNodeView(2, "p2", []string{"tag:four1", "tag:four2"})
nvp3 := makeNodeView(3, "p3", []string{"tag:two", "tag:three1"})
nvp4 := makeNodeView(4, "p4", []string{"tag:two", "tag:three2", "tag:four2"})
for _, tt := range []struct {
name string
hasCap bool
config []tailcfg.RawMessage
want map[string][]*dnstype.Resolver
name string
peers []tailcfg.NodeView
config []tailcfg.RawMessage
isEligibleConnector bool
selfTags []string
want map[string][]tailcfg.NodeView
}{
{
name: "no-capability", // hasCap false should return nil regardless of config.
hasCap: false,
name: "empty",
},
{
name: "no-apps", // hasCap true but no configured apps returns an empty map.
hasCap: true,
want: map[string][]*dnstype.Resolver{},
},
{
name: "bad-config", // bad config should return nil rather than error.
hasCap: true,
name: "bad-config", // bad config should return a nil map rather than error.
config: []tailcfg.RawMessage{tailcfg.RawMessage(`hey`)},
},
{
name: "single-app",
hasCap: true,
name: "no-peers",
config: []tailcfg.RawMessage{tailcfg.RawMessage(appOneBytes)},
want: map[string][]*dnstype.Resolver{
"example.com": resolver("app1"),
},
{
name: "peers-that-are-not-connectors",
config: []tailcfg.RawMessage{tailcfg.RawMessage(appOneBytes)},
peers: []tailcfg.NodeView{
(&tailcfg.Node{
ID: 5,
Name: "p5",
Tags: []string{"tag:one"},
}).View(),
(&tailcfg.Node{
ID: 6,
Name: "p6",
Tags: []string{"tag:one"},
}).View(),
},
},
{
name: "single-app-multi-domain",
hasCap: true,
config: []tailcfg.RawMessage{tailcfg.RawMessage(appThreeBytes)},
want: map[string][]*dnstype.Resolver{
"woo.b.example.com": resolver("app3"),
"hoo.b.example.com": resolver("app3"),
name: "peers-that-dont-match-tags",
config: []tailcfg.RawMessage{tailcfg.RawMessage(appOneBytes)},
peers: []tailcfg.NodeView{
makeNodeView(5, "p5", []string{"tag:seven"}),
makeNodeView(6, "p6", nil),
},
},
{
name: "multi-app-no-overlap",
hasCap: true,
name: "matching-tagged-connector-peers",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appOneBytes),
tailcfg.RawMessage(appTwoBytes),
tailcfg.RawMessage(appThreeBytes),
tailcfg.RawMessage(appFourBytes),
},
want: map[string][]*dnstype.Resolver{
"example.com": resolver("app1"),
"a.example.com": resolver("app2"),
peers: []tailcfg.NodeView{
nvp1,
nvp2,
nvp3,
nvp4,
makeNodeView(5, "p5", nil),
},
want: map[string][]tailcfg.NodeView{
// p5 has no matching tags and so doesn't appear
"example.com": {nvp1},
"a.example.com": {nvp3, nvp4},
"woo.b.example.com": {nvp2, nvp3, nvp4},
"hoo.b.example.com": {nvp3, nvp4},
"c.example.com": {nvp2, nvp4},
},
},
{
name: "domain-collision-last-write-wins",
hasCap: true,
name: "self-connector-exclude-self-domains",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appThreeBytes), // app3: woo.b.example.com, hoo.b.example.com
tailcfg.RawMessage(appFourBytes), // app4: woo.b.example.com, c.example.com
tailcfg.RawMessage(appOneBytes),
tailcfg.RawMessage(appTwoBytes),
tailcfg.RawMessage(appThreeBytes),
tailcfg.RawMessage(appFourBytes),
},
want: map[string][]*dnstype.Resolver{
// app4 overwrites app3 for the shared domain
"woo.b.example.com": resolver("app4"),
"hoo.b.example.com": resolver("app3"),
"c.example.com": resolver("app4"),
peers: []tailcfg.NodeView{
nvp1,
nvp2,
nvp3,
nvp4,
},
isEligibleConnector: true,
selfTags: []string{"tag:three1"},
want: map[string][]tailcfg.NodeView{
// woo.b.example.com and hoo.b.example.com are covered
// by tag:three1, and so is this self-node.
// So those domains should not be routed to peers.
// woo.b.example.com is also covered by another tag,
// but still not included since this connector can route to it.
"example.com": {nvp1},
"a.example.com": {nvp3, nvp4},
"c.example.com": {nvp2, nvp4},
},
},
{
name: "wildcards-are-stripped-and-deduped",
hasCap: true,
config: []tailcfg.RawMessage{tailcfg.RawMessage(appFiveBytes)},
want: map[string][]*dnstype.Resolver{
// *.example.com and example.com should both normalize to example.com.
"example.com": resolver("app5"),
name: "self-eligible-connector-no-matching-tag-include-all-domains",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appOneBytes),
tailcfg.RawMessage(appTwoBytes),
tailcfg.RawMessage(appThreeBytes),
tailcfg.RawMessage(appFourBytes),
},
peers: []tailcfg.NodeView{
nvp1,
nvp2,
nvp3,
nvp4,
},
isEligibleConnector: true,
selfTags: []string{"tag:unrelated"},
want: map[string][]tailcfg.NodeView{
// Self has prefs set but no tags matching any app,
// so no domains are self-routed and all appear.
"example.com": {nvp1},
"a.example.com": {nvp3, nvp4},
"woo.b.example.com": {nvp2, nvp3, nvp4},
"hoo.b.example.com": {nvp3, nvp4},
"c.example.com": {nvp2, nvp4},
},
},
{
name: "domains-are-normalized-and-deduped",
hasCap: true,
config: []tailcfg.RawMessage{tailcfg.RawMessage(appSixBytes)},
want: map[string][]*dnstype.Resolver{
// *.Example.com, EXAMPLE.com, EXAMPLE.COM should all normalize to example.com.
"example.com": resolver("app6"),
name: "self-not-eligible-connector-but-tagged-include-all-domains",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appOneBytes),
tailcfg.RawMessage(appTwoBytes),
tailcfg.RawMessage(appThreeBytes),
tailcfg.RawMessage(appFourBytes),
},
peers: []tailcfg.NodeView{
nvp1,
nvp2,
nvp3,
nvp4,
},
selfTags: []string{"tag:three1"},
want: map[string][]tailcfg.NodeView{
// Even though this self node has a tag for an app
// the prefs don't advertise as connector, so
// should still route through other connectors.
"example.com": {nvp1},
"a.example.com": {nvp3, nvp4},
"woo.b.example.com": {nvp2, nvp3, nvp4},
"hoo.b.example.com": {nvp3, nvp4},
"c.example.com": {nvp2, nvp4},
},
},
{
name: "sub-domains-and-top-domains-do-not-collide",
hasCap: true,
name: "wildcards-are-stripped-and-deduped",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appOneBytes),
tailcfg.RawMessage(appFiveBytes),
},
peers: []tailcfg.NodeView{
nvp1,
},
want: map[string][]tailcfg.NodeView{
// All the domains should be normalized to example.com
"example.com": {nvp1},
},
},
{
name: "domains-are-normalized-and-deduped",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appSixBytes),
},
peers: []tailcfg.NodeView{
nvp1,
},
want: map[string][]tailcfg.NodeView{
// All the domains should be normalized to example.com
"example.com": {nvp1},
},
},
{
name: "sub-domains-and-top-domains-do-not-collide",
config: []tailcfg.RawMessage{
tailcfg.RawMessage(appTwoBytes),
tailcfg.RawMessage(appFiveBytes),
},
want: map[string][]*dnstype.Resolver{
// *.example.com normalizes to example.com; a.example.com remains distinct.
"a.example.com": resolver("app2"),
"example.com": resolver("app5"),
peers: []tailcfg.NodeView{
nvp1,
nvp3,
},
want: map[string][]tailcfg.NodeView{
// The sub.example.com should remain distinct from example.com
"example.com": {nvp1},
"a.example.com": {nvp3},
},
},
} {
@@ -142,12 +245,18 @@ func TestAppDNSRoutes(t *testing.T) {
tailcfg.NodeCapability(AppConnectorsExperimentalAttrName): tt.config,
}
}
selfNode.Tags = append(selfNode.Tags, tt.selfTags...)
selfView := selfNode.View()
got := AppDNSRoutes(func(_ tailcfg.NodeCapability) bool {
return tt.hasCap
}, selfView)
if diff := cmp.Diff(tt.want, got); diff != "" {
t.Fatalf("AppDNSRoutes (-want, +got):\n%s", diff)
peers := map[tailcfg.NodeID]tailcfg.NodeView{}
for _, p := range tt.peers {
peers[p.ID()] = p
}
got := PickSplitDNSPeers(func(_ tailcfg.NodeCapability) bool {
return true
}, selfView, peers, tt.isEligibleConnector)
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("got %v, want %v", got, tt.want)
}
})
}
-7
View File
@@ -51,13 +51,6 @@ while [ "$#" -gt 1 ]; do
ldflags="$ldflags -w -s"
tags="${tags:+$tags,},$(GOOS= GOARCH= $go run ./cmd/featuretags --min)"
;;
--strip)
# --min overrides your flags, when you're using custom tags and want to
# additionally strip symbols to help reduce the size, this is the easiest
# way to do it.
shift
ldflags="$ldflags -w -s"
;;
--box)
if [ ! -z "${TAGS:-}" ]; then
echo "set either --box or \$TAGS, but not both"
-54
View File
@@ -10,55 +10,13 @@ import (
"crypto/tls"
"errors"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"go4.org/mem"
)
// rateLimitedError is returned from cert-fetching methods when the
// upstream ACME CA reported a rate limit. Callers should unpack it via
// [RateLimitRetryAfter].
type rateLimitedError struct {
retryAfter time.Duration
underlying error
}
func (e rateLimitedError) Error() string { return e.underlying.Error() }
func (e rateLimitedError) Unwrap() error { return e.underlying }
// RateLimitRetryAfter reports whether err was a rate-limit failure from
// the upstream ACME CA and, if so, returns the CA's suggested wait
// (zero if none was provided).
func RateLimitRetryAfter(err error) (retryAfter time.Duration, ok bool) {
var rl rateLimitedError
if errors.As(err, &rl) {
return rl.retryAfter, true
}
return 0, false
}
// retryAfterFromHeader parses a Retry-After header, matching the
// delta-seconds + HTTP-date pattern in tempfork/acme/http.go.
func retryAfterFromHeader(h http.Header) time.Duration {
v := h.Get("Retry-After")
if i, err := strconv.Atoi(v); err == nil {
return time.Duration(i) * time.Second
}
t, err := http.ParseTime(v)
if err != nil {
return 0
}
d := time.Until(t)
if d < 0 {
return 0
}
return d
}
// SetDNS adds a DNS TXT record for the given domain name, containing
// the provided TXT value. The intended use case is answering
// LetsEncrypt/ACME dns-01 challenges.
@@ -85,8 +43,6 @@ func (lc *Client) SetDNS(ctx context.Context, name, value string) error {
//
// It returns a cached certificate from disk if it's still valid.
//
// Rate-limit failures can be identified via [RateLimitRetryAfter].
//
// Deprecated: use [Client.CertPair].
func CertPair(ctx context.Context, domain string) (certPEM, keyPEM []byte, err error) {
return defaultClient.CertPair(ctx, domain)
@@ -96,8 +52,6 @@ func CertPair(ctx context.Context, domain string) (certPEM, keyPEM []byte, err e
//
// It returns a cached certificate from disk if it's still valid.
//
// Rate-limit failures can be identified via [RateLimitRetryAfter].
//
// API maturity: this is considered a stable API.
func (lc *Client) CertPair(ctx context.Context, domain string) (certPEM, keyPEM []byte, err error) {
return lc.CertPairWithValidity(ctx, domain, 0)
@@ -111,18 +65,10 @@ func (lc *Client) CertPair(ctx context.Context, domain string) (certPEM, keyPEM
// least the given duration, if permitted by the CA. If the certificate is
// valid, but for less than minValidity, it will be synchronously renewed.
//
// Rate-limit failures can be identified via [RateLimitRetryAfter].
//
// API maturity: this is considered a stable API.
func (lc *Client) CertPairWithValidity(ctx context.Context, domain string, minValidity time.Duration) (certPEM, keyPEM []byte, err error) {
res, err := lc.send(ctx, "GET", fmt.Sprintf("/localapi/v0/cert/%s?type=pair&min_validity=%s", domain, minValidity), 200, nil)
if err != nil {
if hse, ok := errors.AsType[httpStatusError](err); ok && hse.HTTPStatus == http.StatusTooManyRequests {
return nil, nil, rateLimitedError{
retryAfter: retryAfterFromHeader(hse.Header),
underlying: err,
}
}
return nil, nil, err
}
// with ?type=pair, the response PEM is first the one private
-3
View File
@@ -50,9 +50,6 @@ type DebugPortmapOpts struct {
// process.
//
// opts can be nil; if so, default values will be used.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DebugPortmap(ctx context.Context, opts *DebugPortmapOpts) (io.ReadCloser, error) {
vals := make(url.Values)
if opts == nil {
+13 -206
View File
@@ -2,12 +2,6 @@
// SPDX-License-Identifier: BSD-3-Clause
// Package local contains a Go client for the Tailscale LocalAPI.
//
// The APIs in this package vary in maturity: some methods are considered
// stable APIs and are documented as such, while others are not necessarily
// stable and are subject to change between releases. Methods without an
// explicit "API maturity" note in their documentation should be assumed
// to be unstable.
package local
import (
@@ -141,9 +135,6 @@ func (lc *Client) defaultDialer(ctx context.Context, network, addr string) (net.
// authenticating to the local Tailscale daemon vary by platform.
//
// DoLocalRequest may mutate the request to add Authorization headers.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DoLocalRequest(req *http.Request) (*http.Response, error) {
req.Header.Set("Tailscale-Cap", strconv.Itoa(int(tailcfg.CurrentCapabilityVersion)))
lc.tsClientOnce.Do(func() {
@@ -289,7 +280,7 @@ func (lc *Client) sendWithHeaders(
}
if res.StatusCode != wantStatus {
err = fmt.Errorf("%v: %s", res.Status, bytes.TrimSpace(slurp))
return nil, nil, httpStatusError{bestError(err, slurp), res.StatusCode, res.Header}
return nil, nil, httpStatusError{bestError(err, slurp), res.StatusCode}
}
return slurp, res.Header, nil
}
@@ -297,7 +288,6 @@ func (lc *Client) sendWithHeaders(
type httpStatusError struct {
error
HTTPStatus int
Header http.Header
}
func (lc *Client) get200(ctx context.Context, path string) ([]byte, error) {
@@ -326,8 +316,6 @@ func decodeJSON[T any](b []byte) (ret T, err error) {
// For connections proxied by tailscaled, this looks up the owner of the given
// address as TCP first, falling back to UDP; if you want to only check a
// specific address family, use WhoIsProto.
//
// API maturity: this is considered a stable API.
func (lc *Client) WhoIs(ctx context.Context, remoteAddr string) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(remoteAddr))
if err != nil {
@@ -342,8 +330,6 @@ func (lc *Client) WhoIs(ctx context.Context, remoteAddr string) (*apitype.WhoIsR
// WhoIsForService is like [Client.WhoIs] but scopes the returned CapMap to
// capabilities that apply to the named VIP service. This enables per-service
// capability resolution on hosts that advertise multiple VIP services.
//
// API maturity: this is considered a stable API.
func (lc *Client) WhoIsForService(ctx context.Context, remoteAddr string, svcName tailcfg.ServiceName) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(remoteAddr)+"&svc_name="+url.QueryEscape(string(svcName)))
if err != nil {
@@ -359,8 +345,6 @@ func (lc *Client) WhoIsForService(ctx context.Context, remoteAddr string, svcNam
// capabilities that apply to the given destination IP. The IP may be a
// VIP service address, the node's own tailnet address, or any other
// routable IP the node handles.
//
// API maturity: this is considered a stable API.
func (lc *Client) WhoIsForIP(ctx context.Context, remoteAddr string, dst netip.Addr) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(remoteAddr)+"&dst_ip="+url.QueryEscape(dst.String()))
if err != nil {
@@ -379,8 +363,6 @@ var ErrPeerNotFound = errors.New("peer not found")
// WhoIsNodeKey returns the owner of the given wireguard public key.
//
// If not found, the error is ErrPeerNotFound.
//
// API maturity: this is considered a stable API.
func (lc *Client) WhoIsNodeKey(ctx context.Context, key key.NodePublic) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(key.String()))
if err != nil {
@@ -396,8 +378,6 @@ func (lc *Client) WhoIsNodeKey(ctx context.Context, key key.NodePublic) (*apityp
// IP:port, for the given protocol (tcp or udp).
//
// If not found, the error is [ErrPeerNotFound].
//
// API maturity: this is considered a stable API.
func (lc *Client) WhoIsProto(ctx context.Context, proto, remoteAddr string) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?proto="+url.QueryEscape(proto)+"&addr="+url.QueryEscape(remoteAddr))
if err != nil {
@@ -474,9 +454,6 @@ func (lc *Client) SetGauge(ctx context.Context, name string, value int) error {
// TailDaemonLogs returns a stream the Tailscale daemon's logs as they arrive.
// Close the context to stop the stream.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) TailDaemonLogs(ctx context.Context) (io.Reader, error) {
req, err := http.NewRequestWithContext(ctx, "GET", "http://"+apitype.LocalAPIHost+"/localapi/v0/logtap", nil)
if err != nil {
@@ -493,18 +470,12 @@ func (lc *Client) TailDaemonLogs(ctx context.Context) (io.Reader, error) {
}
// EventBusGraph returns a graph of active publishers and subscribers in the eventbus
// as a [eventbus.DebugTopics].
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
// as a [eventbus.DebugTopics]
func (lc *Client) EventBusGraph(ctx context.Context) ([]byte, error) {
return lc.get200(ctx, "/localapi/v0/debug-bus-graph")
}
// EventBusQueues returns a JSON snapshot of event bus queue depths per client.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) EventBusQueues(ctx context.Context) ([]byte, error) {
return lc.get200(ctx, "/localapi/v0/debug-bus-queues")
}
@@ -513,9 +484,6 @@ func (lc *Client) EventBusQueues(ctx context.Context) ([]byte, error) {
// Each pair is a valid event and a nil error, or a zero event a non-nil error.
// In case of error, the iterator ends after the pair reporting the error.
// Iteration stops if ctx ends.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) StreamBusEvents(ctx context.Context) iter.Seq2[eventbus.DebugEvent, error] {
return func(yield func(eventbus.DebugEvent, error) bool) {
req, err := http.NewRequestWithContext(ctx, "GET",
@@ -584,8 +552,6 @@ type BugReportOpts struct {
//
// The opts type specifies options to pass to the Tailscale daemon when
// generating this bug report.
//
// API maturity: this is considered a stable API.
func (lc *Client) BugReportWithOpts(ctx context.Context, opts BugReportOpts) (string, error) {
qparams := make(url.Values)
if opts.Note != "" {
@@ -631,17 +597,12 @@ func (lc *Client) BugReportWithOpts(ctx context.Context, opts BugReportOpts) (st
//
// This is the same as calling [Client.BugReportWithOpts] and only specifying the Note
// field.
//
// API maturity: this is considered a stable API.
func (lc *Client) BugReport(ctx context.Context, note string) (string, error) {
return lc.BugReportWithOpts(ctx, BugReportOpts{Note: note})
}
// DebugAction invokes a debug action, such as "rebind" or "restun".
// These are development tools.
//
// API maturity: this method is not considered a stable API and is
// subject to change or removal between releases.
// These are development tools and subject to change or removal over time.
func (lc *Client) DebugAction(ctx context.Context, action string) error {
body, err := lc.send(ctx, "POST", "/localapi/v0/debug?action="+url.QueryEscape(action), 200, nil)
if err != nil {
@@ -652,10 +613,7 @@ func (lc *Client) DebugAction(ctx context.Context, action string) error {
// DebugActionBody invokes a debug action with a body parameter, such as
// "debug-force-prefer-derp".
// These are development tools.
//
// API maturity: this method is not considered a stable API and is
// subject to change or removal between releases.
// These are development tools and subject to change or removal over time.
func (lc *Client) DebugActionBody(ctx context.Context, action string, rbody io.Reader) error {
body, err := lc.send(ctx, "POST", "/localapi/v0/debug?action="+url.QueryEscape(action), 200, rbody)
if err != nil {
@@ -665,10 +623,7 @@ func (lc *Client) DebugActionBody(ctx context.Context, action string, rbody io.R
}
// DebugResultJSON invokes a debug action and returns its result as something JSON-able.
// These are development tools.
//
// API maturity: this method is not considered a stable API and is
// subject to change or removal between releases.
// These are development tools and subject to change or removal over time.
func (lc *Client) DebugResultJSON(ctx context.Context, action string) (any, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/debug?action="+url.QueryEscape(action), 200, nil)
if err != nil {
@@ -686,10 +641,7 @@ func (lc *Client) DebugResultJSON(ctx context.Context, action string) (any, erro
// callers of [Client.DebugResultJSON] otherwise need to do to get a typed
// value.
//
// These are development tools.
//
// API maturity: this function is not considered a stable API and is
// subject to change or removal between releases.
// These are development tools and subject to change or removal over time.
func GetDebugResultJSON[T any](ctx context.Context, lc *Client, action string) (T, error) {
var v T
body, err := lc.send(ctx, "POST", "/localapi/v0/debug?action="+url.QueryEscape(action), 200, nil)
@@ -731,9 +683,6 @@ func (lc *Client) SetDevStoreKeyValue(ctx context.Context, key, value string) er
// SetComponentDebugLogging sets component's debug logging enabled for
// the provided duration. If the duration is in the past, the debug logging
// is disabled.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) SetComponentDebugLogging(ctx context.Context, component string, d time.Duration) error {
if !buildfeatures.HasDebug {
return feature.ErrUnavailable
@@ -762,8 +711,6 @@ func Status(ctx context.Context) (*ipnstate.Status, error) {
}
// Status returns the Tailscale daemon's status.
//
// API maturity: this is considered a stable API.
func (lc *Client) Status(ctx context.Context) (*ipnstate.Status, error) {
return lc.status(ctx, "")
}
@@ -774,8 +721,6 @@ func StatusWithoutPeers(ctx context.Context) (*ipnstate.Status, error) {
}
// StatusWithoutPeers returns the Tailscale daemon's status, without the peer info.
//
// API maturity: this is considered a stable API.
func (lc *Client) StatusWithoutPeers(ctx context.Context) (*ipnstate.Status, error) {
return lc.status(ctx, "?peers=false")
}
@@ -880,9 +825,6 @@ func (lc *Client) PushFile(ctx context.Context, target tailcfg.StableNodeID, siz
// CheckIPForwarding asks the local Tailscale daemon whether it looks like the
// machine is properly configured to forward IP packets as a subnet router
// or exit node.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) CheckIPForwarding(ctx context.Context) error {
if !buildfeatures.HasAdvertiseRoutes {
return nil
@@ -906,9 +848,6 @@ func (lc *Client) CheckIPForwarding(ctx context.Context) error {
// CheckUDPGROForwarding asks the local Tailscale daemon whether it looks like
// the machine is optimally configured to forward UDP packets as a subnet router
// or exit node.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) CheckUDPGROForwarding(ctx context.Context) error {
body, err := lc.get200(ctx, "/localapi/v0/check-udp-gro-forwarding")
if err != nil {
@@ -958,9 +897,6 @@ func (lc *Client) CheckPrefs(ctx context.Context, p *ipn.Prefs) error {
return err
}
// GetPrefs returns the [ipn.Prefs] of the current Tailscale profile.
//
// API maturity: this is considered a stable API.
func (lc *Client) GetPrefs(ctx context.Context) (*ipn.Prefs, error) {
body, err := lc.get200(ctx, "/localapi/v0/prefs")
if err != nil {
@@ -978,8 +914,6 @@ func (lc *Client) GetPrefs(ctx context.Context) (*ipn.Prefs, error) {
// or a policy restriction. An optional reason or justification for the request can be
// provided as a context value using [apitype.RequestReasonKey]. If permitted by policy,
// access may be granted, and the reason will be logged for auditing purposes.
//
// API maturity: this is considered a stable API.
func (lc *Client) EditPrefs(ctx context.Context, mp *ipn.MaskedPrefs) (*ipn.Prefs, error) {
body, err := lc.send(ctx, "PATCH", "/localapi/v0/prefs", http.StatusOK, jsonBody(mp))
if err != nil {
@@ -990,9 +924,6 @@ func (lc *Client) EditPrefs(ctx context.Context, mp *ipn.MaskedPrefs) (*ipn.Pref
// GetDNSOSConfig returns the system DNS configuration for the current device.
// That is, it returns the DNS configuration that the system would use if Tailscale weren't being used.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) GetDNSOSConfig(ctx context.Context) (*apitype.DNSOSConfig, error) {
if !buildfeatures.HasDNS {
return nil, feature.ErrUnavailable
@@ -1026,26 +957,7 @@ func (lc *Client) QueryDNS(ctx context.Context, name string, queryType string) (
return res.Bytes, res.Resolvers, nil
}
// StartLoginInteractive starts an interactive login, requesting a new
// auth URL from the control plane if a login flow is not already in
// progress. If one is, the existing auth URL is re-sent.
//
// The auth URL is not returned by this method; it is delivered
// asynchronously to IPN bus watchers (see [Client.WatchIPNBus]) as an
// [ipn.Notify] with a non-empty BrowseToURL field. StartLoginInteractive
// returns as soon as the login has been requested; it does not wait for
// the login to complete.
//
// Calling StartLoginInteractive does not itself change the node's
// desired run state, but successfully completing the login does: the
// node's WantRunning pref is set to true, so a stopped node
// ("tailscale down") starts once the login finishes. If the login is
// completed as a different user or node identity than the current
// profile's, the node switches to an existing profile matching the new
// identity if one exists, or else updates the current profile to the
// new identity.
//
// API maturity: this is considered a stable API.
// StartLoginInteractive starts an interactive login.
func (lc *Client) StartLoginInteractive(ctx context.Context) error {
_, err := lc.send(ctx, "POST", "/localapi/v0/login-interactive", http.StatusNoContent, nil)
return err
@@ -1070,8 +982,6 @@ func (lc *Client) Logout(ctx context.Context) error {
// tailscaled), a FQDN, or an IP address.
//
// The ctx is only used for the duration of the call, not the lifetime of the [net.Conn].
//
// API maturity: this is considered a stable API.
func (lc *Client) DialTCP(ctx context.Context, host string, port uint16) (net.Conn, error) {
return lc.UserDial(ctx, "tcp", host, port)
}
@@ -1083,8 +993,6 @@ func (lc *Client) DialTCP(ctx context.Context, host string, port uint16) (net.Co
//
// The ctx is only used for the duration of the call, not the lifetime of the
// [net.Conn].
//
// API maturity: this is considered a stable API.
func (lc *Client) UserDial(ctx context.Context, network, host string, port uint16) (net.Conn, error) {
connCh := make(chan net.Conn, 1)
trace := httptrace.ClientTrace{
@@ -1149,10 +1057,6 @@ func (lc *Client) UserDial(ctx context.Context, network, host string, port uint1
// CurrentDERPMap returns the current DERPMap that is being used by the local tailscaled.
// It is intended to be used with netcheck to see availability of DERPs.
//
// API maturity: this is considered a stable API, though the returned
// [tailcfg.DERPMap] type is subject to minor changes over time; its
// general shape is stable.
func (lc *Client) CurrentDERPMap(ctx context.Context) (*tailcfg.DERPMap, error) {
var derpMap tailcfg.DERPMap
res, err := lc.send(ctx, "GET", "/localapi/v0/derpmap", 200, nil)
@@ -1169,8 +1073,6 @@ func (lc *Client) CurrentDERPMap(ctx context.Context) (*tailcfg.DERPMap, error)
// fetch TLS certificates, equivalent to the DNS.CertDomains field of the
// current netmap. The returned list is sorted in ascending order, and is
// empty if no netmap has been received yet.
//
// API maturity: this is considered a stable API.
func (lc *Client) CertDomains(ctx context.Context) ([]string, error) {
body, err := lc.get200(ctx, "/localapi/v0/cert-domains")
if err != nil {
@@ -1192,13 +1094,11 @@ func (lc *Client) DNSConfig(ctx context.Context) (*tailcfg.DNSConfig, error) {
}
// PeerByID returns a peer's current full [tailcfg.Node] looked up by its
// [tailcfg.NodeID]. It returns an error if no peer with that NodeID is in the
// current netmap.
// [tailcfg.NodeID], in O(1) time on the daemon side. It returns an error
// if no peer with that NodeID is in the current netmap.
//
// It is intended for callers that observed a peer-mutation signal (e.g.
// [ipn.Notify.PeerChangedPatch] or [ipn.Notify.PeersChanged]) and want the
// latest state of the affected node without having to apply the patch
// themselves.
// It is intended for callers that need the latest state of a single peer
// without fetching the entire netmap.
func (lc *Client) PeerByID(ctx context.Context, id tailcfg.NodeID) (*tailcfg.Node, error) {
body, err := lc.get200(ctx, "/localapi/v0/peer-by-id?id="+strconv.FormatInt(int64(id), 10))
if err != nil {
@@ -1207,24 +1107,6 @@ func (lc *Client) PeerByID(ctx context.Context, id tailcfg.NodeID) (*tailcfg.Nod
return decodeJSON[*tailcfg.Node](body)
}
// UserProfile returns the current [tailcfg.UserProfile] for the given
// [tailcfg.UserID]. It returns an error if no user with that UserID is in the
// current netmap.
//
// It is the LocalAPI fallback for IPN-bus consumers that see a UserID
// referenced by a peer Node and want to resolve it to a UserProfile. Sessions
// opted in to [ipn.NotifyPeerChanges] / [ipn.NotifyPeerPatches] also receive
// UserProfiles automatically via [ipn.Notify.UserProfiles].
//
// API maturity: this is considered a stable API.
func (lc *Client) UserProfile(ctx context.Context, id tailcfg.UserID) (*tailcfg.UserProfile, error) {
body, err := lc.get200(ctx, "/localapi/v0/user-profile?id="+strconv.FormatInt(int64(id), 10))
if err != nil {
return nil, err
}
return decodeJSON[*tailcfg.UserProfile](body)
}
// PingOpts contains options for the ping request.
//
// The zero value is valid, which means to use defaults.
@@ -1261,8 +1143,6 @@ func (lc *Client) Ping(ctx context.Context, ip netip.Addr, pingtype tailcfg.Ping
// DisconnectControl shuts down all connections to control, thus making control consider this node inactive. This can be
// run on HA subnet router or app connector replicas before shutting them down to ensure peers get told to switch over
// to another replica whilst there is still some grace period for the existing connections to terminate.
//
// API maturity: this is considered a stable API.
func (lc *Client) DisconnectControl(ctx context.Context) error {
_, _, err := lc.sendWithHeaders(ctx, "POST", "/localapi/v0/disconnect-control", 200, nil, nil)
if err != nil {
@@ -1358,18 +1238,13 @@ func (lc *Client) ReloadConfig(ctx context.Context) (ok bool, err error) {
// SwitchToEmptyProfile creates and switches to a new unnamed profile. The new
// profile is not assigned an ID until it is persisted after a successful login.
// In order to login to the new profile, the user must call
// [Client.StartLoginInteractive].
//
// API maturity: this is considered a stable API.
// In order to login to the new profile, the user must call LoginInteractive.
func (lc *Client) SwitchToEmptyProfile(ctx context.Context) error {
_, err := lc.send(ctx, "PUT", "/localapi/v0/profiles/", http.StatusCreated, nil)
return err
}
// SwitchProfile switches to the given profile.
//
// API maturity: this is considered a stable API.
func (lc *Client) SwitchProfile(ctx context.Context, profile ipn.ProfileID) error {
_, err := lc.send(ctx, "POST", "/localapi/v0/profiles/"+url.PathEscape(string(profile)), 204, nil)
return err
@@ -1404,11 +1279,6 @@ func (lc *Client) QueryFeature(ctx context.Context, feature string) (*tailcfg.Qu
return decodeJSON[*tailcfg.QueryFeatureResponse](body)
}
// DebugDERPRegion reports diagnostic information about the DERP region with
// the given ID or code.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DebugDERPRegion(ctx context.Context, regionIDOrCode string) (*ipnstate.DebugDERPRegionReport, error) {
v := url.Values{"region": {regionIDOrCode}}
body, err := lc.send(ctx, "POST", "/localapi/v0/debug-derp-region?"+v.Encode(), 200, nil)
@@ -1419,9 +1289,6 @@ func (lc *Client) DebugDERPRegion(ctx context.Context, regionIDOrCode string) (*
}
// DebugPacketFilterRules returns the packet filter rules for the current device.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DebugPacketFilterRules(ctx context.Context) ([]tailcfg.FilterRule, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/debug-packet-filter-rules", 200, nil)
if err != nil {
@@ -1433,9 +1300,6 @@ func (lc *Client) DebugPacketFilterRules(ctx context.Context) ([]tailcfg.FilterR
// DebugSetExpireIn marks the current node key to expire in d.
//
// This is meant primarily for debug and testing.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DebugSetExpireIn(ctx context.Context, d time.Duration) error {
v := url.Values{"expiry": {fmt.Sprint(time.Now().Add(d).Unix())}}
_, err := lc.send(ctx, "POST", "/localapi/v0/set-expiry-sooner?"+v.Encode(), 200, nil)
@@ -1444,9 +1308,6 @@ func (lc *Client) DebugSetExpireIn(ctx context.Context, d time.Duration) error {
// DebugPeerRelaySessions returns debug information about the current peer
// relay sessions running through this node.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DebugPeerRelaySessions(ctx context.Context) (*status.ServerStatus, error) {
body, err := lc.send(ctx, "GET", "/localapi/v0/debug-peer-relay-sessions", 200, nil)
if err != nil {
@@ -1459,9 +1320,6 @@ func (lc *Client) DebugPeerRelaySessions(ctx context.Context) (*status.ServerSta
//
// The provided context does not determine the lifetime of the
// returned [io.ReadCloser].
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) StreamDebugCapture(ctx context.Context) (io.ReadCloser, error) {
req, err := http.NewRequestWithContext(ctx, "POST", "http://"+apitype.LocalAPIHost+"/localapi/v0/debug-capture", nil)
if err != nil {
@@ -1488,16 +1346,9 @@ func (lc *Client) StreamDebugCapture(ctx context.Context) (io.ReadCloser, error)
// resources.
//
// A default set of ipn.Notify messages are returned but the set can be modified by mask.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) WatchIPNBus(ctx context.Context, mask ipn.NotifyWatchOpt) (*IPNBusWatcher, error) {
m, err := mask.MarshalText()
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, "GET",
"http://"+apitype.LocalAPIHost+"/localapi/v0/watch-ipn-bus?mask="+string(m),
"http://"+apitype.LocalAPIHost+"/localapi/v0/watch-ipn-bus?mask="+fmt.Sprint(mask),
nil)
if err != nil {
return nil, err
@@ -1521,8 +1372,6 @@ func (lc *Client) WatchIPNBus(ctx context.Context, mask ipn.NotifyWatchOpt) (*IP
// CheckUpdate returns a [*tailcfg.ClientVersion] indicating whether or not an update is available
// to be installed via the LocalAPI. In case the LocalAPI can't install updates, it returns a
// ClientVersion that says that we are up to date.
//
// API maturity: this is considered a stable API.
func (lc *Client) CheckUpdate(ctx context.Context) (*tailcfg.ClientVersion, error) {
body, err := lc.get200(ctx, "/localapi/v0/update/check")
if err != nil {
@@ -1539,8 +1388,6 @@ func (lc *Client) CheckUpdate(ctx context.Context) (*tailcfg.ClientVersion, erro
// To turn it on, there must have been a previously used exit node.
// The most previously used one is reused.
// This is a convenience method for GUIs. To select an actual one, update the prefs.
//
// API maturity: this is considered a stable API.
func (lc *Client) SetUseExitNode(ctx context.Context, on bool) error {
_, err := lc.send(ctx, "POST", "/localapi/v0/set-use-exit-node-enabled?enabled="+strconv.FormatBool(on), http.StatusOK, nil)
return err
@@ -1549,9 +1396,6 @@ func (lc *Client) SetUseExitNode(ctx context.Context, on bool) error {
// DriveSetServerAddr instructs Taildrive to use the server at addr to access
// the filesystem. This is used on platforms like Windows and MacOS to let
// Taildrive know to use the file server running in the GUI app.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DriveSetServerAddr(ctx context.Context, addr string) error {
_, err := lc.send(ctx, "PUT", "/localapi/v0/drive/fileserver-address", http.StatusCreated, strings.NewReader(addr))
return err
@@ -1560,9 +1404,6 @@ func (lc *Client) DriveSetServerAddr(ctx context.Context, addr string) error {
// DriveShareSet adds or updates the given share in the list of shares that
// Taildrive will serve to remote nodes. If a share with the same name already
// exists, the existing share is replaced/updated.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DriveShareSet(ctx context.Context, share *drive.Share) error {
_, err := lc.send(ctx, "PUT", "/localapi/v0/drive/shares", http.StatusCreated, jsonBody(share))
return err
@@ -1570,9 +1411,6 @@ func (lc *Client) DriveShareSet(ctx context.Context, share *drive.Share) error {
// DriveShareRemove removes the share with the given name from the list of
// shares that Taildrive will serve to remote nodes.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DriveShareRemove(ctx context.Context, name string) error {
_, err := lc.send(
ctx,
@@ -1584,9 +1422,6 @@ func (lc *Client) DriveShareRemove(ctx context.Context, name string) error {
}
// DriveShareRename renames the share from old to new name.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DriveShareRename(ctx context.Context, oldName, newName string) error {
_, err := lc.send(
ctx,
@@ -1599,9 +1434,6 @@ func (lc *Client) DriveShareRename(ctx context.Context, oldName, newName string)
// DriveShareList returns the list of shares that drive is currently serving
// to remote nodes.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) DriveShareList(ctx context.Context) ([]*drive.Share, error) {
result, err := lc.get200(ctx, "/localapi/v0/drive/shares")
if err != nil {
@@ -1658,25 +1490,8 @@ func (lc *Client) SuggestExitNode(ctx context.Context) (apitype.ExitNodeSuggesti
return decodeJSON[apitype.ExitNodeSuggestionResponse](body)
}
// SuggestExitNodeWithProbe requests an exit node suggestion based on an immediate routecheck probe,
// waits for the probe to finish, and returns the exit node's details.
func (lc *Client) SuggestExitNodeWithProbe(ctx context.Context) (apitype.ExitNodeSuggestionResponse, error) {
if !buildfeatures.HasRouteCheck {
return apitype.ExitNodeSuggestionResponse{}, feature.ErrUnavailable
}
v := url.Values{"probe": {"true"}}
body, err := lc.send(ctx, "POST", "/localapi/v0/suggest-exit-node?"+v.Encode(), 200, nil)
if err != nil {
return apitype.ExitNodeSuggestionResponse{}, err
}
return decodeJSON[apitype.ExitNodeSuggestionResponse](body)
}
// CheckSOMarkInUse reports whether the socket mark option is in use. This will only
// be true if tailscale is running on Linux and tailscaled uses SO_MARK.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) CheckSOMarkInUse(ctx context.Context) (bool, error) {
body, err := lc.get200(ctx, "/localapi/v0/check-so-mark-in-use")
if err != nil {
@@ -1693,19 +1508,11 @@ func (lc *Client) CheckSOMarkInUse(ctx context.Context) (bool, error) {
}
// ShutdownTailscaled requests a graceful shutdown of tailscaled.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) ShutdownTailscaled(ctx context.Context) error {
_, err := lc.send(ctx, "POST", "/localapi/v0/shutdown", 200, nil)
return err
}
// GetAppConnectorRouteInfo returns the current [appctype.RouteInfo] for this
// node's app connector.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) GetAppConnectorRouteInfo(ctx context.Context) (appctype.RouteInfo, error) {
body, err := lc.get200(ctx, "/localapi/v0/appc-route-info")
if err != nil {
-43
View File
@@ -1,43 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:build !ts_omit_routecheck
package local
import (
"context"
"errors"
"fmt"
"net/http"
"tailscale.com/net/routecheck"
)
// ErrReportPending is returned by [Client.RouteCheck] and [Client.RouteCheckProbe]
// when the report is pending.
var ErrRouteCheckReportUnavailable = errors.New("report pending")
// RouteCheckProbe performs a routecheck probe and waits for its report.
func (lc *Client) RouteCheckProbe(ctx context.Context) (*routecheck.Report, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/routecheck?probe=true", http.StatusOK, nil)
if err != nil {
if hs, ok := errors.AsType[httpStatusError](err); ok && hs.HTTPStatus == http.StatusNoContent {
return nil, ErrRouteCheckReportUnavailable
}
return nil, fmt.Errorf("error %w: %s", err, body)
}
return decodeJSON[*routecheck.Report](body)
}
// RouteCheck requests the report compiled by the latest routecheck probe.
func (lc *Client) RouteCheck(ctx context.Context) (*routecheck.Report, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/routecheck", http.StatusOK, nil)
if err != nil {
if hs, ok := errors.AsType[httpStatusError](err); ok && hs.HTTPStatus == http.StatusNoContent {
return nil, ErrRouteCheckReportUnavailable
}
return nil, fmt.Errorf("error %w: %s", err, body)
}
return decodeJSON[*routecheck.Report](body)
}
-3
View File
@@ -17,9 +17,6 @@ import (
// GetServeConfig return the current serve config.
//
// If the serve config is empty, it returns (nil, nil).
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) GetServeConfig(ctx context.Context) (*ipn.ServeConfig, error) {
body, h, err := lc.sendWithHeaders(ctx, "GET", "/localapi/v0/serve-config", 200, nil, nil)
if err != nil {
-37
View File
@@ -1,37 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:build !ts_omit_serviceclientprefs
package local
import (
"context"
"net/http"
"tailscale.com/client/tailscale/apitype"
"tailscale.com/feature/serviceclientprefs/serviceclient"
)
// GetServiceClientPrefs returns all of the current profile's [serviceclient.Prefs].
//
// API maturity: this method is not considered a stable API and is subject to change between releases.
func (lc *Client) GetServiceClientPrefs(ctx context.Context) (serviceclient.Prefs, error) {
body, err := lc.get200(ctx, "/localapi/v0/prefs/service-clients")
if err != nil {
return nil, err
}
return decodeJSON[serviceclient.Prefs](body)
}
// SetServiceClientPref merges the non-empty fields from an [apitype.ServiceClientPrefRequest] into the
// saved service client prefs for the current profile and returns the full updated set.
//
// API maturity: this method is not considered a stable API and is subject to change between releases.
func (lc *Client) SetServiceClientPref(ctx context.Context, req apitype.ServiceClientPrefRequest) (serviceclient.Prefs, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/prefs/service-clients", http.StatusOK, jsonBody(req))
if err != nil {
return nil, err
}
return decodeJSON[serviceclient.Prefs](body)
}
-3
View File
@@ -13,9 +13,6 @@ import (
)
// GetEffectivePolicy returns the effective policy for the specified scope.
//
// API maturity: this method is not considered a stable API and is
// subject to change between releases.
func (lc *Client) GetEffectivePolicy(ctx context.Context, scope setting.PolicyScope) (*setting.Snapshot, error) {
scopeID, err := scope.MarshalText()
if err != nil {
+29 -94
View File
@@ -18,22 +18,17 @@ import (
"tailscale.com/types/tkatype"
)
// TailnetLockStatus fetches information about the tailnet key authority, if one is configured.
func (lc *Client) TailnetLockStatus(ctx context.Context) (*ipnstate.TailnetLockStatus, error) {
// NetworkLockStatus fetches information about the tailnet key authority, if one is configured.
func (lc *Client) NetworkLockStatus(ctx context.Context) (*ipnstate.NetworkLockStatus, error) {
body, err := lc.send(ctx, "GET", "/localapi/v0/tka/status", 200, nil)
if err != nil {
return nil, fmt.Errorf("error: %w", err)
}
return decodeJSON[*ipnstate.TailnetLockStatus](body)
return decodeJSON[*ipnstate.NetworkLockStatus](body)
}
// Deprecated: use [Client.TailnetLockStatus] instead.
func (lc *Client) NetworkLockStatus(ctx context.Context) (*ipnstate.TailnetLockStatus, error) {
return lc.TailnetLockStatus(ctx)
}
// TailnetLockInit initializes the tailnet key authority.
func (lc *Client) TailnetLockInit(ctx context.Context, keys []tka.Key, disablementValues [][]byte, supportDisablement []byte) (*ipnstate.TailnetLockStatus, error) {
// NetworkLockInit initializes the tailnet key authority.
func (lc *Client) NetworkLockInit(ctx context.Context, keys []tka.Key, disablementValues [][]byte, supportDisablement []byte) (*ipnstate.NetworkLockStatus, error) {
var b bytes.Buffer
type initRequest struct {
Keys []tka.Key
@@ -49,17 +44,12 @@ func (lc *Client) TailnetLockInit(ctx context.Context, keys []tka.Key, disableme
if err != nil {
return nil, fmt.Errorf("error: %w", err)
}
return decodeJSON[*ipnstate.TailnetLockStatus](body)
return decodeJSON[*ipnstate.NetworkLockStatus](body)
}
// Deprecated: use [Client.TailnetLockInit] instead.
func (lc *Client) NetworkLockInit(ctx context.Context, keys []tka.Key, disablementValues [][]byte, supportDisablement []byte) (*ipnstate.TailnetLockStatus, error) {
return lc.TailnetLockInit(ctx, keys, disablementValues, supportDisablement)
}
// TailnetLockWrapPreauthKey wraps a pre-auth key with information to
// NetworkLockWrapPreauthKey wraps a pre-auth key with information to
// enable unattended bringup in the locked tailnet.
func (lc *Client) TailnetLockWrapPreauthKey(ctx context.Context, preauthKey string, tkaKey key.NLPrivate) (string, error) {
func (lc *Client) NetworkLockWrapPreauthKey(ctx context.Context, preauthKey string, tkaKey key.NLPrivate) (string, error) {
encodedPrivate, err := tkaKey.MarshalText()
if err != nil {
return "", err
@@ -81,13 +71,8 @@ func (lc *Client) TailnetLockWrapPreauthKey(ctx context.Context, preauthKey stri
return string(body), nil
}
// Deprecated: use [Client.TailnetLockWrapPreauthKey] instead.
func (lc *Client) NetworkLockWrapPreauthKey(ctx context.Context, preauthKey string, tkaKey key.NLPrivate) (string, error) {
return lc.TailnetLockWrapPreauthKey(ctx, preauthKey, tkaKey)
}
// TailnetLockModify adds and/or removes key(s) to the tailnet key authority.
func (lc *Client) TailnetLockModify(ctx context.Context, addKeys, removeKeys []tka.Key) error {
// NetworkLockModify adds and/or removes key(s) to the tailnet key authority.
func (lc *Client) NetworkLockModify(ctx context.Context, addKeys, removeKeys []tka.Key) error {
var b bytes.Buffer
type modifyRequest struct {
AddKeys []tka.Key
@@ -104,14 +89,9 @@ func (lc *Client) TailnetLockModify(ctx context.Context, addKeys, removeKeys []t
return nil
}
// Deprecated: use [Client.TailnetLockModify] instead.
func (lc *Client) NetworkLockModify(ctx context.Context, addKeys, removeKeys []tka.Key) error {
return lc.TailnetLockModify(ctx, addKeys, removeKeys)
}
// TailnetLockSign signs the specified node-key and transmits that signature to the control plane.
// NetworkLockSign signs the specified node-key and transmits that signature to the control plane.
// rotationPublic, if specified, must be an ed25519 public key.
func (lc *Client) TailnetLockSign(ctx context.Context, nodeKey key.NodePublic, rotationPublic []byte) error {
func (lc *Client) NetworkLockSign(ctx context.Context, nodeKey key.NodePublic, rotationPublic []byte) error {
var b bytes.Buffer
type signRequest struct {
NodeKey key.NodePublic
@@ -128,13 +108,8 @@ func (lc *Client) TailnetLockSign(ctx context.Context, nodeKey key.NodePublic, r
return nil
}
// Deprecated: use [Client.TailnetLockSign] instead.
func (lc *Client) NetworkLockSign(ctx context.Context, nodeKey key.NodePublic, rotationPublic []byte) error {
return lc.TailnetLockSign(ctx, nodeKey, rotationPublic)
}
// TailnetLockAffectedSigs returns all signatures signed by the specified keyID.
func (lc *Client) TailnetLockAffectedSigs(ctx context.Context, keyID tkatype.KeyID) ([]tkatype.MarshaledSignature, error) {
// NetworkLockAffectedSigs returns all signatures signed by the specified keyID.
func (lc *Client) NetworkLockAffectedSigs(ctx context.Context, keyID tkatype.KeyID) ([]tkatype.MarshaledSignature, error) {
body, err := lc.send(ctx, "POST", "/localapi/v0/tka/affected-sigs", 200, bytes.NewReader(keyID))
if err != nil {
return nil, fmt.Errorf("error: %w", err)
@@ -142,29 +117,19 @@ func (lc *Client) TailnetLockAffectedSigs(ctx context.Context, keyID tkatype.Key
return decodeJSON[[]tkatype.MarshaledSignature](body)
}
// Deprecated: use [Client.TailnetLockAffectedSigs] instead.
func (lc *Client) NetworkLockAffectedSigs(ctx context.Context, keyID tkatype.KeyID) ([]tkatype.MarshaledSignature, error) {
return lc.TailnetLockAffectedSigs(ctx, keyID)
}
// TailnetLockLog returns up to maxEntries number of changes to tailnet-lock state.
func (lc *Client) TailnetLockLog(ctx context.Context, maxEntries int) ([]ipnstate.TailnetLockUpdate, error) {
// NetworkLockLog returns up to maxEntries number of changes to tailnet-lock state.
func (lc *Client) NetworkLockLog(ctx context.Context, maxEntries int) ([]ipnstate.NetworkLockUpdate, error) {
v := url.Values{}
v.Set("limit", fmt.Sprint(maxEntries))
body, err := lc.send(ctx, "GET", "/localapi/v0/tka/log?"+v.Encode(), 200, nil)
if err != nil {
return nil, fmt.Errorf("error %w: %s", err, body)
}
return decodeJSON[[]ipnstate.TailnetLockUpdate](body)
return decodeJSON[[]ipnstate.NetworkLockUpdate](body)
}
// Deprecated: use [Client.TailnetLockLog] instead.
func (lc *Client) NetworkLockLog(ctx context.Context, maxEntries int) ([]ipnstate.TailnetLockUpdate, error) {
return lc.TailnetLockLog(ctx, maxEntries)
}
// TailnetLockForceLocalDisable forcibly shuts down tailnet lock on this node.
func (lc *Client) TailnetLockForceLocalDisable(ctx context.Context) error {
// NetworkLockForceLocalDisable forcibly shuts down tailnet lock on this node.
func (lc *Client) NetworkLockForceLocalDisable(ctx context.Context) error {
// This endpoint expects an empty JSON stanza as the payload.
var b bytes.Buffer
if err := json.NewEncoder(&b).Encode(struct{}{}); err != nil {
@@ -177,14 +142,9 @@ func (lc *Client) TailnetLockForceLocalDisable(ctx context.Context) error {
return nil
}
// Deprecated: use [Client.TailnetLockForceLocalDisable] instead.
func (lc *Client) NetworkLockForceLocalDisable(ctx context.Context) error {
return lc.TailnetLockForceLocalDisable(ctx)
}
// TailnetLockVerifySigningDeeplink verifies the tailnet lock deeplink contained
// NetworkLockVerifySigningDeeplink verifies the tailnet lock deeplink contained
// in url and returns information extracted from it.
func (lc *Client) TailnetLockVerifySigningDeeplink(ctx context.Context, url string) (*tka.DeeplinkValidationResult, error) {
func (lc *Client) NetworkLockVerifySigningDeeplink(ctx context.Context, url string) (*tka.DeeplinkValidationResult, error) {
vr := struct {
URL string
}{url}
@@ -197,13 +157,8 @@ func (lc *Client) TailnetLockVerifySigningDeeplink(ctx context.Context, url stri
return decodeJSON[*tka.DeeplinkValidationResult](body)
}
// Deprecated: use [Client.TailnetLockVerifySigningDeeplink] instead.
func (lc *Client) NetworkLockVerifySigningDeeplink(ctx context.Context, url string) (*tka.DeeplinkValidationResult, error) {
return lc.TailnetLockVerifySigningDeeplink(ctx, url)
}
// TailnetLockGenRecoveryAUM generates an AUM for recovering from a tailnet-lock key compromise.
func (lc *Client) TailnetLockGenRecoveryAUM(ctx context.Context, removeKeys []tkatype.KeyID, forkFrom tka.AUMHash) ([]byte, error) {
// NetworkLockGenRecoveryAUM generates an AUM for recovering from a tailnet-lock key compromise.
func (lc *Client) NetworkLockGenRecoveryAUM(ctx context.Context, removeKeys []tkatype.KeyID, forkFrom tka.AUMHash) ([]byte, error) {
vr := struct {
Keys []tkatype.KeyID
ForkFrom string
@@ -217,13 +172,8 @@ func (lc *Client) TailnetLockGenRecoveryAUM(ctx context.Context, removeKeys []tk
return body, nil
}
// Deprecated: use [Client.TailnetLockGenRecoveryAUM] instead.
func (lc *Client) NetworkLockGenRecoveryAUM(ctx context.Context, removeKeys []tkatype.KeyID, forkFrom tka.AUMHash) ([]byte, error) {
return lc.TailnetLockGenRecoveryAUM(ctx, removeKeys, forkFrom)
}
// TailnetLockCosignRecoveryAUM co-signs a recovery AUM using the node's tailnet lock key.
func (lc *Client) TailnetLockCosignRecoveryAUM(ctx context.Context, aum tka.AUM) ([]byte, error) {
// NetworkLockCosignRecoveryAUM co-signs a recovery AUM using the node's tailnet lock key.
func (lc *Client) NetworkLockCosignRecoveryAUM(ctx context.Context, aum tka.AUM) ([]byte, error) {
r := bytes.NewReader(aum.Serialize())
body, err := lc.send(ctx, "POST", "/localapi/v0/tka/cosign-recovery-aum", 200, r)
if err != nil {
@@ -233,13 +183,8 @@ func (lc *Client) TailnetLockCosignRecoveryAUM(ctx context.Context, aum tka.AUM)
return body, nil
}
// Deprecated: use [Client.TailnetLockCosignRecoveryAUM] instead.
func (lc *Client) NetworkLockCosignRecoveryAUM(ctx context.Context, aum tka.AUM) ([]byte, error) {
return lc.TailnetLockCosignRecoveryAUM(ctx, aum)
}
// TailnetLockSubmitRecoveryAUM submits a recovery AUM to the control plane.
func (lc *Client) TailnetLockSubmitRecoveryAUM(ctx context.Context, aum tka.AUM) error {
// NetworkLockSubmitRecoveryAUM submits a recovery AUM to the control plane.
func (lc *Client) NetworkLockSubmitRecoveryAUM(ctx context.Context, aum tka.AUM) error {
r := bytes.NewReader(aum.Serialize())
_, err := lc.send(ctx, "POST", "/localapi/v0/tka/submit-recovery-aum", 200, r)
if err != nil {
@@ -248,20 +193,10 @@ func (lc *Client) TailnetLockSubmitRecoveryAUM(ctx context.Context, aum tka.AUM)
return nil
}
// Deprecated: use [Client.TailnetLockSubmitRecoveryAUM] instead.
func (lc *Client) NetworkLockSubmitRecoveryAUM(ctx context.Context, aum tka.AUM) error {
return lc.TailnetLockSubmitRecoveryAUM(ctx, aum)
}
// TailnetLockDisable shuts down tailnet-lock across the tailnet.
func (lc *Client) TailnetLockDisable(ctx context.Context, secret []byte) error {
// NetworkLockDisable shuts down tailnet-lock across the tailnet.
func (lc *Client) NetworkLockDisable(ctx context.Context, secret []byte) error {
if _, err := lc.send(ctx, "POST", "/localapi/v0/tka/disable", 200, bytes.NewReader(secret)); err != nil {
return fmt.Errorf("error: %w", err)
}
return nil
}
// Deprecated: use [Client.TailnetLockDisable] instead.
func (lc *Client) NetworkLockDisable(ctx context.Context, secret []byte) error {
return lc.TailnetLockDisable(ctx, secret)
}
+26 -40
View File
@@ -69,11 +69,6 @@ func (menu *Menu) Run(client *local.Client) {
go menu.lc.SetGauge(menu.bgCtx, "systray_running", 1)
defer menu.lc.SetGauge(menu.bgCtx, "systray_running", 0)
// set initial title, which is used by the systray package as the ID of the StatusNotifierItem.
// This value will get overwritten later as the client status changes.
// This must be called before systray.Run.
systray.SetTitle("tailscale")
systray.Run(menu.onReady, menu.onExit)
}
@@ -177,6 +172,10 @@ See https://tailscale.com/kb/1597/linux-systray for more information.`)
}
setAppIcon(disconnected)
// set initial title, which is used by the systray package as the ID of the StatusNotifierItem.
// This value will get overwritten later as the client status changes.
systray.SetTitle("tailscale")
menu.rebuild()
menu.mu.Lock()
@@ -293,23 +292,21 @@ func (menu *Menu) rebuild() {
accounts := systray.AddMenuItem(account, "")
setRemoteIcon(accounts, menu.curProfile.UserProfile.ProfilePicURL)
time.Sleep(newMenuDelay)
if len(menu.allProfiles) > 1 {
for _, profile := range menu.allProfiles {
title := profileTitle(profile)
var item *systray.MenuItem
if profile.ID == menu.curProfile.ID {
item = accounts.AddSubMenuItemCheckbox(title, "", true)
} else {
item = accounts.AddSubMenuItem(title, "")
}
setRemoteIcon(item, profile.UserProfile.ProfilePicURL)
onClick(ctx, item, func(ctx context.Context) {
select {
case <-ctx.Done():
case menu.accountsCh <- profile.ID:
}
})
for _, profile := range menu.allProfiles {
title := profileTitle(profile)
var item *systray.MenuItem
if profile.ID == menu.curProfile.ID {
item = accounts.AddSubMenuItemCheckbox(title, "", true)
} else {
item = accounts.AddSubMenuItem(title, "")
}
setRemoteIcon(item, profile.UserProfile.ProfilePicURL)
onClick(ctx, item, func(ctx context.Context) {
select {
case <-ctx.Done():
case menu.accountsCh <- profile.ID:
}
})
}
}
@@ -355,27 +352,16 @@ func (menu *Menu) rebuild() {
// profileTitle returns the title string for a profile menu item.
func profileTitle(profile ipn.LoginProfile) string {
tailnet := ""
title := profile.Name
if profile.NetworkProfile.DomainName != "" {
tailnet = profile.NetworkProfile.DisplayNameOrDefault()
if runtime.GOOS == "windows" || runtime.GOOS == "darwin" {
// windows and mac don't support multi-line menu
title += " (" + profile.NetworkProfile.DisplayNameOrDefault() + ")"
} else {
title += "\n" + profile.NetworkProfile.DisplayNameOrDefault()
}
}
// windows and mac don't support multi-line menu items.
multiline := runtime.GOOS != "windows" && runtime.GOOS != "darwin"
return formatProfileTitle(profile.Name, tailnet, multiline)
}
// formatProfileTitle builds a profile menu label from a login name and an
// optional tailnet name. The tailnet portion is omitted when it matches the
// login name, so single-user tailnets don't show the same string twice.
func formatProfileTitle(name, tailnet string, multiline bool) string {
if tailnet == "" || strings.EqualFold(name, tailnet) {
return name
}
if multiline {
return name + "\n" + tailnet
}
return name + " (" + tailnet + ")"
return title
}
var (
-27
View File
@@ -13,33 +13,6 @@ import (
"tailscale.com/types/key"
)
func TestProfileTitleMultiline(t *testing.T) {
t.Parallel()
tests := []struct {
name string
login string
tailnet string
multiline bool
want string
}{
{"no_tailnet", "alice@example.com", "", true, "alice@example.com"},
{"dup_exact", "example.com", "example.com", true, "example.com"},
{"dup_casefold", "Example.com", "example.com", false, "Example.com"},
{"distinct_multiline", "alice@example.com", "example.com", true, "alice@example.com\nexample.com"},
{"distinct_singleline", "alice@example.com", "example.com", false, "alice@example.com (example.com)"},
{"empty", "", "", true, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := formatProfileTitle(tt.login, tt.tailnet, tt.multiline); got != tt.want {
t.Errorf("profileTitleMultiline; got %v, want %v", got, tt.want)
}
})
}
}
func TestRecommendedIsActive(t *testing.T) {
t.Parallel()
+1 -18
View File
@@ -76,7 +76,7 @@ type ReloadConfigResponse struct {
type ExitNodeSuggestionResponse struct {
ID tailcfg.StableNodeID
Name string
Location tailcfg.LocationView `json:",omitzero"`
Location tailcfg.LocationView `json:",omitempty"`
}
// DNSOSConfig mimics dns.OSConfig without forcing us to import the entire dns package
@@ -104,20 +104,3 @@ type OptionalFeatures struct {
// are not guaranteed to be present.)
Features map[string]bool
}
// ServiceClientPrefRequest is the body POSTed to the LocalAPI endpoint /localapi/v0/prefs/service-clients.
// Empty values for Client, Username, and DatabaseName mean "don't change this value".
type ServiceClientPrefRequest struct {
// Key is the identifier for the service client pref. Required. Format is "<serviceName>:<port>"
// where serviceName is a [tailcfg.ServiceName], e.g. "svc:my-db:5432".
Key string
// Client is the name of the client that the user picked in the service launch. Optional.
Client string `json:",omitzero"`
// Username is the username that the user entered in the service launch. Optional.
Username string `json:",omitzero"`
// DatabaseName is the database name that the user entered in the service launch. Optional.
DatabaseName string `json:",omitzero"`
}
+1 -1
View File
@@ -22,7 +22,7 @@ type Key struct {
// KeyCapabilities are the capabilities of a Key.
type KeyCapabilities struct {
Devices KeyDeviceCapabilities `json:"devices"`
Devices KeyDeviceCapabilities `json:"devices,omitempty"`
}
// KeyDeviceCapabilities are the device-related capabilities of a Key.
+1 -2
View File
@@ -199,8 +199,7 @@ func (s *Server) controlSupportsCheckMode(ctx context.Context) bool {
if err != nil {
return true
}
return strings.HasSuffix(controlURL.Host, ".tailscale.com") ||
controlURL.Host == "control.tailscale" // for natlab tests
return strings.HasSuffix(controlURL.Host, ".tailscale.com")
}
// awaitUserAuth blocks until the given session auth has been completed
@@ -61,7 +61,7 @@ export default function ExitNodeSelector({
none, // not using exit nodes
advertising, // advertising as exit node
using, // using another exit node
offline, // selected exit node is offline
offline, // selected exit node node is offline
] = useMemo(
() => [
selected.ID === noExitNode.ID,
+6 -95
View File
@@ -11,7 +11,6 @@ import (
"bufio"
"bytes"
"compress/gzip"
"context"
"encoding/json"
"errors"
"fmt"
@@ -38,25 +37,6 @@ import (
"tailscale.com/version/distro"
)
// GokrazyUpdateArgs contains arguments for updating a Gokrazy appliance from a
// GAF fetched from a URL.
type GokrazyUpdateArgs struct {
// URL is the GAF download URL.
URL string
// AllowUnsigned permits installing a GAF without signature verification.
// It is intended for tests that serve a GAF from a fileserver that does
// not publish distsign.pub.
AllowUnsigned bool
// Logf is optional; nil discards log messages.
Logf logger.Logf
}
// GokrazyUpdateFromURL updates a Gokrazy appliance from a GAF fetched from a
// URL, if Gokrazy update support is linked into the binary.
var GokrazyUpdateFromURL feature.Hook[func(context.Context, GokrazyUpdateArgs) error]
const (
StableTrack = "stable"
UnstableTrack = "unstable"
@@ -217,17 +197,6 @@ func (up *Updater) getUpdateFunction() (fn updateFunction, canAutoUpdate bool) {
// release cadence with Synology Package Center and use their
// auto-update mechanism.
return up.updateSynology, false
case distro.Gokrazy:
// Only the official Tailscale appliance image (built with the
// ts_appliance build tag, which causes hostinfo to report
// Package="tsapp") is auto-updatable. A user running a custom
// Gokrazy build that happens to include tailscaled must not be
// updated with our stock GAFs. TS_FORCE_ALLOW_TSAPP_UPDATE is an
// escape hatch for callers who know what they're doing.
if hi.Package != "tsapp" && !envknob.Bool("TS_FORCE_ALLOW_TSAPP_UPDATE") {
return nil, false
}
return up.updateGokrazy, true
case distro.Debian: // includes Ubuntu
return up.updateDebLike, true
case distro.Arch:
@@ -361,7 +330,7 @@ func (up *Updater) updateSynology() error {
if err != nil {
return err
}
latest, err := LatestPackages(up.Track)
latest, err := latestPackages(up.Track)
if err != nil {
return err
}
@@ -895,56 +864,6 @@ func (up *Updater) updateFreeBSD() (err error) {
return nil
}
// updateGokrazy fetches the latest signed GAF for this gokrazy device variant
// (vm-amd64, vm-arm64, or pi-arm64) from up.PkgsAddr and applies it via the
// local gokrazy init update API.
func (up *Updater) updateGokrazy() error {
if !GokrazyUpdateFromURL.IsSet() {
return errors.New("gokrazy update support is not linked into this binary")
}
variant, err := gokrazyDeviceVariant()
if err != nil {
return err
}
latest, err := LatestPackages(up.Track)
if err != nil {
return err
}
gafName, ok := latest.GAFs[variant]
if !ok {
return fmt.Errorf("no GAF for device %q on %q track", variant, up.Track)
}
if latest.GAFsVersion == "" {
return fmt.Errorf("no GAF version on %q track", up.Track)
}
if !up.confirm(latest.GAFsVersion) {
return nil
}
gafURL := fmt.Sprintf("%s/%s/%s", strings.TrimRight(up.PkgsAddr, "/"), up.Track, gafName)
up.Logf("Updating to %s (%s)", latest.GAFsVersion, gafURL)
return GokrazyUpdateFromURL.Get()(context.Background(), GokrazyUpdateArgs{
URL: gafURL,
Logf: up.Logf,
})
}
// gokrazyDeviceVariant returns the GAFs JSON key for the current gokrazy
// device, e.g. "vm-amd64", "vm-arm64", or "pi-arm64". On arm64, it reads the
// device-tree model to tell a Raspberry Pi apart from a VM.
func gokrazyDeviceVariant() (string, error) {
switch runtime.GOARCH {
case "amd64":
return "vm-amd64", nil
case "arm64":
b, _ := os.ReadFile("/sys/firmware/devicetree/base/model")
if strings.HasPrefix(strings.Trim(string(b), "\x00\r\n\t "), "Raspberry Pi") {
return "pi-arm64", nil
}
return "vm-arm64", nil
}
return "", fmt.Errorf("unsupported gokrazy GOARCH %q", runtime.GOARCH)
}
func (up *Updater) updateLinuxBinary() error {
// Root is needed to overwrite binaries and restart systemd unit.
if err := requireRoot(); err != nil {
@@ -1305,7 +1224,7 @@ func LatestTailscaleVersion(track string) (string, error) {
track = CurrentTrack
}
latest, err := LatestPackages(track)
latest, err := latestPackages(track)
if err != nil {
return "", err
}
@@ -1317,11 +1236,8 @@ func LatestTailscaleVersion(track string) (string, error) {
ver = latest.MacZipsVersion
case "linux":
ver = latest.TarballsVersion
switch distro.Get() {
case distro.Synology:
if distro.Get() == distro.Synology {
ver = latest.SPKsVersion
case distro.Gokrazy:
ver = latest.GAFsVersion
}
}
@@ -1331,8 +1247,7 @@ func LatestTailscaleVersion(track string) (string, error) {
return ver, nil
}
// TrackPackages is the JSON shape served at <pkgs>/<track>/?mode=json.
type TrackPackages struct {
type trackPackages struct {
Version string
Tarballs map[string]string
TarballsVersion string
@@ -1340,8 +1255,6 @@ type TrackPackages struct {
ExesVersion string
MSIs map[string]string
MSIsVersion string
GAFs map[string]string
GAFsVersion string
MacZips map[string]string
MacZipsVersion string
SPKs map[string]map[string]string
@@ -1350,16 +1263,14 @@ type TrackPackages struct {
var tailscaleHTTPEndpoint = "https://pkgs.tailscale.com"
// LatestPackages fetches the package manifest served at
// <pkgs>/<track>/?mode=json for the current runtime.GOOS.
func LatestPackages(track string) (*TrackPackages, error) {
func latestPackages(track string) (*trackPackages, error) {
url := fmt.Sprintf("%s/%s/?mode=json&os=%s", tailscaleHTTPEndpoint, track, runtime.GOOS)
res, err := http.Get(url)
if err != nil {
return nil, fmt.Errorf("fetching latest tailscale version: %w", err)
}
defer res.Body.Close()
var latest TrackPackages
var latest trackPackages
if err := json.NewDecoder(res.Body).Decode(&latest); err != nil {
return nil, fmt.Errorf("decoding JSON: %v: %w", res.Status, err)
}
-239
View File
@@ -1,239 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:build linux
package clientupdate
import (
"archive/zip"
"context"
"fmt"
"hash/crc32"
"io"
"net"
"net/http"
"os"
"strings"
"time"
"tailscale.com/clientupdate/distsign"
"tailscale.com/types/logger"
"tailscale.com/util/progresstracking"
)
const (
gokrazyUpdateSocket = "/run/gokrazy-http.sock"
gokrazyUpdateBaseURL = "http://gokrazy-local-unixsock"
)
// GokrazyUpdateFromURL downloads a Gokrazy archive format file from args.URL,
// installs its partitions using the local gokrazy init update API, switches to
// the new root partition, and asks gokrazy to reboot.
//
// The local gokrazy API is reached over gokrazyUpdateSocket. The
// gokrazyUpdateBaseURL host is only a net/http URL sentinel; it is not resolved
// with DNS.
func init() {
GokrazyUpdateFromURL.Set(gokrazyUpdateFromURL)
}
func gokrazyUpdateFromURL(ctx context.Context, args GokrazyUpdateArgs) error {
logf := args.Logf
if logf == nil {
logf = logger.Discard
}
tmp, err := os.CreateTemp("", "tailscale-gokrazy-*.gaf")
if err != nil {
return err
}
tmpName := tmp.Name()
tmp.Close()
defer os.Remove(tmpName)
logf("downloading %s", args.URL)
if args.AllowUnsigned {
if err := downloadUnverified(ctx, logf, args.URL, tmpName); err != nil {
return err
}
} else {
if err := distsign.DownloadVerified(ctx, logf, args.URL, tmpName); err != nil {
return err
}
}
zr, err := zip.OpenReader(tmpName)
if err != nil {
return err
}
defer zr.Close()
logf("download complete")
gokClient := gokrazyHTTPClient()
for _, part := range []struct {
name string
path string
}{
{"root.img", "/update/root"},
{"boot.img", "/update/boot"},
{"mbr.img", "/update/mbr"},
} {
logf("writing %s...", part.name)
if err := putGokrazyGAFMember(ctx, gokClient, zr.File, part.name, part.path); err != nil {
return err
}
logf("wrote %s", part.name)
}
if err := postGokrazy(ctx, gokClient, "/update/switch"); err != nil {
return err
}
logf("switched boot target")
if err := postGokrazy(ctx, gokClient, "/reboot?async=true&kexec_merge_cmdline=true"); err != nil {
return err
}
logf("reboot requested")
return nil
}
// downloadUnverified saves the GAF at srcURL to dstPath without verifying
// a signature. It is used only when args.AllowUnsigned is set, for tests
// that serve the GAF from a fileserver that does not publish distsign.pub
// and for the gafpush "sftp the GAF onto the appliance and update from a
// local path" flow, which uses a "file://" URL.
func downloadUnverified(ctx context.Context, logf logger.Logf, srcURL, dstPath string) error {
if after, ok := strings.CutPrefix(srcURL, "file://"); ok {
return copyLocalFile(after, dstPath, logf)
}
req, err := http.NewRequestWithContext(ctx, "GET", srcURL, nil)
if err != nil {
return err
}
res, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return fmt.Errorf("download GAF: %s", res.Status)
}
f, err := os.Create(dstPath)
if err != nil {
return err
}
total := res.ContentLength
pw := progresstracking.NewWriter(io.Discard, total, time.Second, func(done int64) {
if total > 0 {
logf("downloading: %d / %d MB (%.0f%%)", done>>20, total>>20, float64(done)/float64(total)*100)
}
})
if _, err := io.Copy(f, io.TeeReader(res.Body, pw)); err != nil {
f.Close()
return err
}
return f.Close()
}
// copyLocalFile copies the GAF at src to dst. Used by the "file://" branch
// of downloadUnverified. The source file is left in place; callers that
// staged it (e.g. gafpush) clean up after the update completes.
func copyLocalFile(src, dst string, logf logger.Logf) error {
sf, err := os.Open(src)
if err != nil {
return err
}
defer sf.Close()
df, err := os.Create(dst)
if err != nil {
return err
}
fi, err := sf.Stat()
if err != nil {
df.Close()
return err
}
total := fi.Size()
logf("copying local GAF %s (%d MB)", src, total>>20)
pw := progresstracking.NewWriter(io.Discard, total, time.Second, func(done int64) {
if total > 0 {
logf("copying: %d / %d MB (%.0f%%)", done>>20, total>>20, float64(done)/float64(total)*100)
}
})
if _, err := io.Copy(df, io.TeeReader(sf, pw)); err != nil {
df.Close()
return err
}
return df.Close()
}
func gokrazyHTTPClient() *http.Client {
tr := http.DefaultTransport.(*http.Transport).Clone()
tr.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, "unix", gokrazyUpdateSocket)
}
return &http.Client{
Transport: tr,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
}
func putGokrazyGAFMember(ctx context.Context, hc *http.Client, files []*zip.File, name, path string) error {
var zf *zip.File
for _, f := range files {
if f.Name == name {
zf = f
break
}
}
if zf == nil {
return fmt.Errorf("GAF is missing %s", name)
}
rc, err := zf.Open()
if err != nil {
return err
}
defer rc.Close()
h := crc32.NewIEEE()
body := io.TeeReader(rc, h)
req, err := http.NewRequestWithContext(ctx, "PUT", gokrazyUpdateBaseURL+path, body)
if err != nil {
return err
}
req.ContentLength = int64(zf.UncompressedSize64)
req.Header.Set("X-Gokrazy-Update-Hash", "crc32")
res, err := hc.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
resBody, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
if res.StatusCode != http.StatusOK {
return fmt.Errorf("PUT %s: %s: %s", path, res.Status, strings.TrimSpace(string(resBody)))
}
if got, want := strings.TrimSpace(string(resBody)), fmt.Sprintf("%08x", h.Sum32()); got != want {
return fmt.Errorf("PUT %s: gokrazy checksum = %q; want %q", path, got, want)
}
return nil
}
func postGokrazy(ctx context.Context, hc *http.Client, path string) error {
req, err := http.NewRequestWithContext(ctx, "POST", gokrazyUpdateBaseURL+path, nil)
if err != nil {
return err
}
res, err := hc.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
return fmt.Errorf("POST %s: %s: %s", path, res.Status, strings.TrimSpace(string(body)))
}
return nil
}
+1 -1
View File
@@ -373,7 +373,7 @@ func TestCheckOutdatedAlpineRepo(t *testing.T) {
testServ := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
version := TrackPackages{
version := trackPackages{
MSIsVersion: tt.latestHTTPVersion,
MacZipsVersion: tt.latestHTTPVersion,
TarballsVersion: tt.latestHTTPVersion,
+23 -7
View File
@@ -56,11 +56,9 @@ import (
"github.com/hdevalence/ed25519consensus"
"golang.org/x/crypto/blake2s"
"tailscale.com/feature"
"tailscale.com/net/netutil"
"tailscale.com/types/logger"
"tailscale.com/util/httpm"
"tailscale.com/util/must"
"tailscale.com/util/progresstracking"
)
const (
@@ -331,7 +329,7 @@ func fetch(url string, limit int64) ([]byte, error) {
// download writes the response body of url into a local file at dst, up to
// limit bytes. On success, the returned value is a BLAKE2s hash of the file.
func (c *Client) download(ctx context.Context, url, dst string, limit int64) ([]byte, int64, error) {
tr := netutil.NewDefaultTransport()
tr := http.DefaultTransport.(*http.Transport).Clone()
tr.Proxy = feature.HookProxyFromEnvironment.GetOrNil()
defer tr.CloseIdleConnections()
hc := &http.Client{
@@ -374,10 +372,7 @@ func (c *Client) download(ctx context.Context, url, dst string, limit int64) ([]
return nil, 0, err
}
defer of.Close()
total := res.ContentLength
pw := progresstracking.NewWriter(io.Discard, total, 2*time.Second, func(done int64) {
c.logf("Downloaded %v/%v (%.1f%%)", done, total, float64(done)/float64(total)*100)
})
pw := &progressWriter{total: res.ContentLength, logf: c.logf}
h := NewPackageHash()
n, err := io.Copy(io.MultiWriter(of, h, pw), io.LimitReader(dlRes.Body, limit))
if err != nil {
@@ -392,10 +387,31 @@ func (c *Client) download(ctx context.Context, url, dst string, limit int64) ([]
if err := of.Close(); err != nil {
return nil, n, err
}
pw.print()
return h.Sum(nil), h.Len(), nil
}
type progressWriter struct {
done int64
total int64
lastPrint time.Time
logf logger.Logf
}
func (pw *progressWriter) Write(p []byte) (n int, err error) {
pw.done += int64(len(p))
if time.Since(pw.lastPrint) > 2*time.Second {
pw.print()
}
return len(p), nil
}
func (pw *progressWriter) print() {
pw.lastPrint = time.Now()
pw.logf("Downloaded %v/%v (%.1f%%)", pw.done, pw.total, float64(pw.done)/float64(pw.total)*100)
}
func parsePrivateKey(data []byte, typeTag string) (ed25519.PrivateKey, error) {
b, rest := pem.Decode(data)
if b == nil {
-42
View File
@@ -1,42 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package distsign
import (
"context"
"fmt"
"net/url"
"strings"
"tailscale.com/types/logger"
)
// DownloadVerified is a convenience wrapper around [Client.Download]
// for callers that have a full URL (e.g.
// https://pkgs.tailscale.com/unstable/foo.gaf) rather than a base URL
// plus path. It splits srcURL into a base ("scheme://host") and a path,
// constructs a [Client] for the base, and downloads with signature
// verification to dstPath.
func DownloadVerified(ctx context.Context, logf logger.Logf, srcURL, dstPath string) error {
if logf == nil {
logf = logger.Discard
}
u, err := url.Parse(srcURL)
if err != nil {
return fmt.Errorf("parsing URL %q: %w", srcURL, err)
}
if u.Scheme == "" || u.Host == "" {
return fmt.Errorf("URL %q is missing scheme or host", srcURL)
}
base := &url.URL{Scheme: u.Scheme, User: u.User, Host: u.Host}
path := strings.TrimPrefix(u.Path, "/")
if path == "" {
return fmt.Errorf("URL %q has no path component", srcURL)
}
c, err := NewClient(logf, base.String())
if err != nil {
return err
}
return c.Download(ctx, path, dstPath)
}
+7 -40
View File
@@ -143,32 +143,19 @@ func main() {
log.Printf("Using cigocached at %s", *srvURL)
}
c.remote = &cachers.HTTPClient{
BaseURL: *srvURL,
Disk: c.disk,
HTTPClient: httpClient(srvHost, *srvHostDial),
AccessToken: *token,
Verbose: *verbose,
BestEffortHTTP: true,
AsyncPutTimeout: asyncPutTimeout,
AsyncPutMaxConcurrent: 10,
BaseURL: *srvURL,
Disk: c.disk,
HTTPClient: httpClient(srvHost, *srvHostDial),
AccessToken: *token,
Verbose: *verbose,
BestEffortHTTP: true,
}
}
var p *cacheproc.Process
p = &cacheproc.Process{
Close: func() error {
if c.remote != nil {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if !c.remote.Shutdown(ctx) {
log.Printf("cigocacher: timed out waiting for background PUTs to drain")
}
// Always surface dropped PUTs.
if timedOut, canceled := c.remote.PutsTimedOut.Load(), c.remote.PutsCanceled.Load(); timedOut+canceled > 0 {
log.Printf("cigocacher: %d background PUTs timed out, %d canceled", timedOut, canceled)
}
}
if c.verbose {
log.Printf("cigocacher: closing; %d gets (%d hits, %d misses, %d errors); %d puts (%d errors)",
log.Printf("gocacheprog: closing; %d gets (%d hits, %d misses, %d errors); %d puts (%d errors)",
p.Gets.Load(), p.GetHits.Load(), p.GetMisses.Load(), p.GetErrors.Load(), p.Puts.Load(), p.PutErrors.Load())
}
return c.close()
@@ -351,23 +338,3 @@ func fetchStats(cl *http.Client, baseURL, accessToken string) (string, error) {
}
return string(b), nil
}
const (
// minPutTimeout is the floor we clamp to for small objects where the time is
// dominated by fixed overheads like connection establishment, waiting for a
// busy server to service the request etc.
minPutTimeout = 5 * time.Second
// maxPutTimeout is the ceiling we clamp to for large objects.
maxPutTimeout = 30 * time.Second
// minAverageBandwidth is the minimum average bandwidth (2MiB/s) we require
// for PUTs to complete within the timeout in its linear scaling region.
minAverageBandwidth = 2 * 1 << 20 / float64(time.Second)
)
// asyncPutTimeout returns a size-dependent timeout for async PUTs to the remote
// gocached server. It returns 5s for size <= 10MiB, 30s for size >= 60MiB and
// scales linearly in between.
func asyncPutTimeout(size int64) time.Duration {
timeout := time.Duration(float64(size) / minAverageBandwidth)
return min(max(minPutTimeout, timeout), maxPutTimeout)
}
-25
View File
@@ -1,25 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"testing"
"time"
)
func TestAsyncPutTimeout(t *testing.T) {
for size, expected := range map[int64]time.Duration{
0: 5 * time.Second,
10: 5 * time.Second,
10 * 1 << 20: 5 * time.Second,
20 * 1 << 20: 10 * time.Second,
40 * 1 << 20: 20 * time.Second,
60 * 1 << 20: 30 * time.Second,
10 * 1 << 30: 30 * time.Second,
} {
if actual := asyncPutTimeout(size); actual != expected {
t.Errorf("for size %d, expected %v, but got %v", size, expected, actual)
}
}
}
+1 -1
View File
@@ -169,7 +169,7 @@ func gen(buf *bytes.Buffer, it *codegen.ImportTracker, typ *types.Named) {
writef("}")
case *types.Map:
elem := ft.Elem()
if sliceType, isSlice := elem.Underlying().(*types.Slice); isSlice {
if sliceType, isSlice := elem.(*types.Slice); isSlice {
n := it.QualifiedName(sliceType.Elem())
writef("if dst.%s != nil {", fname)
writef("\tdst.%s = map[%s]%s{}", fname, it.QualifiedName(ft.Key()), it.QualifiedName(elem))
-10
View File
@@ -283,13 +283,3 @@ func TestDeeplyNestedMap(t *testing.T) {
t.Errorf("Clone() aliased FourLevels map: new nested key appeared in original")
}
}
func TestMapWithNamedSliceValues(t *testing.T) {
orig := &clonerex.MapWithNamedSliceValues{
M: map[string]clonerex.NamedSlice{"k": {"foo", "bar"}},
}
cloned := orig.Clone()
if diff := cmp.Diff(orig, cloned); diff != "" {
t.Errorf("Clone() mismatch (-orig +cloned):\n%s", diff)
}
}
+4 -13
View File
@@ -1,13 +1,11 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:generate go run tailscale.com/cmd/cloner -clonefunc=true -type SliceContainer,InterfaceContainer,MapWithPointers,DeeplyNestedMap,NamedMapContainer,MapSlicePointerContainer,MapWithNamedSliceValues
//go:generate go run tailscale.com/cmd/cloner -clonefunc=true -type SliceContainer,InterfaceContainer,MapWithPointers,DeeplyNestedMap,NamedMapContainer,MapSlicePointerContainer
// Package clonerex is an example package for the cloner tool.
package clonerex
import "maps"
type SliceContainer struct {
Slice []*int
}
@@ -51,7 +49,9 @@ func (m NamedMap) Clone() NamedMap {
return nil
}
m2 := make(NamedMap, len(m))
maps.Copy(m2, m)
for k, v := range m {
m2[k] = v
}
return m2
}
@@ -72,12 +72,3 @@ type DeeplyNestedMap struct {
ThreeLevels map[string]map[string]map[string]int
FourLevels map[string]map[string]map[string]map[string]*SliceContainer
}
// MapWithNamedSliceValues has a map with a named slice type for values. This
// tests that the generator treats these values like any other slice and not a
// struct.
type MapWithNamedSliceValues struct {
M map[string]NamedSlice
}
type NamedSlice []string
+1 -32
View File
@@ -209,31 +209,9 @@ var _MapSlicePointerContainerCloneNeedsRegeneration = MapSlicePointerContainer(s
Routes map[string][]*SliceContainer
}{})
// Clone makes a deep copy of MapWithNamedSliceValues.
// The result aliases no memory with the original.
func (src *MapWithNamedSliceValues) Clone() *MapWithNamedSliceValues {
if src == nil {
return nil
}
dst := new(MapWithNamedSliceValues)
*dst = *src
if dst.M != nil {
dst.M = map[string]NamedSlice{}
for k := range src.M {
dst.M[k] = append([]string{}, src.M[k]...)
}
}
return dst
}
// A compilation failure here means this code must be regenerated, with the command at the top of this file.
var _MapWithNamedSliceValuesCloneNeedsRegeneration = MapWithNamedSliceValues(struct {
M map[string]NamedSlice
}{})
// Clone duplicates src into dst and reports whether it succeeded.
// To succeed, <src, dst> must be of types <*T, *T> or <*T, **T>,
// where T is one of SliceContainer,InterfaceContainer,MapWithPointers,DeeplyNestedMap,NamedMapContainer,MapSlicePointerContainer,MapWithNamedSliceValues.
// where T is one of SliceContainer,InterfaceContainer,MapWithPointers,DeeplyNestedMap,NamedMapContainer,MapSlicePointerContainer.
func Clone(dst, src any) bool {
switch src := src.(type) {
case *SliceContainer:
@@ -290,15 +268,6 @@ func Clone(dst, src any) bool {
*dst = src.Clone()
return true
}
case *MapWithNamedSliceValues:
switch dst := dst.(type) {
case *MapWithNamedSliceValues:
*dst = *src.Clone()
return true
case **MapWithNamedSliceValues:
*dst = src.Clone()
return true
}
}
return false
}
+41 -68
View File
@@ -27,7 +27,7 @@ import (
"tailscale.com/kube/egressservices"
"tailscale.com/kube/kubeclient"
"tailscale.com/kube/kubetypes"
"tailscale.com/types/views"
"tailscale.com/types/netmap"
"tailscale.com/util/httpm"
"tailscale.com/util/linuxfw"
"tailscale.com/util/mak"
@@ -55,10 +55,9 @@ type egressProxy struct {
tsClient *local.Client // never nil
netmapChan chan netmapState // chan to receive netmap state updates on
netmapChan chan *netmap.NetworkMap // chan to receive netmap updates on
podIPv4 string // empty if Pod does not have IPv4 address
podIPv6 string // empty if Pod does not have IPv6 address
podIPv4 string // never empty string, currently only IPv4 is supported
// tailnetFQDNs is the egress service FQDN to tailnet IP mappings that
// were last used to configure firewall rules for this proxy.
@@ -88,7 +87,7 @@ type httpClient interface {
// - the mounted egress config has changed
// - the proxy's tailnet IP addresses have changed
// - tailnet IPs have changed for any backend targets specified by tailnet FQDN
func (ep *egressProxy) run(ctx context.Context, nm netmapState, opts egressProxyRunOpts) error {
func (ep *egressProxy) run(ctx context.Context, nm *netmap.NetworkMap, opts egressProxyRunOpts) error {
ep.configure(opts)
var tickChan <-chan time.Time
var eventChan <-chan fsnotify.Event
@@ -137,9 +136,8 @@ type egressProxyRunOpts struct {
kc kubeclient.Client
tsClient *local.Client
stateSecret string
netmapChan chan netmapState
netmapChan chan *netmap.NetworkMap
podIPv4 string
podIPv6 string
tailnetAddrs []netip.Prefix
}
@@ -152,7 +150,6 @@ func (ep *egressProxy) configure(opts egressProxyRunOpts) {
ep.stateSecret = opts.stateSecret
ep.netmapChan = opts.netmapChan
ep.podIPv4 = opts.podIPv4
ep.podIPv6 = opts.podIPv6
ep.tailnetAddrs = opts.tailnetAddrs
ep.client = &http.Client{} // default HTTP client
sleepDuration := time.Second
@@ -168,7 +165,7 @@ func (ep *egressProxy) configure(opts egressProxyRunOpts) {
// any firewall rules need to be updated. Currently using status in state Secret as a reference for what is the current
// firewall configuration is good enough because - the status is keyed by the Pod IP - we crash the Pod on errors such
// as failed firewall update
func (ep *egressProxy) sync(ctx context.Context, nm netmapState) error {
func (ep *egressProxy) sync(ctx context.Context, nm *netmap.NetworkMap) error {
cfgs, err := ep.getConfigs()
if err != nil {
return fmt.Errorf("error retrieving egress service configs: %w", err)
@@ -189,15 +186,16 @@ func (ep *egressProxy) sync(ctx context.Context, nm netmapState) error {
return nil
}
// addrsHaveChanged returns true if the provided netmap state contains tailnet address change for this proxy node.
func (ep *egressProxy) addrsHaveChanged(nm netmapState) bool {
return !views.SliceEqual(views.SliceOf(ep.tailnetAddrs), nm.self.Addresses())
// addrsHaveChanged returns true if the provided netmap update contains tailnet address change for this proxy node.
// Netmap must not be nil.
func (ep *egressProxy) addrsHaveChanged(nm *netmap.NetworkMap) bool {
return !reflect.DeepEqual(ep.tailnetAddrs, nm.SelfNode.Addresses())
}
// syncEgressConfigs adds and deletes firewall rules to match the desired
// configuration. It uses the provided status to determine what is currently
// applied and updates the status after a successful sync.
func (ep *egressProxy) syncEgressConfigs(cfgs egressservices.Configs, status *egressservices.Status, nm netmapState) (*egressservices.Status, error) {
func (ep *egressProxy) syncEgressConfigs(cfgs egressservices.Configs, status *egressservices.Status, nm *netmap.NetworkMap) (*egressservices.Status, error) {
if !(wantsServicesConfigured(cfgs) || hasServicesConfigured(status)) {
return nil, nil
}
@@ -236,7 +234,7 @@ func (ep *egressProxy) syncEgressConfigs(cfgs egressservices.Configs, status *eg
// family.
for _, t := range tailnetTargetIPs {
var local netip.Addr
for _, pfx := range nm.self.Addresses().All() {
for _, pfx := range nm.SelfNode.Addresses().All() {
if !pfx.IsSingleIP() {
continue
}
@@ -252,9 +250,6 @@ func (ep *egressProxy) syncEgressConfigs(cfgs egressservices.Configs, status *eg
if err := ep.nfr.EnsureSNATForDst(local, t); err != nil {
return nil, fmt.Errorf("error setting up SNAT rule: %w", err)
}
if err := ep.nfr.ClampMSSToPMTU(tailscaleTunInterface, t); err != nil {
return nil, fmt.Errorf("error clamping MSS to PMTU: %w", err)
}
}
}
// Update the status. Status will be written back to the state Secret by the caller.
@@ -421,7 +416,7 @@ func (ep *egressProxy) getStatus(ctx context.Context) (*egressservices.Status, e
if err := json.Unmarshal([]byte(raw), status); err != nil {
return nil, fmt.Errorf("error unmarshalling previous config: %w", err)
}
if status.PodIPv4 == ep.podIPv4 && status.PodIPv6 == ep.podIPv6 {
if reflect.DeepEqual(status.PodIPv4, ep.podIPv4) {
return status, nil
}
return nil, nil
@@ -429,13 +424,12 @@ func (ep *egressProxy) getStatus(ctx context.Context) (*egressservices.Status, e
// setStatus writes egress proxy's currently configured firewall to the state
// Secret and updates proxy's tailnet addresses.
func (ep *egressProxy) setStatus(ctx context.Context, status *egressservices.Status, nm netmapState) error {
func (ep *egressProxy) setStatus(ctx context.Context, status *egressservices.Status, nm *netmap.NetworkMap) error {
// Pod IP is used to determine if a stored status applies to THIS proxy Pod.
if status == nil {
status = &egressservices.Status{}
}
status.PodIPv4 = ep.podIPv4
status.PodIPv6 = ep.podIPv6
secret, err := ep.kc.GetSecret(ctx, ep.stateSecret)
if err != nil {
return fmt.Errorf("error retrieving state Secret: %w", err)
@@ -453,7 +447,7 @@ func (ep *egressProxy) setStatus(ctx context.Context, status *egressservices.Sta
if err := ep.kc.JSONPatchResource(ctx, ep.stateSecret, kubeclient.TypeSecrets, []kubeclient.JSONPatch{patch}); err != nil {
return fmt.Errorf("error patching state Secret: %w", err)
}
ep.tailnetAddrs = nm.self.Addresses().AsSlice()
ep.tailnetAddrs = nm.SelfNode.Addresses().AsSlice()
return nil
}
@@ -463,7 +457,7 @@ func (ep *egressProxy) setStatus(ctx context.Context, status *egressservices.Sta
// FQDN, resolve the FQDN and return the resolved IPs. It checks if the
// netfilter runner supports IPv6 NAT and skips any IPv6 addresses if it
// doesn't.
func (ep *egressProxy) tailnetTargetIPsForSvc(svc egressservices.Config, nm netmapState) (addrs []netip.Addr, err error) {
func (ep *egressProxy) tailnetTargetIPsForSvc(svc egressservices.Config, nm *netmap.NetworkMap) (addrs []netip.Addr, err error) {
if svc.TailnetTarget.IP != "" {
addr, err := netip.ParseAddr(svc.TailnetTarget.IP)
if err != nil {
@@ -479,8 +473,8 @@ func (ep *egressProxy) tailnetTargetIPsForSvc(svc egressservices.Config, nm netm
if svc.TailnetTarget.FQDN == "" {
return nil, errors.New("unexpected egress service config- neither tailnet target IP nor FQDN is set")
}
if !nm.self.Valid() {
log.Printf("netmap state is not available, unable to determine backend addresses for %s", svc.TailnetTarget.FQDN)
if nm == nil {
log.Printf("netmap is not available, unable to determine backend addresses for %s", svc.TailnetTarget.FQDN)
return addrs, nil
}
egressAddrs, err := resolveTailnetFQDN(nm, svc.TailnetTarget.FQDN)
@@ -507,26 +501,26 @@ func (ep *egressProxy) tailnetTargetIPsForSvc(svc egressservices.Config, nm netm
return addrs, nil
}
// shouldResync parses netmap state update and returns true if the update contains
// shouldResync parses netmap update and returns true if the update contains
// changes for which the egress proxy's firewall should be reconfigured.
func (ep *egressProxy) shouldResync(nm netmapState) bool {
if !nm.self.Valid() {
func (ep *egressProxy) shouldResync(nm *netmap.NetworkMap) bool {
if nm == nil {
return false
}
// If proxy's tailnet addresses have changed, resync.
if !views.SliceEqual(nm.self.Addresses(), views.SliceOf(ep.tailnetAddrs)) {
if !reflect.DeepEqual(nm.SelfNode.Addresses().AsSlice(), ep.tailnetAddrs) {
log.Printf("node addresses have changed, trigger egress config resync")
ep.tailnetAddrs = nm.self.Addresses().AsSlice()
ep.tailnetAddrs = nm.SelfNode.Addresses().AsSlice()
return true
}
// If the IPs for any of the egress services configured via FQDN have
// changed, resync.
for fqdn, ips := range ep.targetFQDNs {
for nn := range nm.peers() {
for _, nn := range nm.Peers {
if equalFQDNs(nn.Name(), fqdn) {
if !views.SliceEqual(views.SliceOf(ips), nn.Addresses()) {
if !reflect.DeepEqual(ips, nn.Addresses().AsSlice()) {
log.Printf("backend addresses for egress target %q have changed old IPs %v, new IPs %v trigger egress config resync", nn.Name(), ips, nn.Addresses().AsSlice())
return true
}
@@ -626,8 +620,6 @@ func servicesStatusIsEqual(st, st1 *egressservices.Status) bool {
}
st.PodIPv4 = ""
st1.PodIPv4 = ""
st.PodIPv6 = ""
st1.PodIPv6 = ""
return reflect.DeepEqual(*st, *st1)
}
@@ -679,29 +671,24 @@ func (ep *egressProxy) waitTillSafeToShutdown(ctx context.Context, cfgs egressse
continue
}
svc := s
// TODO(beckypauley): In dual-stack clusters, this is a best-effort check as we do not control which IP family is used.
// This confirms removal from routing on this node for one family only. The other IP family then relies on the longSleep below.
wg.Go(func() {
log.Printf("Ensuring that cluster traffic is no longer routed to %q via this Pod...", svc)
podIP, header := ep.podIPv4, kubetypes.PodIPv4Header
if podIP == "" {
podIP, header = ep.podIPv6, kubetypes.PodIPv6Header
}
if ep.podDrained(ctx, svc, hep, podIP, header, hp) {
return
}
ticker := time.NewTicker(ep.shortSleep)
defer ticker.Stop()
for {
select {
case <-ctx.Done(): // kubelet's HTTP request timeout
if ctx.Err() != nil { // kubelet's HTTP request timeout
log.Printf("Cluster traffic for %s did not stop being routed to this Pod.", svc)
return
case <-ticker.C:
if ep.podDrained(ctx, svc, hep, podIP, header, hp) {
return
}
}
found, err := lookupPodRoute(ctx, hep, ep.podIPv4, hp, ep.client)
if err != nil {
log.Printf("unable to reach endpoint %q, assuming the routing rules for this Pod have been deleted: %v", hep, err)
break
}
if !found {
log.Printf("service %q is no longer routed through this Pod", svc)
break
}
log.Printf("service %q is still routed through this Pod, waiting...", svc)
time.Sleep(ep.shortSleep)
}
})
}
@@ -715,9 +702,9 @@ func (ep *egressProxy) waitTillSafeToShutdown(ctx context.Context, cfgs egressse
// lookupPodRoute calls the healthcheck endpoint repeat times and returns true if the endpoint returns with the podIP
// header at least once.
func lookupPodRoute(ctx context.Context, hep, podIP, podIPHeader string, repeat int, client httpClient) (bool, error) {
func lookupPodRoute(ctx context.Context, hep, podIP string, repeat int, client httpClient) (bool, error) {
for range repeat {
f, err := lookup(ctx, hep, podIP, podIPHeader, client)
f, err := lookup(ctx, hep, podIP, client)
if err != nil {
return false, err
}
@@ -729,7 +716,7 @@ func lookupPodRoute(ctx context.Context, hep, podIP, podIPHeader string, repeat
}
// lookup calls the healthcheck endpoint and returns true if the response contains the podIP header.
func lookup(ctx context.Context, hep, podIP, podIPHeader string, client httpClient) (bool, error) {
func lookup(ctx context.Context, hep, podIP string, client httpClient) (bool, error) {
req, err := http.NewRequestWithContext(ctx, httpm.GET, hep, nil)
if err != nil {
return false, fmt.Errorf("error creating new HTTP request: %v", err)
@@ -744,7 +731,7 @@ func lookup(ctx context.Context, hep, podIP, podIPHeader string, client httpClie
return true, nil
}
defer resp.Body.Close()
gotIP := resp.Header.Get(podIPHeader)
gotIP := resp.Header.Get(kubetypes.PodIPv4Header)
return strings.EqualFold(podIP, gotIP), nil
}
@@ -773,17 +760,3 @@ func (ep *egressProxy) getHEPPings() (int, error) {
}
return hp, nil
}
func (ep *egressProxy) podDrained(ctx context.Context, svc, hep, podIP, header string, hp int) bool {
found, err := lookupPodRoute(ctx, hep, podIP, header, hp, ep.client)
if err != nil {
log.Printf("unable to reach endpoint %q, assuming the routing rules for this Pod have been deleted: %v", hep, err)
return true
}
if !found {
log.Printf("service %q is no longer routed through this Pod", svc)
return true
}
log.Printf("service %q is still routed through this Pod, waiting...", svc)
return false
}
+1 -3
View File
@@ -15,7 +15,6 @@ import (
"strings"
"sync"
"testing"
"time"
"tailscale.com/kube/egressservices"
"tailscale.com/kube/kubetypes"
@@ -270,8 +269,7 @@ func TestWaitTillSafeToShutdown(t *testing.T) {
}
ep := &egressProxy{
podIPv4: podIP,
shortSleep: time.Millisecond,
podIPv4: podIP,
client: &mockHTTPClient{
podIP: podIP,
anotherIP: anotherIP,
+1 -7
View File
@@ -265,13 +265,7 @@ func ensureIngressRulesAdded(cfgs map[string]ingressservices.Config, nfr linuxfw
func addDNATRuleForSvc(nfr linuxfw.NetfilterRunner, serviceName string, tsIP, clusterIP netip.Addr) error {
log.Printf("adding DNAT rule for Tailscale Service %s with IP %s to Kubernetes Service IP %s", serviceName, tsIP, clusterIP)
if err := nfr.EnsureDNATRuleForSvc(serviceName, tsIP, clusterIP); err != nil {
return err
}
if err := nfr.ClampMSSToPMTU(tailscaleTunInterface, clusterIP); err != nil {
return fmt.Errorf("error clamping MSS to PMTU: %w", err)
}
return nil
return nfr.EnsureDNATRuleForSvc(serviceName, tsIP, clusterIP)
}
// ensureIngressRulesDeleted takes a map of Tailscale Services and rules and ensures that the firewall rules are deleted.
+3 -39
View File
@@ -7,7 +7,6 @@ package main
import (
"net/netip"
"slices"
"testing"
"tailscale.com/kube/ingressservices"
@@ -23,7 +22,6 @@ func TestSyncIngressConfigs(t *testing.T) {
TailscaleServiceIP netip.Addr
ClusterIP netip.Addr
}
wantClampedAddrs []netip.Addr // cluster IPs that should have MSS clamping applied
}{
{
name: "add_new_rules_when_no_existing_config",
@@ -37,7 +35,6 @@ func TestSyncIngressConfigs(t *testing.T) {
}{
"svc:foo": makeWantService("100.64.0.1", "10.0.0.1"),
},
wantClampedAddrs: []netip.Addr{netip.MustParseAddr("10.0.0.1")},
},
{
name: "add_multiple_services",
@@ -55,11 +52,6 @@ func TestSyncIngressConfigs(t *testing.T) {
"svc:bar": makeWantService("100.64.0.2", "10.0.0.2"),
"svc:baz": makeWantService("100.64.0.3", "10.0.0.3"),
},
wantClampedAddrs: []netip.Addr{
netip.MustParseAddr("10.0.0.1"),
netip.MustParseAddr("10.0.0.2"),
netip.MustParseAddr("10.0.0.3"),
},
},
{
name: "add_both_ipv4_and_ipv6_rules",
@@ -73,10 +65,6 @@ func TestSyncIngressConfigs(t *testing.T) {
}{
"svc:foo": makeWantService("2001:db8::1", "2001:db8::2"),
},
wantClampedAddrs: []netip.Addr{
netip.MustParseAddr("10.0.0.1"),
netip.MustParseAddr("2001:db8::2"),
},
},
{
name: "add_ipv6_only_rules",
@@ -90,7 +78,6 @@ func TestSyncIngressConfigs(t *testing.T) {
}{
"svc:ipv6": makeWantService("2001:db8::10", "2001:db8::20"),
},
wantClampedAddrs: []netip.Addr{netip.MustParseAddr("2001:db8::20")},
},
{
name: "delete_all_rules_when_config_removed",
@@ -107,7 +94,6 @@ func TestSyncIngressConfigs(t *testing.T) {
TailscaleServiceIP netip.Addr
ClusterIP netip.Addr
}{},
wantClampedAddrs: nil, // no rules added, no clamping
},
{
name: "add_remove_modify",
@@ -131,10 +117,6 @@ func TestSyncIngressConfigs(t *testing.T) {
"svc:foo": makeWantService("100.64.0.1", "10.0.0.2"),
"svc:new": makeWantService("100.64.0.4", "10.0.0.4"),
},
wantClampedAddrs: []netip.Addr{
netip.MustParseAddr("10.0.0.2"),
netip.MustParseAddr("10.0.0.4"),
},
},
{
name: "update_with_outdated_status",
@@ -170,17 +152,12 @@ func TestSyncIngressConfigs(t *testing.T) {
"svc:web-ipv6": makeWantService("2001:db8::10", "2001:db8::20"),
"svc:api": makeWantService("100.64.0.20", "10.0.0.20"),
},
wantClampedAddrs: []netip.Addr{
netip.MustParseAddr("10.0.0.10"),
netip.MustParseAddr("10.0.0.20"),
netip.MustParseAddr("2001:db8::20"),
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
nfr := linuxfw.NewFakeNetfilterRunner()
var nfr linuxfw.NetfilterRunner = linuxfw.NewFakeNetfilterRunner()
ep := &ingressProxy{
nfr: nfr,
@@ -193,7 +170,8 @@ func TestSyncIngressConfigs(t *testing.T) {
t.Fatalf("syncIngressConfigs failed: %v", err)
}
gotServices := nfr.GetServiceState()
fake := nfr.(*linuxfw.FakeNetfilterRunner)
gotServices := fake.GetServiceState()
if len(gotServices) != len(tt.wantServices) {
t.Errorf("got %d services, want %d", len(gotServices), len(tt.wantServices))
}
@@ -210,20 +188,6 @@ func TestSyncIngressConfigs(t *testing.T) {
t.Errorf("service %s: got ClusterIP %v, want %v", svc, got.ClusterIP, want.ClusterIP)
}
}
gotClamped := nfr.GetClampedAddrs()
slices.SortFunc(gotClamped, func(a, b netip.Addr) int { return a.Compare(b) })
slices.SortFunc(tt.wantClampedAddrs, func(a, b netip.Addr) int { return a.Compare(b) })
if len(gotClamped) != len(tt.wantClampedAddrs) {
t.Errorf("ClampMSSToPMTU: got %v, want %v", gotClamped, tt.wantClampedAddrs)
} else {
for i := range gotClamped {
if gotClamped[i] != tt.wantClampedAddrs[i] {
t.Errorf("ClampMSSToPMTU: got %v, want %v", gotClamped, tt.wantClampedAddrs)
break
}
}
}
})
}
}
+55 -212
View File
@@ -120,7 +120,6 @@ import (
"errors"
"fmt"
"io/fs"
"iter"
"log"
"math"
"net"
@@ -136,13 +135,11 @@ import (
"syscall"
"time"
"github.com/benbjohnson/immutable"
"golang.org/x/sys/unix"
"tailscale.com/client/local"
"tailscale.com/health"
"tailscale.com/ipn"
"tailscale.com/ipn/ipnstate"
kubeutils "tailscale.com/k8s-operator"
"tailscale.com/kube/authkey"
healthz "tailscale.com/kube/health"
@@ -152,170 +149,21 @@ import (
"tailscale.com/kube/services"
"tailscale.com/tailcfg"
"tailscale.com/types/logger"
"tailscale.com/types/views"
"tailscale.com/types/netmap"
"tailscale.com/util/deephash"
"tailscale.com/util/def"
"tailscale.com/util/dnsname"
"tailscale.com/util/linuxfw"
)
func newNetfilterRunner(logf logger.Logf) (linuxfw.NetfilterRunner, error) {
if def.Bool(os.Getenv("TS_TEST_FAKE_NETFILTER"), false) {
if defaultBool("TS_TEST_FAKE_NETFILTER", false) {
return linuxfw.NewFakeIPTablesRunner(), nil
}
return linuxfw.New(logf, "")
}
func getAutoAdvertiseBool() bool {
return def.Bool(os.Getenv("TS_EXPERIMENTAL_SERVICE_AUTO_ADVERTISEMENT"), true)
}
const containerbootWatchMask = ipn.NotifyInitialStatus |
ipn.NotifyPeerChanges |
ipn.NotifyNoNetMap
func notifyState(n ipn.Notify) (_ ipn.State, ok bool) {
if n.State != nil {
return *n.State, true
}
if n.InitialStatus != nil && n.InitialStatus.BackendState != "" {
if state, ok := ipn.StateFromString(n.InitialStatus.BackendState); ok {
return state, true
}
}
return ipn.NoState, false
}
var netmapStatePeerIDHasher = immutable.NewHasher(tailcfg.NodeID(0))
type netmapState struct {
self tailcfg.NodeView
peersByID *immutable.Map[tailcfg.NodeID, tailcfg.NodeView]
peersByName *immutable.Map[string, tailcfg.NodeView] // keyed by tailcfg.Node.Name when NodeID is unavailable
certDomains views.Slice[string]
dnsExtraRecords views.Slice[tailcfg.DNSRecord]
}
func (s netmapState) updateFromNotify(n ipn.Notify) netmapState {
if n.InitialStatus != nil {
s = s.updateFromStatus(n.InitialStatus)
}
if n.SelfChange != nil {
s.self = n.SelfChange.View()
}
for _, p := range n.PeersChanged {
s = s.upsertPeer(p.View())
}
for _, id := range n.PeersRemoved {
if s.peersByID != nil {
s.peersByID = s.peersByID.Delete(id)
}
}
return s
}
// processNotify updates the netmap state from an IPN bus Notify. On
// SelfChange it also refetches DNS via the LocalAPI dns-config
// endpoint; the bus carries no DNS delta.
func (s netmapState) processNotify(ctx context.Context, client *local.Client, n ipn.Notify) netmapState {
s = s.updateFromNotify(n)
if n.SelfChange != nil {
dns, err := client.DNSConfig(ctx)
if err != nil {
log.Printf("error refreshing DNS config from tailscaled: %v", err)
} else if dns != nil {
s.dnsExtraRecords = views.SliceOf(dns.ExtraRecords)
s.certDomains = views.SliceOf(dns.CertDomains)
}
}
return s
}
func (s netmapState) updateFromStatus(st *ipnstate.Status) netmapState {
s.certDomains = views.SliceOf(st.CertDomains)
s.dnsExtraRecords = views.SliceOf(st.ExtraRecords)
if st.Self != nil {
s.self = nodeFromPeerStatus(st.Self).View()
}
if len(st.Peer) != 0 {
s.peersByID = nil
s.peersByName = nil
for _, ps := range st.Peer {
s = s.upsertPeer(nodeFromPeerStatus(ps).View())
}
}
return s
}
func (s netmapState) upsertPeer(n tailcfg.NodeView) netmapState {
if !n.Valid() {
return s
}
if s.peersByID == nil {
s.peersByID = immutable.NewMap[tailcfg.NodeID, tailcfg.NodeView](netmapStatePeerIDHasher)
}
if s.peersByName == nil {
s.peersByName = immutable.NewMap[string, tailcfg.NodeView](nil)
}
if n.ID() != 0 {
s.peersByID = s.peersByID.Set(n.ID(), n)
if name := n.Name(); name != "" {
s.peersByName = s.peersByName.Delete(name)
}
return s
}
if n.Name() != "" {
s.peersByName = s.peersByName.Set(n.Name(), n)
}
return s
}
func nodeFromPeerStatus(ps *ipnstate.PeerStatus) *tailcfg.Node {
if ps == nil {
return nil
}
n := &tailcfg.Node{
ID: ps.NodeID,
StableID: ps.ID,
Name: ps.DNSName,
Key: ps.PublicKey,
}
for _, ip := range ps.TailscaleIPs {
n.Addresses = append(n.Addresses, netip.PrefixFrom(ip, ip.BitLen()))
}
if ps.AllowedIPs != nil {
n.AllowedIPs = ps.AllowedIPs.AsSlice()
}
return n
}
func (s netmapState) peers() iter.Seq[tailcfg.NodeView] {
return func(yield func(tailcfg.NodeView) bool) {
if s.peersByID != nil {
it := s.peersByID.Iterator()
for {
_, p, ok := it.Next()
if !ok {
break
}
if !yield(p) {
return
}
}
}
if s.peersByName != nil {
it := s.peersByName.Iterator()
for {
_, p, ok := it.Next()
if !ok {
break
}
if !yield(p) {
return
}
}
}
}
return defaultBool("TS_EXPERIMENTAL_SERVICE_AUTO_ADVERTISEMENT", true)
}
func main() {
@@ -424,7 +272,7 @@ func run() error {
mux := http.NewServeMux()
log.Printf("Running healthcheck endpoint at %s/healthz", cfg.HealthCheckAddrPort)
healthCheck = healthz.RegisterHealthHandlers(mux, cfg.PodIPv4, cfg.PodIPv6, log.Printf)
healthCheck = healthz.RegisterHealthHandlers(mux, cfg.PodIPv4, log.Printf)
close := runHTTPServer(mux, cfg.HealthCheckAddrPort)
defer close()
@@ -440,7 +288,7 @@ func run() error {
if cfg.localHealthEnabled() {
log.Printf("Running healthcheck endpoint at %s/healthz", cfg.LocalAddrPort)
healthCheck = healthz.RegisterHealthHandlers(mux, cfg.PodIPv4, cfg.PodIPv6, log.Printf)
healthCheck = healthz.RegisterHealthHandlers(mux, cfg.PodIPv4, log.Printf)
}
if cfg.egressSvcsTerminateEPEnabled() {
@@ -458,7 +306,7 @@ func run() error {
}
}
w, err := client.WatchIPNBus(bootCtx, containerbootWatchMask|ipn.NotifyInitialPrefs|ipn.NotifyInitialHealthState)
w, err := client.WatchIPNBus(bootCtx, ipn.NotifyInitialNetMap|ipn.NotifyInitialPrefs|ipn.NotifyInitialState|ipn.NotifyInitialHealthState|ipn.NotifyRateLimit)
if err != nil {
return fmt.Errorf("failed to watch tailscaled for updates: %w", err)
}
@@ -498,7 +346,7 @@ func run() error {
if err := tailscaleUp(bootCtx, cfg); err != nil {
return fmt.Errorf("failed to auth tailscale: %w", err)
}
w, err = client.WatchIPNBus(bootCtx, containerbootWatchMask)
w, err = client.WatchIPNBus(bootCtx, ipn.NotifyInitialNetMap|ipn.NotifyInitialState|ipn.NotifyRateLimit)
if err != nil {
return fmt.Errorf("rewatching tailscaled for updates after auth: %w", err)
}
@@ -518,8 +366,8 @@ authLoop:
return fmt.Errorf("failed to read from tailscaled: %w", err)
}
if state, ok := notifyState(n); ok {
switch state {
if n.State != nil {
switch *n.State {
case ipn.NeedsLogin:
if isOneStepConfig(cfg) {
// This could happen if this is the first time tailscaled was run for this
@@ -555,7 +403,7 @@ authLoop:
// deadline to continue monitoring for changes.
break authLoop
default:
log.Printf("tailscaled in state %q, waiting", state)
log.Printf("tailscaled in state %q, waiting", *n.State)
}
}
@@ -610,7 +458,7 @@ authLoop:
}
}
w, err = client.WatchIPNBus(ctx, containerbootWatchMask)
w, err = client.WatchIPNBus(ctx, ipn.NotifyInitialNetMap|ipn.NotifyInitialState|ipn.NotifyRateLimit)
if err != nil {
return fmt.Errorf("rewatching tailscaled for updates after auth: %w", err)
}
@@ -689,7 +537,7 @@ authLoop:
failedResolveAttempts++
}
var egressSvcsNotify chan netmapState
var egressSvcsNotify chan *netmap.NetworkMap
notifyChan := make(chan ipn.Notify)
errChan := make(chan error)
go func() {
@@ -703,7 +551,12 @@ authLoop:
}
}
}()
var nmState netmapState
// Peer set changes (Add/Remove) no longer ride on the IPN bus; poll
// periodically so egress FQDN resolution and peer-aware work picks
// them up. SelfChange covers prompt self changes.
const peerPollInterval = 15 * time.Second
peerPoll := time.NewTicker(peerPollInterval)
defer peerPoll.Stop()
var wg sync.WaitGroup
runLoop:
@@ -721,17 +574,19 @@ runLoop:
return fmt.Errorf("failed to read from tailscaled: %w", err)
case err := <-cfgWatchErrChan:
return fmt.Errorf("failed to watch tailscaled config: %w", err)
case <-peerPoll.C:
processNetmap = true
case n := <-notifyChan:
nmState = nmState.processNotify(ctx, client, n)
if state, ok := notifyState(n); ok && state != ipn.Running {
// TODO: (ChaosInTheCRD) Add node removed check when supported by ipn
if n.State != nil && *n.State != ipn.Running {
// Something's gone wrong and we've left the authenticated state.
// Our container image never recovered gracefully from this, and the
// control flow required to make it work now is hard. So, just crash
// the container and rely on the container runtime to restart us,
// whereupon we'll go through initial auth again.
return fmt.Errorf("tailscaled left running state (now in state %q), exiting", state)
return fmt.Errorf("tailscaled left running state (now in state %q), exiting", *n.State)
}
if n.InitialStatus != nil || n.SelfChange != nil || len(n.PeersChanged) != 0 || len(n.PeersRemoved) != 0 || len(n.PeerChangedPatch) != 0 {
if n.SelfChange != nil {
processNetmap = true
}
case <-tc:
@@ -761,12 +616,13 @@ runLoop:
if !processNetmap {
continue
}
self := nmState.self
if !self.Valid() {
nm, err := fetchNetMap(ctx, client)
if err != nil {
log.Printf("error fetching netmap: %v", err)
continue
}
{
addrs = self.Addresses().AsSlice()
if nm != nil {
addrs = nm.SelfNode.Addresses().AsSlice()
newCurrentIPs := deephash.Hash(&addrs)
ipsHaveChanged := newCurrentIPs != currentIPs
@@ -778,14 +634,14 @@ runLoop:
// Kubernetes Secret to clean up tailnet nodes
// for proxies whose route setup continuously
// fails.
deviceID := self.StableID()
deviceID := nm.SelfNode.StableID()
if hasKubeStateStore(cfg) && deephash.Update(&currentDeviceID, &deviceID) {
if err := kc.storeDeviceID(ctx, deviceID); err != nil {
if err := kc.storeDeviceID(ctx, nm.SelfNode.StableID()); err != nil {
return fmt.Errorf("storing device ID in Kubernetes Secret: %w", err)
}
}
if cfg.TailnetTargetFQDN != "" {
egressAddrs, err := resolveTailnetFQDN(nmState, cfg.TailnetTargetFQDN)
egressAddrs, err := resolveTailnetFQDN(nm, cfg.TailnetTargetFQDN)
if err != nil {
log.Print(err.Error())
break
@@ -841,10 +697,7 @@ runLoop:
backendAddrs = newBackendAddrs
}
if cfg.ServeConfigPath != "" {
var cd string
if nmState.certDomains.Len() != 0 {
cd = nmState.certDomains.At(0)
}
cd := certDomainFromNetmap(nm)
if cd == "" {
cd = kubetypes.ValueNoHTTPS
}
@@ -887,9 +740,9 @@ runLoop:
// set up ensures that the operator does not
// advertize endpoints of broken proxies.
// TODO (irbekrm): instead of using the IP and FQDN, have some other mechanism for the proxy signal that it is 'Ready'.
deviceEndpoints := []any{self.Name(), self.Addresses()}
deviceEndpoints := []any{nm.SelfNode.Name(), nm.SelfNode.Addresses()}
if hasKubeStateStore(cfg) && deephash.Update(&currentDeviceEndpoints, &deviceEndpoints) {
if err := kc.storeDeviceEndpoints(ctx, self.Name(), addrs); err != nil {
if err := kc.storeDeviceEndpoints(ctx, nm.SelfNode.Name(), nm.SelfNode.Addresses().AsSlice()); err != nil {
return fmt.Errorf("storing device IPs and FQDN in Kubernetes Secret: %w", err)
}
}
@@ -918,7 +771,7 @@ runLoop:
}
if egressSvcsNotify != nil {
egressSvcsNotify <- nmState
egressSvcsNotify <- nm
}
}
if !startupTasksDone {
@@ -940,7 +793,7 @@ runLoop:
// will crash this node.
if cfg.EgressProxiesCfgPath != "" {
log.Printf("configuring egress proxy using configuration file at %s", cfg.EgressProxiesCfgPath)
egressSvcsNotify = make(chan netmapState)
egressSvcsNotify = make(chan *netmap.NetworkMap)
opts := egressProxyRunOpts{
cfgPath: cfg.EgressProxiesCfgPath,
nfr: nfr,
@@ -949,11 +802,10 @@ runLoop:
stateSecret: cfg.KubeSecret,
netmapChan: egressSvcsNotify,
podIPv4: cfg.PodIPv4,
podIPv6: cfg.PodIPv6,
tailnetAddrs: addrs,
}
go func() {
if err := ep.run(ctx, nmState, opts); err != nil {
if err := ep.run(ctx, nm, opts); err != nil {
egressSvcsErrorChan <- err
}
}()
@@ -1133,53 +985,44 @@ func runHTTPServer(mux *http.ServeMux, addr string) (close func() error) {
}
}
// fetchNetMap fetches the current netmap from tailscaled via the
// "current-netmap" localapi debug action. The debug action's payload
// shape is intentionally not part of any stable API; containerboot
// reads its own internal-package types out of it. New external consumers
// should not rely on this — see [local.Client.Status] and friends.
func fetchNetMap(ctx context.Context, lc *local.Client) (*netmap.NetworkMap, error) {
return local.GetDebugResultJSON[*netmap.NetworkMap](ctx, lc, "current-netmap")
}
// resolveTailnetFQDN resolves a tailnet FQDN to a list of IP prefixes, which
// can be either a peer device, a Tailscale Service, or a 4via6 synthesized
// DNS name (e.g. "10-1-0-5-via-7.tailnet.ts.net").
func resolveTailnetFQDN(nm netmapState, fqdn string) ([]netip.Prefix, error) {
// can be either a peer device or a Tailscale Service.
func resolveTailnetFQDN(nm *netmap.NetworkMap, fqdn string) ([]netip.Prefix, error) {
dnsFQDN, err := dnsname.ToFQDN(fqdn)
if err != nil {
return nil, fmt.Errorf("error parsing %q as FQDN: %w", fqdn, err)
}
// Check all peer devices first.
var ret []netip.Prefix
for p := range nm.peers() {
for _, p := range nm.Peers {
if strings.EqualFold(p.Name(), dnsFQDN.WithTrailingDot()) {
ret = p.Addresses().AsSlice()
break
return p.Addresses().AsSlice(), nil
}
}
if ret != nil {
return ret, nil
}
// If not found yet, check for a matching Tailscale Service.
if svcIPs := serviceIPsFromNetMap(nm, dnsFQDN); len(svcIPs) != 0 {
return svcIPs, nil
}
// If not found yet, check for a matching 4via6 DNS name.
if addr, ok := kubeutils.ResolveViaDomain(dnsFQDN.WithTrailingDot()); ok {
prefix := netip.PrefixFrom(addr, addr.BitLen())
for nn := range nm.peers() {
for _, allowedIP := range nn.AllowedIPs().All() {
if allowedIP.Contains(addr) {
return []netip.Prefix{prefix}, nil
}
}
}
return nil, fmt.Errorf("resolved 4via6 address %v for %q but no peer advertises a route containing it", addr, fqdn)
}
return nil, fmt.Errorf("could not find Tailscale node, service or 4via6 address %q; it either does not exist, or not reachable because of ACLs", fqdn)
return nil, fmt.Errorf("could not find Tailscale node or service %q; it either does not exist, or not reachable because of ACLs", fqdn)
}
// serviceIPsFromNetMap returns all IPs of a Tailscale Service if its FQDN is
// found in the netmap. Note that Tailscale Services are not a first-class
// object in the netmap, so we guess based on DNS ExtraRecords and AllowedIPs.
func serviceIPsFromNetMap(nm netmapState, fqdn dnsname.FQDN) []netip.Prefix {
func serviceIPsFromNetMap(nm *netmap.NetworkMap, fqdn dnsname.FQDN) []netip.Prefix {
var extraRecords []tailcfg.DNSRecord
for _, rec := range nm.dnsExtraRecords.All() {
for _, rec := range nm.DNS.ExtraRecords {
recFQDN, err := dnsname.ToFQDN(rec.Name)
if err != nil {
continue
@@ -1201,7 +1044,7 @@ func serviceIPsFromNetMap(nm netmapState, fqdn dnsname.FQDN) []netip.Prefix {
continue
}
ipPrefix := netip.PrefixFrom(ip, ip.BitLen())
for ps := range nm.peers() {
for _, ps := range nm.Peers {
for _, allowedIP := range ps.AllowedIPs().All() {
if allowedIP == ipPrefix {
prefixes = append(prefixes, ipPrefix)
+80 -123
View File
@@ -7,7 +7,6 @@ package main
import (
"bytes"
"context"
_ "embed"
"encoding/base64"
"encoding/json"
@@ -33,18 +32,15 @@ import (
"github.com/google/go-cmp/cmp"
"golang.org/x/sys/unix"
"tailscale.com/client/local"
"tailscale.com/cmd/testwrapper/flakytest"
"tailscale.com/health"
"tailscale.com/ipn"
"tailscale.com/ipn/ipnstate"
"tailscale.com/kube/egressservices"
"tailscale.com/kube/kubeclient"
"tailscale.com/kube/kubetypes"
"tailscale.com/net/memnet"
"tailscale.com/tailcfg"
"tailscale.com/tstest"
"tailscale.com/types/key"
"tailscale.com/types/netmap"
)
const configFileAuthKey = "some-auth-key"
@@ -56,7 +52,6 @@ func TestContainerBoot(t *testing.T) {
t.Fatalf("Building containerboot: %v", err)
}
egressStatus := egressSvcStatus("foo", "foo.tailnetxyz.ts.net", "100.64.0.2")
egressStatusUpdated := egressSvcStatus("foo", "foo.tailnetxyz.ts.net", "100.64.0.3")
metricsURL := func(port int) string {
return fmt.Sprintf("http://127.0.0.1:%d/metrics", port)
@@ -76,6 +71,12 @@ func TestContainerBoot(t *testing.T) {
// Waits below to be true before proceeding to the next phase.
Notify *ipn.Notify
// If non-nil, install this NetMap on the fake LocalAPI before
// sending Notify. This is the replacement for the old
// Notify.NetMap field; reactive consumers fetch the current
// netmap via /localapi/v0/netmap on their own.
NetMap *netmap.NetworkMap
// WantCmds is the commands that containerboot should run in this phase.
WantCmds []string
@@ -391,12 +392,19 @@ func TestContainerBoot(t *testing.T) {
Name: "test-node.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
},
PeersChanged: []*tailcfg.Node{
{
},
NetMap: &netmap.NetworkMap{
SelfNode: (&tailcfg.Node{
StableID: tailcfg.StableNodeID("myID"),
Name: "test-node.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
}).View(),
Peers: []tailcfg.NodeView{
(&tailcfg.Node{
StableID: tailcfg.StableNodeID("ipv6ID"),
Name: "ipv6-node.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("::1/128")},
},
}).View(),
},
},
WantLog: "no forwarding rules for egress addresses [::1/128], host supports IPv6: false",
@@ -638,6 +646,13 @@ func TestContainerBoot(t *testing.T) {
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
},
},
NetMap: &netmap.NetworkMap{
SelfNode: (&tailcfg.Node{
StableID: tailcfg.StableNodeID("newID"),
Name: "new-name.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
}).View(),
},
WantKubeSecret: map[string]string{
"authkey": "tskey-key",
"device_fqdn": "new-name.test.ts.net.",
@@ -1099,12 +1114,19 @@ func TestContainerBoot(t *testing.T) {
Name: "test-node.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
},
PeersChanged: []*tailcfg.Node{
{
},
NetMap: &netmap.NetworkMap{
SelfNode: (&tailcfg.Node{
StableID: tailcfg.StableNodeID("myID"),
Name: "test-node.test.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.1/32")},
}).View(),
Peers: []tailcfg.NodeView{
(&tailcfg.Node{
StableID: tailcfg.StableNodeID("fooID"),
Name: "foo.tailnetxyz.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.2/32")},
},
}).View(),
},
},
WantKubeSecret: map[string]string{
@@ -1119,23 +1141,6 @@ func TestContainerBoot(t *testing.T) {
egressSvcTerminateURL(env.localAddrPort): 200,
},
},
{
Notify: &ipn.Notify{
PeersChanged: []*tailcfg.Node{{
StableID: tailcfg.StableNodeID("fooID"),
Name: "foo.tailnetxyz.ts.net.",
Addresses: []netip.Prefix{netip.MustParsePrefix("100.64.0.3/32")},
}},
},
WantKubeSecret: map[string]string{
"egress-services": string(mustJSON(t, egressStatusUpdated)),
"authkey": "tskey-key",
"device_fqdn": "test-node.test.ts.net.",
"device_id": "myID",
"device_ips": `["100.64.0.1"]`,
kubetypes.KeyCapVer: capver,
},
},
},
}
},
@@ -1290,11 +1295,17 @@ func TestContainerBoot(t *testing.T) {
t.Fatalf("phase %d: updating mtime for %q: %v", i, path, err)
}
}
if p.Notify != nil && p.Notify.InitialStatus == nil {
// Shallow-copy before mutating to avoid a race with
// parallel subtests that share the same *ipn.Notify.
p.Notify = new(*p.Notify)
p.Notify.InitialStatus = statusFromNotify(p.Notify)
nmForFake := p.NetMap
if nmForFake == nil && p.Notify != nil && p.Notify.SelfChange != nil {
// Synthesize a minimal netmap from SelfChange so
// containerboot's NetMap() fetch returns
// something usable when the test only set Notify.
nmForFake = &netmap.NetworkMap{
SelfNode: p.Notify.SelfChange.View(),
}
}
if nmForFake != nil {
env.lapi.SetNetMap(nmForFake)
}
env.lapi.Notify(p.Notify)
if p.Signal != nil {
@@ -1488,6 +1499,7 @@ type localAPI struct {
sync.Mutex
cond *sync.Cond
notify *ipn.Notify
netmap *netmap.NetworkMap // served by /localapi/v0/netmap
}
func (lc *localAPI) Start() error {
@@ -1524,45 +1536,44 @@ func (lc *localAPI) Notify(n *ipn.Notify) {
lc.cond.Broadcast()
}
func statusFromNotify(n *ipn.Notify) *ipnstate.Status {
st := new(ipnstate.Status)
if n.State != nil {
st.BackendState = n.State.String()
}
if n.SelfChange != nil {
st.Self = peerStatusFromNode(n.SelfChange.View())
}
if len(n.PeersChanged) != 0 {
st.Peer = map[key.NodePublic]*ipnstate.PeerStatus{}
for _, p := range n.PeersChanged {
pv := p.View()
st.Peer[pv.Key()] = peerStatusFromNode(pv)
}
}
return st
}
func peerStatusFromNode(n tailcfg.NodeView) *ipnstate.PeerStatus {
ps := &ipnstate.PeerStatus{
ID: n.StableID(),
NodeID: n.ID(),
PublicKey: n.Key(),
DNSName: n.Name(),
}
for _, p := range n.Addresses().All() {
if p.IsSingleIP() {
ps.TailscaleIPs = append(ps.TailscaleIPs, p.Addr())
}
}
if n.AllowedIPs().Len() != 0 {
v := n.AllowedIPs()
ps.AllowedIPs = &v
}
return ps
// SetNetMap installs the netmap that the fake /localapi/v0/netmap endpoint
// will return.
func (lc *localAPI) SetNetMap(nm *netmap.NetworkMap) {
lc.Lock()
defer lc.Unlock()
lc.netmap = nm
}
func (lc *localAPI) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/localapi/v0/netmap":
w.Header().Set("Content-Type", "application/json")
lc.Lock()
nm := lc.netmap
lc.Unlock()
if nm == nil {
http.Error(w, "no netmap", http.StatusServiceUnavailable)
return
}
json.NewEncoder(w).Encode(nm)
return
case "/localapi/v0/debug":
// containerboot fetches the netmap via the "current-netmap"
// debug action; serve it like /localapi/v0/netmap above.
if r.URL.Query().Get("action") != "current-netmap" {
http.Error(w, "unsupported debug action", http.StatusNotFound)
return
}
w.Header().Set("Content-Type", "application/json")
lc.Lock()
nm := lc.netmap
lc.Unlock()
if nm == nil {
http.Error(w, "no netmap", http.StatusServiceUnavailable)
return
}
json.NewEncoder(w).Encode(nm)
return
case "/localapi/v0/serve-config":
switch r.Method {
case "GET":
@@ -1948,57 +1959,3 @@ func newTestEnv(t *testing.T) testEnv {
healthAddrPort: healthAddrPort,
}
}
// TestProcessNotifyRefreshesDNSOnSelfChange verifies that a SelfChange
// notification triggers a DNS refresh; without it, VIPServices created
// after pod boot are invisible to resolveTailnetFQDN.
func TestProcessNotifyRefreshesDNSOnSelfChange(t *testing.T) {
extraRec := tailcfg.DNSRecord{
Name: "my-ingress.tailnet.ts.net.",
Type: "A",
Value: "100.99.10.20",
}
dnsCfg := &tailcfg.DNSConfig{
ExtraRecords: []tailcfg.DNSRecord{extraRec},
CertDomains: []string{"node.tailnet.ts.net"},
}
lal := memnet.Listen("local-tailscaled.sock:80")
defer lal.Close()
mux := http.NewServeMux()
mux.HandleFunc("/localapi/v0/dns-config", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(dnsCfg); err != nil {
t.Errorf("encoding dns config: %v", err)
}
})
srv := &http.Server{Handler: mux}
go srv.Serve(lal)
t.Cleanup(func() { srv.Shutdown(context.Background()) })
client := &local.Client{Dial: lal.Dial}
// Empty starting state, as if the InitialStatus captured at pod
// boot carried no ExtraRecords because the VIPService didn't exist
// yet at that time.
var s netmapState
n := ipn.Notify{
SelfChange: &tailcfg.Node{
ID: 1,
Name: "self.tailnet.ts.net.",
},
}
got := s.processNotify(context.Background(), client, n)
if got.dnsExtraRecords.Len() != 1 {
t.Fatalf("dnsExtraRecords.Len() = %d, want 1", got.dnsExtraRecords.Len())
}
if rec := got.dnsExtraRecords.At(0); rec.Name != extraRec.Name {
t.Errorf("dnsExtraRecords[0].Name = %q, want %q", rec.Name, extraRec.Name)
}
if got.certDomains.Len() != 1 || got.certDomains.At(0) != "node.tailnet.ts.net" {
t.Errorf("certDomains = %v, want [node.tailnet.ts.net]", got.certDomains.AsSlice())
}
}
+8
View File
@@ -24,6 +24,7 @@ import (
"tailscale.com/kube/kubetypes"
klc "tailscale.com/kube/localclient"
"tailscale.com/kube/services"
"tailscale.com/types/netmap"
)
// watchServeConfigChanges watches path for changes, and when it sees one, reads
@@ -141,6 +142,13 @@ func refreshAdvertiseServices(ctx context.Context, sc *ipn.ServeConfig, lc klc.L
return nil
}
func certDomainFromNetmap(nm *netmap.NetworkMap) string {
if len(nm.DNS.CertDomains) == 0 {
return ""
}
return nm.DNS.CertDomains[0]
}
func updateServeConfig(ctx context.Context, sc *ipn.ServeConfig, certDomain string, lc klc.LocalClient) error {
if !isValidHTTPSConfig(certDomain, sc) {
return nil
+64 -40
View File
@@ -6,7 +6,6 @@
package main
import (
"cmp"
"context"
"errors"
"fmt"
@@ -19,7 +18,6 @@ import (
"tailscale.com/ipn/conffile"
"tailscale.com/kube/kubeclient"
"tailscale.com/util/def"
)
// settings is all the configuration for containerboot.
@@ -91,50 +89,47 @@ type settings struct {
func configFromEnv() (*settings, error) {
cfg := &settings{
AuthKey: cmp.Or(os.Getenv("TS_AUTHKEY"), os.Getenv("TS_AUTH_KEY")),
ClientID: os.Getenv("TS_CLIENT_ID"),
ClientSecret: os.Getenv("TS_CLIENT_SECRET"),
IDToken: os.Getenv("TS_ID_TOKEN"),
Audience: os.Getenv("TS_AUDIENCE"),
Hostname: os.Getenv("TS_HOSTNAME"),
AuthKey: defaultEnvs([]string{"TS_AUTHKEY", "TS_AUTH_KEY"}, ""),
ClientID: defaultEnv("TS_CLIENT_ID", ""),
ClientSecret: defaultEnv("TS_CLIENT_SECRET", ""),
IDToken: defaultEnv("TS_ID_TOKEN", ""),
Audience: defaultEnv("TS_AUDIENCE", ""),
Hostname: defaultEnv("TS_HOSTNAME", ""),
Routes: defaultEnvStringPointer("TS_ROUTES"),
ServeConfigPath: os.Getenv("TS_SERVE_CONFIG"),
ProxyTargetIP: os.Getenv("TS_DEST_IP"),
ProxyTargetDNSName: os.Getenv("TS_EXPERIMENTAL_DEST_DNS_NAME"),
TailnetTargetIP: os.Getenv("TS_TAILNET_TARGET_IP"),
TailnetTargetFQDN: os.Getenv("TS_TAILNET_TARGET_FQDN"),
DaemonExtraArgs: os.Getenv("TS_TAILSCALED_EXTRA_ARGS"),
ExtraArgs: os.Getenv("TS_EXTRA_ARGS"),
ServeConfigPath: defaultEnv("TS_SERVE_CONFIG", ""),
ProxyTargetIP: defaultEnv("TS_DEST_IP", ""),
ProxyTargetDNSName: defaultEnv("TS_EXPERIMENTAL_DEST_DNS_NAME", ""),
TailnetTargetIP: defaultEnv("TS_TAILNET_TARGET_IP", ""),
TailnetTargetFQDN: defaultEnv("TS_TAILNET_TARGET_FQDN", ""),
DaemonExtraArgs: defaultEnv("TS_TAILSCALED_EXTRA_ARGS", ""),
ExtraArgs: defaultEnv("TS_EXTRA_ARGS", ""),
InKubernetes: os.Getenv("KUBERNETES_SERVICE_HOST") != "",
UserspaceMode: def.Bool(os.Getenv("TS_USERSPACE"), true),
StateDir: os.Getenv("TS_STATE_DIR"),
UserspaceMode: defaultBool("TS_USERSPACE", true),
StateDir: defaultEnv("TS_STATE_DIR", ""),
AcceptDNS: defaultEnvBoolPointer("TS_ACCEPT_DNS"),
KubeSecret: func() string {
if os.Getenv("KUBERNETES_SERVICE_HOST") == "" {
return os.Getenv("TS_KUBE_SECRET")
if os.Getenv("KUBERNETES_SERVICE_HOST") != "" {
return defaultEnv("TS_KUBE_SECRET", "tailscale")
}
// An explicitly empty TS_KUBE_SECRET disables Secret storage, so
// unset and empty must stay distinguishable: def.LookupEnv keeps
// an explicit "" rather than falling back to the default.
return def.LookupEnv("TS_KUBE_SECRET", "tailscale")
return defaultEnv("TS_KUBE_SECRET", "")
}(),
SOCKSProxyAddr: os.Getenv("TS_SOCKS5_SERVER"),
HTTPProxyAddr: os.Getenv("TS_OUTBOUND_HTTP_PROXY_LISTEN"),
Socket: cmp.Or(os.Getenv("TS_SOCKET"), "/tmp/tailscaled.sock"),
AuthOnce: def.Bool(os.Getenv("TS_AUTH_ONCE"), false),
Root: cmp.Or(os.Getenv("TS_TEST_ONLY_ROOT"), "/"),
SOCKSProxyAddr: defaultEnv("TS_SOCKS5_SERVER", ""),
HTTPProxyAddr: defaultEnv("TS_OUTBOUND_HTTP_PROXY_LISTEN", ""),
Socket: defaultEnv("TS_SOCKET", "/tmp/tailscaled.sock"),
AuthOnce: defaultBool("TS_AUTH_ONCE", false),
Root: defaultEnv("TS_TEST_ONLY_ROOT", "/"),
TailscaledConfigFilePath: tailscaledConfigFilePath(),
AllowProxyingClusterTrafficViaIngress: def.Bool(os.Getenv("EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS"), false),
PodIP: os.Getenv("POD_IP"),
EnableForwardingOptimizations: def.Bool(os.Getenv("TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS"), false),
HealthCheckAddrPort: os.Getenv("TS_HEALTHCHECK_ADDR_PORT"),
LocalAddrPort: cmp.Or(os.Getenv("TS_LOCAL_ADDR_PORT"), "[::]:9002"),
MetricsEnabled: def.Bool(os.Getenv("TS_ENABLE_METRICS"), false),
HealthCheckEnabled: def.Bool(os.Getenv("TS_ENABLE_HEALTH_CHECK"), false),
DebugAddrPort: os.Getenv("TS_DEBUG_ADDR_PORT"),
EgressProxiesCfgPath: os.Getenv("TS_EGRESS_PROXIES_CONFIG_PATH"),
IngressProxiesCfgPath: os.Getenv("TS_INGRESS_PROXIES_CONFIG_PATH"),
PodUID: os.Getenv("POD_UID"),
AllowProxyingClusterTrafficViaIngress: defaultBool("EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS", false),
PodIP: defaultEnv("POD_IP", ""),
EnableForwardingOptimizations: defaultBool("TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS", false),
HealthCheckAddrPort: defaultEnv("TS_HEALTHCHECK_ADDR_PORT", ""),
LocalAddrPort: defaultEnv("TS_LOCAL_ADDR_PORT", "[::]:9002"),
MetricsEnabled: defaultBool("TS_ENABLE_METRICS", false),
HealthCheckEnabled: defaultBool("TS_ENABLE_HEALTH_CHECK", false),
DebugAddrPort: defaultEnv("TS_DEBUG_ADDR_PORT", ""),
EgressProxiesCfgPath: defaultEnv("TS_EGRESS_PROXIES_CONFIG_PATH", ""),
IngressProxiesCfgPath: defaultEnv("TS_INGRESS_PROXIES_CONFIG_PATH", ""),
PodUID: defaultEnv("POD_UID", ""),
}
podIPs, ok := os.LookupEnv("POD_IPS")
@@ -158,7 +153,7 @@ func configFromEnv() (*settings, error) {
// If cert share is enabled, set the replica as read or write. Only 0th
// replica should be able to write.
isInCertShareMode := def.Bool(os.Getenv("TS_EXPERIMENTAL_CERT_SHARE"), false)
isInCertShareMode := defaultBool("TS_EXPERIMENTAL_CERT_SHARE", false)
if isInCertShareMode {
cfg.CertShareMode = "ro"
podName := os.Getenv("POD_NAME")
@@ -459,6 +454,15 @@ func (cfg *settings) egressSvcsTerminateEPEnabled() bool {
return cfg.LocalAddrPort != "" && cfg.EgressProxiesCfgPath != ""
}
// defaultEnv returns the value of the given envvar name, or defVal if
// unset.
func defaultEnv(name, defVal string) string {
if v, ok := os.LookupEnv(name); ok {
return v
}
return defVal
}
// defaultEnvStringPointer returns a pointer to the given envvar value if set, else
// returns nil. This is useful in cases where we need to distinguish between a
// variable being set to empty string vs unset.
@@ -480,3 +484,23 @@ func defaultEnvBoolPointer(name string) *bool {
}
return &ret
}
func defaultEnvs(names []string, defVal string) string {
for _, name := range names {
if v, ok := os.LookupEnv(name); ok {
return v
}
}
return defVal
}
// defaultBool returns the boolean value of the given envvar name, or
// defVal if unset or not a bool.
func defaultBool(name string, defVal bool) bool {
v := os.Getenv(name)
ret, err := strconv.ParseBool(v)
if err != nil {
return defVal
}
return ret
}
-73
View File
@@ -7,7 +7,6 @@ package main
import (
"net/netip"
"os"
"strings"
"testing"
)
@@ -229,78 +228,6 @@ func TestValidateAuthMethods(t *testing.T) {
}
}
func TestConfigFromEnvEmptyDefaults(t *testing.T) {
tests := []struct {
env string
get func(*settings) string
want string
}{
{
env: "TS_SOCKET",
get: func(c *settings) string { return c.Socket },
want: "/tmp/tailscaled.sock",
},
{
env: "TS_LOCAL_ADDR_PORT",
get: func(c *settings) string { return c.LocalAddrPort },
want: "[::]:9002",
},
{
env: "TS_TEST_ONLY_ROOT",
get: func(c *settings) string { return c.Root },
want: "/",
},
}
for _, tt := range tests {
t.Run(tt.env, func(t *testing.T) {
t.Setenv(tt.env, "")
cfg, err := configFromEnv()
if err != nil {
t.Fatal(err)
}
if got := tt.get(cfg); got != tt.want {
t.Errorf(`%s set to empty "": got %q, want default %q`, tt.env, got, tt.want)
}
})
}
}
func TestConfigFromEnvKubeSecret(t *testing.T) {
tests := []struct {
name string
inKubernetes bool
unset bool
value string
want string
}{
{name: "in_kubernetes_unset", inKubernetes: true, unset: true, want: "tailscale"},
{name: "in_kubernetes_empty", inKubernetes: true, value: "", want: ""},
{name: "in_kubernetes_set", inKubernetes: true, value: "custom", want: "custom"},
{name: "not_in_kubernetes_unset", inKubernetes: false, unset: true, want: ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// t.Setenv registers a t.Cleanup to restore the original value, so
// route the unset cases through it rather than a bare os.Unsetenv.
t.Setenv("KUBERNETES_SERVICE_HOST", "10.96.0.1")
if !tt.inKubernetes {
os.Unsetenv("KUBERNETES_SERVICE_HOST")
}
t.Setenv("TS_KUBE_SECRET", tt.value)
if tt.unset {
os.Unsetenv("TS_KUBE_SECRET")
}
cfg, err := configFromEnv()
if err != nil {
t.Fatal(err)
}
if cfg.KubeSecret != tt.want {
t.Errorf("KubeSecret = %q, want %q", cfg.KubeSecret, tt.want)
}
})
}
}
func TestHandlesKubeIPV6(t *testing.T) {
t.Setenv("TS_LOCAL_ADDR_PORT", "fd7a:115c:a1e0::6c34:352:9002")
t.Setenv("POD_IPS", "fd7a:115c:a1e0::6c34:352")
+2 -14
View File
@@ -150,15 +150,7 @@ func tailscaleUp(ctx context.Context, cfg *settings) error {
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
// A canceled context kills the command, and cmd.Run can
// report the subprocess's death ("signal: killed") rather
// than the context error that caused it. Return the
// context error so that callers (and ultimately main) can
// recognize a graceful shutdown with errors.Is.
return fmt.Errorf("tailscale up failed: %w", ctxErr)
}
return fmt.Errorf("tailscale up failed: %w", err)
return fmt.Errorf("tailscale up failed: %v", err)
}
return nil
}
@@ -188,11 +180,7 @@ func tailscaleSet(ctx context.Context, cfg *settings) error {
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
if ctxErr := ctx.Err(); ctxErr != nil {
// See the equivalent check in tailscaleUp.
return fmt.Errorf("tailscale set failed: %w", ctxErr)
}
return fmt.Errorf("tailscale set failed: %w", err)
return fmt.Errorf("tailscale set failed: %v", err)
}
return nil
}
+11 -27
View File
@@ -23,7 +23,6 @@ import (
"os"
"path/filepath"
"regexp"
"slices"
"time"
"golang.org/x/crypto/acme"
@@ -36,34 +35,21 @@ var unsafeHostnameCharacters = regexp.MustCompile(`[^a-zA-Z0-9-\.]`)
type certProvider interface {
// TLSConfig creates a new TLS config suitable for net/http.Server servers.
//
// The returned Config must have a GetCertificate function set. The
// *tls.Certificate values it returns may be shared and cached, so
// callers must not mutate them.
// The returned Config must have a GetCertificate function set and that
// function must return a unique *tls.Certificate for each call. The
// returned *tls.Certificate will be mutated by the caller to append to the
// (*tls.Certificate).Certificate field.
TLSConfig() *tls.Config
// HTTPHandler handle ACME related request, if any.
HTTPHandler(fallback http.Handler) http.Handler
}
func certProviderByCertMode(mode, dir, hostname string, ipCerts bool, eabKID, eabKey, email string) (certProvider, error) {
func certProviderByCertMode(mode, dir, hostname, eabKID, eabKey, email string) (certProvider, error) {
if dir == "" {
return nil, errors.New("missing required --certdir flag")
}
if ipCerts && mode != "letsencrypt" {
return nil, errors.New("--acme-ip-certs requires --certmode=letsencrypt")
}
switch mode {
case "letsencrypt", "gcp":
if net.ParseIP(hostname) != nil {
if mode == "gcp" {
return nil, errors.New("--certmode=gcp requires --hostname to be a DNS name, not an IP address")
}
if !ipCerts {
return nil, errors.New("--hostname is an IP address; use --certmode=manual for a self-signed cert, or set --acme-ip-certs to get LetsEncrypt IP address certs")
}
// IP-only server: certs are issued on demand per
// connection, so there is no hostname cert provider.
return newIPCertManager(dir, email, "", nil)
}
certManager := &autocert.Manager{
Prompt: autocert.AcceptTOS,
HostPolicy: autocert.HostWhitelist(hostname),
@@ -96,9 +82,6 @@ func certProviderByCertMode(mode, dir, hostname string, ipCerts bool, eabKID, ea
} else if hostname == "derp.tailscale.com" {
certManager.Email = "security@tailscale.com"
}
if ipCerts {
return newIPCertManager(dir, email, "", certManager)
}
return certManager, nil
case "manual":
return NewManualCertManager(dir, hostname)
@@ -174,11 +157,12 @@ func (m *manualCertManager) getCertificate(hi *tls.ClientHelloInfo) (*tls.Certif
return nil, fmt.Errorf("cert mismatch with hostname: %q", hi.ServerName)
}
// Return a shallow copy of the cert with a capacity-clamped chain
// so callers can never mutate the manager's long-lived certificate.
certCopy := *m.cert
certCopy.Certificate = slices.Clip(certCopy.Certificate)
return &certCopy, nil
// Return a shallow copy of the cert so the caller can append to its
// Certificate field.
certCopy := new(tls.Certificate)
*certCopy = *m.cert
certCopy.Certificate = certCopy.Certificate[:len(certCopy.Certificate):len(certCopy.Certificate)]
return certCopy, nil
}
func (m *manualCertManager) HTTPHandler(fallback http.Handler) http.Handler {
+6 -6
View File
@@ -91,7 +91,7 @@ func TestCertIP(t *testing.T) {
t.Fatalf("Error closing key.pem: %v", err)
}
cp, err := certProviderByCertMode("manual", dir, hostname, false, "", "", "")
cp, err := certProviderByCertMode("manual", dir, hostname, "", "", "")
if err != nil {
t.Fatal(err)
}
@@ -174,25 +174,25 @@ func TestGCPCertMode(t *testing.T) {
dir := t.TempDir()
// Missing EAB credentials
_, err := certProviderByCertMode("gcp", dir, "test.example.com", false, "", "", "test@example.com")
_, err := certProviderByCertMode("gcp", dir, "test.example.com", "", "", "test@example.com")
if err == nil {
t.Fatal("expected error when EAB credentials are missing")
}
// Missing email
_, err = certProviderByCertMode("gcp", dir, "test.example.com", false, "kid", "dGVzdC1rZXk", "")
_, err = certProviderByCertMode("gcp", dir, "test.example.com", "kid", "dGVzdC1rZXk", "")
if err == nil {
t.Fatal("expected error when email is missing")
}
// Invalid base64
_, err = certProviderByCertMode("gcp", dir, "test.example.com", false, "kid", "not-valid!", "test@example.com")
_, err = certProviderByCertMode("gcp", dir, "test.example.com", "kid", "not-valid!", "test@example.com")
if err == nil {
t.Fatal("expected error for invalid base64")
}
// Valid base64url (no padding)
cp, err := certProviderByCertMode("gcp", dir, "test.example.com", false, "kid", "dGVzdC1rZXk", "test@example.com")
cp, err := certProviderByCertMode("gcp", dir, "test.example.com", "kid", "dGVzdC1rZXk", "test@example.com")
if err != nil {
t.Fatalf("base64url: %v", err)
}
@@ -201,7 +201,7 @@ func TestGCPCertMode(t *testing.T) {
}
// Valid standard base64 (with padding, gcloud format)
cp, err = certProviderByCertMode("gcp", dir, "test.example.com", false, "kid", "dGVzdC1rZXk=", "test@example.com")
cp, err = certProviderByCertMode("gcp", dir, "test.example.com", "kid", "dGVzdC1rZXk=", "test@example.com")
if err != nil {
t.Fatalf("base64: %v", err)
}
+16 -21
View File
@@ -6,9 +6,10 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
github.com/axiomhq/hyperloglog from tailscale.com/derp/derpserver
github.com/beorn7/perks/quantile from github.com/prometheus/client_golang/prometheus
💣 github.com/cespare/xxhash/v2 from github.com/prometheus/client_golang/prometheus
github.com/coder/websocket from tailscale.com/derp/derpserver+
github.com/coder/websocket from tailscale.com/cmd/derper+
github.com/coder/websocket/internal/errd from github.com/coder/websocket
github.com/coder/websocket/internal/util from github.com/coder/websocket
github.com/coder/websocket/internal/xsync from github.com/coder/websocket
github.com/creachadair/msync/throttle from github.com/tailscale/setec/client/setec
W 💣 github.com/dblohm7/wingoes from tailscale.com/util/winutil
github.com/dgryski/go-metro from github.com/axiomhq/hyperloglog
@@ -19,7 +20,6 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
github.com/go-json-experiment/json/internal/jsonopts from github.com/go-json-experiment/json+
github.com/go-json-experiment/json/internal/jsonwire from github.com/go-json-experiment/json+
github.com/go-json-experiment/json/jsontext from github.com/go-json-experiment/json+
github.com/go-json-experiment/json/v1 from tailscale.com/net/routecheck+
💣 github.com/go4org/hashtriemap from tailscale.com/derp/derpserver
github.com/golang/groupcache/lru from tailscale.com/net/dnscache
github.com/hdevalence/ed25519consensus from tailscale.com/tka
@@ -91,7 +91,6 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/envknob from tailscale.com/client/local+
tailscale.com/feature from tailscale.com/tsweb+
tailscale.com/feature/buildfeatures from tailscale.com/feature+
tailscale.com/feature/serviceclientprefs/serviceclient from tailscale.com/client/local
tailscale.com/health from tailscale.com/net/tlsdial+
tailscale.com/hostinfo from tailscale.com/net/netmon+
tailscale.com/ipn from tailscale.com/client/local
@@ -108,23 +107,19 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
💣 tailscale.com/net/netns from tailscale.com/derp/derphttp
tailscale.com/net/netutil from tailscale.com/client/local
tailscale.com/net/netx from tailscale.com/net/dnscache+
tailscale.com/net/routecheck from tailscale.com/client/local
tailscale.com/net/routecheck/peernode from tailscale.com/net/routecheck
tailscale.com/net/sockstats from tailscale.com/derp/derphttp
tailscale.com/net/stun from tailscale.com/net/stunserver
tailscale.com/net/stunserver from tailscale.com/cmd/derper
L tailscale.com/net/tcpinfo from tailscale.com/derp/derpserver
tailscale.com/net/tlsdial from tailscale.com/derp/derphttp
tailscale.com/net/tlsdial/blockblame from tailscale.com/net/tlsdial
tailscale.com/net/traffic from tailscale.com/net/routecheck
tailscale.com/net/tsaddr from tailscale.com/ipn+
tailscale.com/net/udprelay/status from tailscale.com/client/local
tailscale.com/net/wsconn from tailscale.com/derp/derpserver
tailscale.com/net/wsconn from tailscale.com/cmd/derper
tailscale.com/paths from tailscale.com/client/local
💣 tailscale.com/safesocket from tailscale.com/client/local
tailscale.com/syncs from tailscale.com/cmd/derper+
tailscale.com/tailcfg from tailscale.com/client/local+
tailscale.com/tempfork/acme from tailscale.com/cmd/derper
tailscale.com/tka from tailscale.com/client/local+
tailscale.com/tsconst from tailscale.com/net/netmon+
tailscale.com/tstime from tailscale.com/derp+
@@ -140,7 +135,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/types/key from tailscale.com/client/local+
tailscale.com/types/lazy from tailscale.com/version+
tailscale.com/types/logger from tailscale.com/cmd/derper+
tailscale.com/types/netmap from tailscale.com/ipn+
tailscale.com/types/netmap from tailscale.com/ipn
tailscale.com/types/opt from tailscale.com/envknob+
tailscale.com/types/persist from tailscale.com/ipn+
tailscale.com/types/preftype from tailscale.com/ipn
@@ -169,7 +164,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/util/syspolicy/pkey from tailscale.com/ipn+
tailscale.com/util/syspolicy/policyclient from tailscale.com/ipn
tailscale.com/util/syspolicy/ptype from tailscale.com/util/syspolicy/policyclient+
tailscale.com/util/syspolicy/setting from tailscale.com/client/local+
tailscale.com/util/syspolicy/setting from tailscale.com/client/local
tailscale.com/util/testenv from tailscale.com/net/bakedroots+
tailscale.com/util/usermetric from tailscale.com/health
tailscale.com/util/vizerror from tailscale.com/tailcfg+
@@ -249,22 +244,22 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
crypto/internal/boring/bbig from crypto/ecdsa+
crypto/internal/boring/sig from crypto/internal/boring
crypto/internal/constanttime from crypto/internal/fips140/edwards25519+
crypto/internal/fips140 from crypto/fips140+
crypto/internal/fips140 from crypto/internal/fips140/aes+
crypto/internal/fips140/aes from crypto/aes+
crypto/internal/fips140/aes/gcm from crypto/cipher+
crypto/internal/fips140/alias from crypto/cipher+
crypto/internal/fips140/bigmod from crypto/internal/fips140/ecdsa+
crypto/internal/fips140/check from crypto/fips140+
crypto/internal/fips140/drbg from crypto/hpke+
crypto/internal/fips140/check from crypto/internal/fips140/aes+
crypto/internal/fips140/drbg from crypto/internal/fips140/aes/gcm+
crypto/internal/fips140/ecdh from crypto/ecdh
crypto/internal/fips140/ecdsa from crypto/ecdsa
crypto/internal/fips140/ed25519 from crypto/ed25519
crypto/internal/fips140/edwards25519 from crypto/internal/fips140/ed25519
crypto/internal/fips140/edwards25519/field from crypto/ecdh+
crypto/internal/fips140/hkdf from crypto/hkdf+
crypto/internal/fips140/hkdf from crypto/internal/fips140/tls13+
crypto/internal/fips140/hmac from crypto/hmac+
crypto/internal/fips140/mlkem from crypto/mlkem
crypto/internal/fips140/nistec from crypto/ecdsa+
crypto/internal/fips140/nistec from crypto/elliptic+
crypto/internal/fips140/nistec/fiat from crypto/internal/fips140/nistec
crypto/internal/fips140/rsa from crypto/rsa
crypto/internal/fips140/sha256 from crypto/internal/fips140/check+
@@ -315,7 +310,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
go/token from google.golang.org/protobuf/internal/strs
hash from crypto+
hash/crc32 from compress/gzip+
hash/fnv from google.golang.org/protobuf/internal/detrand+
hash/fnv from google.golang.org/protobuf/internal/detrand
hash/maphash from go4.org/mem+
html from net/http/pprof+
html/template from tailscale.com/cmd/derper+
@@ -330,13 +325,13 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
internal/filepathlite from os+
internal/fmtsort from fmt+
internal/goarch from crypto/internal/fips140deps/cpu+
internal/godebug from crypto/ed25519+
internal/godebug from crypto/internal/fips140deps/godebug+
internal/godebugs from internal/godebug+
internal/goexperiment from net/http/pprof+
internal/goos from crypto/x509+
internal/msan from internal/runtime/maps+
internal/nettrace from net+
internal/oserror from internal/syscall/windows+
internal/oserror from io/fs+
internal/poll from net+
internal/profile from net/http/pprof
internal/profilerecord from runtime+
@@ -346,9 +341,9 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
internal/runtime/atomic from internal/runtime/exithook+
L internal/runtime/cgroup from runtime
internal/runtime/exithook from runtime
internal/runtime/gc from internal/runtime/gc/scan+
internal/runtime/gc from runtime+
internal/runtime/gc/scan from runtime
internal/runtime/maps from hash/maphash+
internal/runtime/maps from reflect+
internal/runtime/math from internal/runtime/maps+
internal/runtime/pprof/label from runtime+
internal/runtime/sys from crypto/subtle+
@@ -362,7 +357,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
internal/synctest from sync
internal/syscall/execenv from os+
LD internal/syscall/unix from crypto/internal/sysrand+
W internal/syscall/windows from crypto/internal/fips140deps/time+
W internal/syscall/windows from crypto/internal/sysrand+
W internal/syscall/windows/registry from mime+
W internal/syscall/windows/sysdll from internal/syscall/windows+
internal/testlog from os
+12 -5
View File
@@ -62,11 +62,10 @@ var (
configPath = flag.String("c", "", "config file path")
certMode = flag.String("certmode", "letsencrypt", "mode for getting a cert. possible options: manual, letsencrypt, gcp")
certDir = flag.String("certdir", tsweb.DefaultCertDir("derper-certs"), "directory to store ACME (e.g. LetsEncrypt) certs, if addr's port is :443")
hostname = flag.String("hostname", "derp.tailscale.com", "TLS host name for certs, if addr's port is :443. It can be an IP address when --certmode=manual (to avoid SNI checks) or when --acme-ip-certs is set (to run an IP-only server with no hostname cert)")
hostname = flag.String("hostname", "derp.tailscale.com", "TLS host name for certs, if addr's port is :443. When --certmode=manual, this can be an IP address to avoid SNI checks")
acmeEABKid = flag.String("acme-eab-kid", "", "ACME External Account Binding (EAB) Key ID (required for --certmode=gcp)")
acmeEABKey = flag.String("acme-eab-key", "", "ACME External Account Binding (EAB) HMAC key, base64-encoded (required for --certmode=gcp)")
acmeEmail = flag.String("acme-email", "", "ACME account contact email address (required for --certmode=gcp, optional for letsencrypt)")
acmeIPCerts = flag.Bool("acme-ip-certs", false, "whether to serve LetsEncrypt certs for the server's IP addresses: when a client connects by IP address (sending no TLS SNI, or an IP address SNI matching the connection's destination IP), get and serve a LetsEncrypt cert for that IP, using the short-lived (~6 day) ACME certificate profile. This works for both IPv4 and IPv6 with no per-address configuration. It requires --certmode=letsencrypt and the ACME server must be able to reach port 80 at each such IP for the HTTP-01 challenge.")
runSTUN = flag.Bool("stun", true, "whether to run a STUN server. It will bind to the same IP (if any) as the --addr flag value.")
runDERP = flag.Bool("derp", true, "whether to run a DERP server. The only reason to set this false is if you're decommissioning a server but want to keep its bootstrap DNS functionality still running.")
flagHome = flag.String("home", "", "what to serve at the root path. It may be left empty (the default, for a default homepage), \"blank\" for a blank page, or a URL to redirect to")
@@ -263,7 +262,7 @@ func main() {
mux := http.NewServeMux()
if *runDERP {
derpHandler := derpserver.Handler(s)
derpHandler = derpserver.AddWebSocketSupport(s, derpHandler)
derpHandler = addWebSocketSupport(s, derpHandler)
mux.Handle("/derp", derpHandler)
} else {
mux.Handle("/derp", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -350,12 +349,20 @@ func main() {
if serveTLS {
log.Printf("derper: serving on %s with TLS", *addr)
var certManager certProvider
certManager, err = certProviderByCertMode(*certMode, *certDir, *hostname, *acmeIPCerts, *acmeEABKid, *acmeEABKey, *acmeEmail)
certManager, err = certProviderByCertMode(*certMode, *certDir, *hostname, *acmeEABKid, *acmeEABKey, *acmeEmail)
if err != nil {
log.Fatalf("derper: can not start cert provider: %v", err)
}
httpsrv.TLSConfig = certManager.TLSConfig()
s.ModifyTLSConfigToAddMetaCert(httpsrv.TLSConfig)
getCert := httpsrv.TLSConfig.GetCertificate
httpsrv.TLSConfig.GetCertificate = func(hi *tls.ClientHelloInfo) (*tls.Certificate, error) {
cert, err := getCert(hi)
if err != nil {
return nil, err
}
cert.Certificate = append(cert.Certificate, s.MetaCert())
return cert, nil
}
// Disable TLS 1.0 and 1.1, which are obsolete and have security issues.
httpsrv.TLSConfig.MinVersion = tls.VersionTLS12
httpsrv.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-497
View File
@@ -1,497 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"io"
"log"
"net"
"net/http"
"net/netip"
"os"
"path/filepath"
"slices"
"strings"
"sync"
"time"
"tailscale.com/atomicfile"
"tailscale.com/tailcfg"
"tailscale.com/tempfork/acme"
)
// shortlivedProfile is the ACME certificate profile required by
// LetsEncrypt for IP address certificates. Certificates issued under
// it are valid for about six days.
// See https://letsencrypt.org/docs/profiles/.
const shortlivedProfile = "shortlived"
// ipCertManager is a certProvider that obtains and renews LetsEncrypt
// TLS certificates for the server's IP addresses on demand, using the
// short-lived ACME certificate profile and the HTTP-01 challenge
// served on the derper's plaintext HTTP port.
//
// Clients connecting to an IP address usually send no SNI, so the
// requested IP address is taken from the TCP connection's local
// address. That works for however many IPv4 and IPv6 addresses the
// server has, with no configuration. Clients that do send an IP
// address in the SNI get a certificate only if it matches the
// connection's local address, so a client can never make us request a
// certificate for an address that isn't ours.
//
// Connections with a DNS name in the SNI are passed through to the
// optional next provider (the regular autocert manager for the
// --hostname certificate), if any.
type ipCertManager struct {
certDir string
email string // optional ACME account contact
client *acme.Client
next certProvider // provider for DNS hostname connections, or nil
nextTLS *tls.Config // next.TLSConfig(), or nil
mu sync.Mutex
certs map[netip.Addr]*ipCertEntry
tokens map[string]string // HTTP-01 challenge URL path => response body
}
// ipCertEntry is the issuance state for one IP address.
// All fields are guarded by ipCertManager.mu.
type ipCertEntry struct {
cert *tls.Certificate // current cert with Leaf set, or nil if not yet issued
flight chan struct{} // non-nil while an issuance is running; closed when it finishes
flightErr error // result of the last finished issuance
nextAttempt time.Time // earliest time of the next issuance attempt, after a failure
retryDelay time.Duration // backoff to apply after the next failure
}
// newIPCertManager returns an ipCertManager storing its ACME account
// key and issued certificates in certdir.
//
// If directoryURL is empty, the LetsEncrypt production directory is
// used; tests point it at a fake ACME server. If next is non-nil,
// connections with a DNS name in the SNI are served by it.
func newIPCertManager(certdir, email, directoryURL string, next certProvider) (*ipCertManager, error) {
if err := os.MkdirAll(certdir, 0700); err != nil {
return nil, err
}
accountKey, err := loadOrCreateAccountKey(filepath.Join(certdir, "acme-account.key"))
if err != nil {
return nil, fmt.Errorf("ACME account key: %w", err)
}
m := &ipCertManager{
certDir: certdir,
email: email,
client: &acme.Client{
Key: accountKey,
DirectoryURL: directoryURL,
UserAgent: "tailscale-derper",
},
next: next,
certs: make(map[netip.Addr]*ipCertEntry),
tokens: make(map[string]string),
}
if next != nil {
m.nextTLS = next.TLSConfig()
}
go m.renewLoop()
return m, nil
}
func loadOrCreateAccountKey(path string) (*ecdsa.PrivateKey, error) {
if pemBytes, err := os.ReadFile(path); err == nil {
block, _ := pem.Decode(pemBytes)
if block == nil {
return nil, fmt.Errorf("invalid PEM in %s", path)
}
return x509.ParseECPrivateKey(block.Bytes)
} else if !os.IsNotExist(err) {
return nil, err
}
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, err
}
der, err := x509.MarshalECPrivateKey(key)
if err != nil {
return nil, err
}
pemBytes := pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: der})
if err := atomicfile.WriteFile(path, pemBytes, 0600); err != nil {
return nil, err
}
return key, nil
}
// certPaths returns the cert and key file paths for ip in m.certDir.
// Colons in IPv6 addresses are replaced with dots to keep the names
// filesystem-safe.
func (m *ipCertManager) certPaths(ip netip.Addr) (crtPath, keyPath string) {
base := strings.ReplaceAll(ip.String(), ":", ".")
return filepath.Join(m.certDir, base+".crt"), filepath.Join(m.certDir, base+".key")
}
func (m *ipCertManager) TLSConfig() *tls.Config {
var conf *tls.Config
if m.nextTLS != nil {
conf = m.nextTLS.Clone()
} else {
conf = &tls.Config{
NextProtos: []string{
"http/1.1",
},
}
}
conf.GetCertificate = m.getCertificate
return conf
}
// connLocalIP returns the local (server side) IP address of the
// connection that sent the ClientHello.
func connLocalIP(hi *tls.ClientHelloInfo) (netip.Addr, bool) {
if hi.Conn == nil {
return netip.Addr{}, false
}
ta, ok := hi.Conn.LocalAddr().(*net.TCPAddr)
if !ok {
return netip.Addr{}, false
}
ip := ta.AddrPort().Addr().Unmap()
return ip, ip.IsValid()
}
func (m *ipCertManager) getCertificate(hi *tls.ClientHelloInfo) (*tls.Certificate, error) {
connIP, connIPOK := connLocalIP(hi)
if hi.ServerName != "" {
sniIP, err := netip.ParseAddr(hi.ServerName)
if err != nil {
// The SNI is a DNS name; let the hostname provider handle it.
if m.nextTLS != nil && m.nextTLS.GetCertificate != nil {
return m.nextTLS.GetCertificate(hi)
}
return nil, fmt.Errorf("no certificate for hostname %q; this server only serves IP address certificates", hi.ServerName)
}
if !connIPOK || sniIP.Unmap() != connIP {
return nil, fmt.Errorf("requested certificate for IP %v does not match the connection's IP address", sniIP)
}
}
if !connIPOK {
return nil, errors.New("unable to determine the connection's local IP address")
}
ctx := hi.Context()
if ctx == nil {
ctx = context.Background()
}
return m.certForIP(ctx, connIP)
}
// certForIP returns the current certificate for ip, obtaining one
// first if there is no unexpired certificate for it. Concurrent
// callers for the same IP share a single issuance.
func (m *ipCertManager) certForIP(ctx context.Context, ip netip.Addr) (*tls.Certificate, error) {
m.mu.Lock()
e := m.entryLocked(ip)
if e.cert != nil && time.Now().Before(e.cert.Leaf.NotAfter) {
defer m.mu.Unlock()
return clipCert(e.cert), nil
}
if e.flight == nil && time.Now().Before(e.nextAttempt) {
m.mu.Unlock()
return nil, fmt.Errorf("cert issuance for %v failed recently; next attempt no earlier than %v", ip, e.nextAttempt.Format(time.RFC3339))
}
flight := m.startFlightLocked(ip, e)
m.mu.Unlock()
select {
case <-flight:
case <-ctx.Done():
return nil, ctx.Err()
}
m.mu.Lock()
defer m.mu.Unlock()
if e.cert == nil {
return nil, e.flightErr
}
return clipCert(e.cert), nil
}
func (m *ipCertManager) entryLocked(ip netip.Addr) *ipCertEntry {
e, ok := m.certs[ip]
if !ok {
e = &ipCertEntry{}
m.certs[ip] = e
}
return e
}
// clipCert returns a shallow copy of cert with a capacity-clamped
// chain so callers can never mutate the manager's long-lived
// certificate.
func clipCert(cert *tls.Certificate) *tls.Certificate {
certCopy := *cert
certCopy.Certificate = slices.Clip(certCopy.Certificate)
return &certCopy
}
// startFlightLocked starts an issuance for ip if none is running and
// returns a channel that is closed when it finishes.
func (m *ipCertManager) startFlightLocked(ip netip.Addr, e *ipCertEntry) chan struct{} {
if e.flight != nil {
return e.flight
}
done := make(chan struct{})
e.flight = done
go func() {
err := m.issue(ip)
m.mu.Lock()
defer m.mu.Unlock()
e.flight = nil
e.flightErr = err
if err != nil {
if e.retryDelay == 0 {
e.retryDelay = time.Minute
}
e.nextAttempt = time.Now().Add(e.retryDelay)
e.retryDelay = min(e.retryDelay*2, 30*time.Minute)
log.Printf("derper: acme: getting cert for %v: %v (next attempt in %v)", ip, err, time.Until(e.nextAttempt).Round(time.Second))
} else {
e.retryDelay = 0
e.nextAttempt = time.Time{}
}
close(done)
}()
return done
}
// issue obtains a certificate for ip, preferring a still-fresh one
// cached on disk over a new ACME order.
func (m *ipCertManager) issue(ip netip.Addr) error {
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
if cert, err := m.loadCachedCert(ip); err == nil && !certNeedsRenewal(cert.Leaf) {
m.mu.Lock()
m.entryLocked(ip).cert = cert
m.mu.Unlock()
log.Printf("derper: acme: loaded cached cert for %v (expires %v)", ip, cert.Leaf.NotAfter)
return nil
}
return m.obtainCert(ctx, ip)
}
// loadCachedCert loads a previously issued certificate for ip from
// disk, if present and still valid.
func (m *ipCertManager) loadCachedCert(ip netip.Addr) (*tls.Certificate, error) {
crtPath, keyPath := m.certPaths(ip)
cert, err := tls.LoadX509KeyPair(crtPath, keyPath)
if err != nil {
return nil, err
}
leaf, err := x509.ParseCertificate(cert.Certificate[0])
if err != nil {
return nil, err
}
now := time.Now()
if now.Before(leaf.NotBefore) || now.After(leaf.NotAfter) {
return nil, fmt.Errorf("cached cert is expired or not yet valid (NotAfter %v)", leaf.NotAfter)
}
if err := leaf.VerifyHostname(ip.String()); err != nil {
return nil, err
}
cert.Leaf = leaf
return &cert, nil
}
// HTTPHandler returns a handler serving HTTP-01 challenge responses on
// the derper's plaintext HTTP port, sending all other requests to the
// next provider's handler, if any, and otherwise to fallback.
func (m *ipCertManager) HTTPHandler(fallback http.Handler) http.Handler {
if m.next != nil {
fallback = m.next.HTTPHandler(fallback)
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.HasPrefix(r.URL.Path, "/.well-known/acme-challenge/") {
fallback.ServeHTTP(w, r)
return
}
m.mu.Lock()
response, ok := m.tokens[r.URL.Path]
m.mu.Unlock()
if !ok {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "text/plain")
io.WriteString(w, response)
})
}
func (m *ipCertManager) setToken(path, response string) {
m.mu.Lock()
defer m.mu.Unlock()
m.tokens[path] = response
}
func (m *ipCertManager) deleteToken(path string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.tokens, path)
}
// certNeedsRenewal reports whether leaf has less than a third of its
// lifetime remaining. LetsEncrypt short-lived certs are valid for
// about six days, so renewal happens roughly every four.
func certNeedsRenewal(leaf *x509.Certificate) bool {
total := leaf.NotAfter.Sub(leaf.NotBefore)
return time.Until(leaf.NotAfter) < total/3
}
// renewLoop runs for the lifetime of the process, renewing each issued
// certificate as it approaches expiry.
func (m *ipCertManager) renewLoop() {
for {
time.Sleep(time.Hour)
m.mu.Lock()
now := time.Now()
for ip, e := range m.certs {
if e.cert != nil && certNeedsRenewal(e.cert.Leaf) && e.flight == nil && now.After(e.nextAttempt) {
m.startFlightLocked(ip, e)
}
}
m.mu.Unlock()
}
}
// obtainCert does one ACME issuance flow for ip: it registers the
// account if needed, orders a short-lived profile certificate for the
// IP address identifier, fulfills the HTTP-01 challenges, and installs
// and caches the issued certificate.
func (m *ipCertManager) obtainCert(ctx context.Context, ip netip.Addr) error {
ipStr := ip.String()
var contact []string
if m.email != "" {
contact = []string{"mailto:" + m.email}
}
_, err := m.client.Register(ctx, &acme.Account{Contact: contact}, acme.AcceptTOS)
if err != nil && !errors.Is(err, acme.ErrAccountAlreadyExists) {
return fmt.Errorf("register: %w", err)
}
order, err := m.client.AuthorizeOrder(ctx, acme.IPIDs(ipStr), acme.WithOrderProfile(shortlivedProfile))
if err != nil {
return fmt.Errorf("new order: %w", err)
}
for _, authzURL := range order.AuthzURLs {
if err := m.fulfillAuthz(ctx, authzURL); err != nil {
return err
}
}
order, err = m.client.WaitOrder(ctx, order.URI)
if err != nil {
return fmt.Errorf("waiting for order: %w", err)
}
certKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return err
}
csr, err := x509.CreateCertificateRequest(rand.Reader, &x509.CertificateRequest{
IPAddresses: []net.IP{ip.AsSlice()},
}, certKey)
if err != nil {
return err
}
der, _, err := m.client.CreateOrderCert(ctx, order.FinalizeURL, csr, true)
if err != nil {
return fmt.Errorf("finalizing order: %w", err)
}
leaf, err := x509.ParseCertificate(der[0])
if err != nil {
return fmt.Errorf("parsing issued cert: %w", err)
}
if err := leaf.VerifyHostname(ipStr); err != nil {
return fmt.Errorf("issued cert: %w", err)
}
keyDER, err := x509.MarshalECPrivateKey(certKey)
if err != nil {
return err
}
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: keyDER})
var chainPEM []byte
for _, b := range der {
chainPEM = append(chainPEM, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: b})...)
}
crtPath, keyPath := m.certPaths(ip)
if err := atomicfile.WriteFile(keyPath, keyPEM, 0600); err != nil {
return err
}
if err := atomicfile.WriteFile(crtPath, chainPEM, 0644); err != nil {
return err
}
m.mu.Lock()
m.entryLocked(ip).cert = &tls.Certificate{
Certificate: der,
PrivateKey: certKey,
Leaf: leaf,
}
m.mu.Unlock()
dn := &tailcfg.DERPNode{
Name: "custom",
RegionID: 900,
HostName: ipStr,
}
dnJSON, _ := json.Marshal(dn)
log.Printf("derper: acme: got cert for %v (expires %v). Configure it in DERPMap using (https://tailscale.com/s/custom-derp):\n %s", ip, leaf.NotAfter, dnJSON)
return nil
}
// fulfillAuthz completes the HTTP-01 challenge for one authorization,
// if it is still pending.
func (m *ipCertManager) fulfillAuthz(ctx context.Context, authzURL string) error {
authz, err := m.client.GetAuthorization(ctx, authzURL)
if err != nil {
return fmt.Errorf("getting authorization: %w", err)
}
if authz.Status != acme.StatusPending {
return nil
}
var challenge *acme.Challenge
for _, c := range authz.Challenges {
if c.Type == "http-01" {
challenge = c
break
}
}
if challenge == nil {
return errors.New("authorization offers no http-01 challenge")
}
response, err := m.client.HTTP01ChallengeResponse(challenge.Token)
if err != nil {
return err
}
path := m.client.HTTP01ChallengePath(challenge.Token)
m.setToken(path, response)
defer m.deleteToken(path)
if _, err := m.client.Accept(ctx, challenge); err != nil {
return fmt.Errorf("accepting challenge: %w", err)
}
if _, err := m.client.WaitAuthorization(ctx, authz.URI); err != nil {
return fmt.Errorf("waiting for authorization: %w", err)
}
return nil
}
-486
View File
@@ -1,486 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/base64"
"encoding/json"
"encoding/pem"
"fmt"
"io"
"math/big"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
)
// fakeIPACME is a minimal fake ACME (RFC 8555) certificate authority
// for testing ipCertManager. It implements just enough of the protocol
// for http-01 order flows with IP address identifiers and the
// "shortlived" profile: one order at a time, no JWS signature
// verification, no nonce tracking.
type fakeIPACME struct {
t *testing.T
srv *httptest.Server
challengeBase string // base URL at which http-01 challenges are fetched
caKey *ecdsa.PrivateKey
caCert *x509.Certificate
mu sync.Mutex
orders int // number of orders created
gotProfile string // profile of the last order
gotIDType string // identifier type of the last order
gotIDValue string // identifier value of the last order
authzStatus string // "pending" or "valid"
orderStatus string // "pending", "ready", or "valid"
token string
certPEM []byte
}
func newFakeIPACME(t *testing.T) *fakeIPACME {
caKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
caTmpl := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "fake IP ACME root"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(24 * time.Hour),
IsCA: true,
KeyUsage: x509.KeyUsageCertSign,
BasicConstraintsValid: true,
}
caDER, err := x509.CreateCertificate(rand.Reader, caTmpl, caTmpl, &caKey.PublicKey, caKey)
if err != nil {
t.Fatal(err)
}
caCert, err := x509.ParseCertificate(caDER)
if err != nil {
t.Fatal(err)
}
f := &fakeIPACME{
t: t,
caKey: caKey,
caCert: caCert,
}
f.srv = httptest.NewServer(http.HandlerFunc(f.serveHTTP))
t.Cleanup(f.srv.Close)
return f
}
func (f *fakeIPACME) directoryURL() string { return f.srv.URL + "/directory" }
func (f *fakeIPACME) numOrders() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.orders
}
func (f *fakeIPACME) serveHTTP(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Replay-Nonce", "test-nonce")
writeJSON := func(v any) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(v)
}
orderJSON := func() any {
return map[string]any{
"status": f.orderStatus,
"identifiers": []map[string]string{{"type": f.gotIDType, "value": f.gotIDValue}},
"authorizations": []string{f.srv.URL + "/authz/1"},
"finalize": f.srv.URL + "/finalize/1",
"certificate": f.srv.URL + "/cert/1",
}
}
switch {
case r.URL.Path == "/directory":
writeJSON(map[string]any{
"newNonce": f.srv.URL + "/new-nonce",
"newAccount": f.srv.URL + "/new-account",
"newOrder": f.srv.URL + "/new-order",
"revokeCert": f.srv.URL + "/revoke-cert",
"keyChange": f.srv.URL + "/key-change",
"meta": map[string]any{
"profiles": map[string]string{
"classic": "the default profile",
shortlivedProfile: "six day certificates",
},
},
})
case r.URL.Path == "/new-nonce":
// The Replay-Nonce header was already set above.
case r.URL.Path == "/new-account":
w.Header().Set("Location", f.srv.URL+"/account/1")
w.WriteHeader(http.StatusCreated)
writeJSON(map[string]any{"status": "valid"})
case r.URL.Path == "/new-order":
var req struct {
Identifiers []struct{ Type, Value string } `json:"identifiers"`
Profile string `json:"profile"`
}
if err := decodeJWSPayload(r, &req); err != nil {
f.t.Errorf("new-order payload: %v", err)
}
f.mu.Lock()
f.orders++
f.gotProfile = req.Profile
if len(req.Identifiers) == 1 {
f.gotIDType = req.Identifiers[0].Type
f.gotIDValue = req.Identifiers[0].Value
}
f.authzStatus = "pending"
f.orderStatus = "pending"
f.token = fmt.Sprintf("tok-%d", f.orders)
f.mu.Unlock()
w.Header().Set("Location", f.srv.URL+"/order/1")
w.WriteHeader(http.StatusCreated)
writeJSON(orderJSON())
case r.URL.Path == "/authz/1":
f.mu.Lock()
defer f.mu.Unlock()
writeJSON(map[string]any{
"status": f.authzStatus,
"identifier": map[string]string{"type": f.gotIDType, "value": f.gotIDValue},
"challenges": []map[string]string{{
"type": "http-01",
"url": f.srv.URL + "/chal/1",
"token": f.token,
"status": f.authzStatus,
}},
})
case r.URL.Path == "/chal/1":
if err := f.validateChallenge(); err != nil {
f.t.Errorf("challenge validation: %v", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
f.mu.Lock()
f.authzStatus = "valid"
f.orderStatus = "ready"
f.mu.Unlock()
writeJSON(map[string]any{"type": "http-01", "status": "valid", "token": f.token})
case r.URL.Path == "/order/1":
f.mu.Lock()
defer f.mu.Unlock()
w.Header().Set("Location", f.srv.URL+"/order/1")
writeJSON(orderJSON())
case r.URL.Path == "/finalize/1":
var req struct {
CSR string `json:"csr"`
}
if err := decodeJWSPayload(r, &req); err != nil {
f.t.Errorf("finalize payload: %v", err)
}
if err := f.issueCert(req.CSR); err != nil {
f.t.Errorf("issuing cert: %v", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
f.mu.Lock()
f.orderStatus = "valid"
f.mu.Unlock()
w.Header().Set("Location", f.srv.URL+"/order/1")
writeJSON(orderJSON())
case r.URL.Path == "/cert/1":
f.mu.Lock()
defer f.mu.Unlock()
w.Header().Set("Content-Type", "application/pem-certificate-chain")
w.Write(f.certPEM)
default:
f.t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
http.NotFound(w, r)
}
}
// validateChallenge fetches the http-01 challenge response from the
// server under test, standing in for the CA dialing port 80 at the IP
// address being validated.
func (f *fakeIPACME) validateChallenge() error {
f.mu.Lock()
token := f.token
base := f.challengeBase
f.mu.Unlock()
res, err := http.Get(base + "/.well-known/acme-challenge/" + token)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return fmt.Errorf("status %d", res.StatusCode)
}
body, err := io.ReadAll(res.Body)
if err != nil {
return err
}
if !strings.HasPrefix(string(body), token+".") {
return fmt.Errorf("challenge response %q does not start with %q", body, token+".")
}
return nil
}
// issueCert signs a certificate for the CSR (base64url DER), valid for
// six days like a LetsEncrypt shortlived profile certificate.
func (f *fakeIPACME) issueCert(csrB64 string) error {
csrDER, err := base64.RawURLEncoding.DecodeString(csrB64)
if err != nil {
return err
}
csr, err := x509.ParseCertificateRequest(csrDER)
if err != nil {
return err
}
if len(csr.IPAddresses) != 1 {
return fmt.Errorf("CSR has %d IP addresses; want 1", len(csr.IPAddresses))
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(2),
IPAddresses: csr.IPAddresses,
NotBefore: time.Now().Add(-time.Minute),
NotAfter: time.Now().Add(6 * 24 * time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, f.caCert, csr.PublicKey, f.caKey)
if err != nil {
return err
}
var buf []byte
buf = append(buf, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})...)
buf = append(buf, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: f.caCert.Raw})...)
f.mu.Lock()
f.certPEM = buf
f.mu.Unlock()
return nil
}
// decodeJWSPayload decodes the payload of a JWS-encoded ACME request
// without verifying its signature.
func decodeJWSPayload(r *http.Request, v any) error {
var req struct{ Payload string }
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
return err
}
b, err := base64.RawURLEncoding.DecodeString(req.Payload)
if err != nil {
return err
}
return json.Unmarshal(b, v)
}
// ipConn is a stub net.Conn whose LocalAddr is the given IP, standing
// in for the accepted TLS connection whose destination address decides
// which certificate to serve.
type ipConn struct {
net.Conn
local net.Addr
}
func (c ipConn) LocalAddr() net.Addr { return c.local }
// helloFor returns a ClientHelloInfo as ipCertManager.getCertificate
// would see it for a connection to localIP with the given SNI value.
func helloFor(t *testing.T, localIP, sni string) *tls.ClientHelloInfo {
t.Helper()
ip := net.ParseIP(localIP)
if ip == nil {
t.Fatalf("bad IP %q", localIP)
}
return &tls.ClientHelloInfo{
ServerName: sni,
Conn: ipConn{local: &net.TCPAddr{IP: ip, Port: 443}},
}
}
// stubCertProvider is a certProvider returning a fixed certificate,
// standing in for the autocert manager handling DNS hostname
// connections.
type stubCertProvider struct {
cert *tls.Certificate
}
func (p *stubCertProvider) TLSConfig() *tls.Config {
return &tls.Config{
GetCertificate: func(hi *tls.ClientHelloInfo) (*tls.Certificate, error) {
return p.cert, nil
},
}
}
func (p *stubCertProvider) HTTPHandler(fallback http.Handler) http.Handler { return fallback }
// TestIPCertManager exercises the on-demand issuance flow of
// ipCertManager against a fake ACME CA: certs are ordered for whatever
// IP address a connection arrives on (IPv4 and IPv6), with the
// shortlived profile, answering the http-01 challenge, and reusing the
// on-disk cache.
func TestIPCertManager(t *testing.T) {
const ip4 = "203.0.113.7"
const ip6 = "2001:db8::7"
dir := t.TempDir()
ca := newFakeIPACME(t)
m, err := newIPCertManager(dir, "test@example.com", ca.directoryURL(), nil)
if err != nil {
t.Fatal(err)
}
// Serve the manager's HTTP-01 challenge handler like derper's port
// 80 listener does.
challengeSrv := httptest.NewServer(m.HTTPHandler(http.NotFoundHandler()))
defer challengeSrv.Close()
ca.challengeBase = challengeSrv.URL
// A connection to the IPv4 address with no SNI mints a cert for it.
cert, err := m.getCertificate(helloFor(t, ip4, ""))
if err != nil {
t.Fatal(err)
}
if err := cert.Leaf.VerifyHostname(ip4); err != nil {
t.Errorf("issued cert not valid for %v: %v", ip4, err)
}
if got, want := ca.gotProfile, shortlivedProfile; got != want {
t.Errorf("order profile = %q; want %q", got, want)
}
if ca.gotIDType != "ip" || ca.gotIDValue != ip4 {
t.Errorf("order identifier = %q %q; want %q %q", ca.gotIDType, ca.gotIDValue, "ip", ip4)
}
if n := ca.numOrders(); n != 1 {
t.Errorf("orders created = %d; want 1", n)
}
// A connection to the IPv6 address mints a second, separate cert.
cert6, err := m.getCertificate(helloFor(t, ip6, ""))
if err != nil {
t.Fatal(err)
}
if err := cert6.Leaf.VerifyHostname(ip6); err != nil {
t.Errorf("issued cert not valid for %v: %v", ip6, err)
}
if ca.gotIDType != "ip" || ca.gotIDValue != ip6 {
t.Errorf("order identifier = %q %q; want %q %q", ca.gotIDType, ca.gotIDValue, "ip", ip6)
}
if n := ca.numOrders(); n != 2 {
t.Errorf("orders created = %d; want 2", n)
}
// An SNI containing the connection's own IP address is served from
// the cache.
if _, err := m.getCertificate(helloFor(t, ip4, ip4)); err != nil {
t.Errorf("getCertificate with matching IP SNI: %v", err)
}
if n := ca.numOrders(); n != 2 {
t.Errorf("orders created after cached hit = %d; want 2", n)
}
// An SNI naming some other IP address is rejected.
if _, err := m.getCertificate(helloFor(t, ip4, "203.0.113.8")); err == nil {
t.Error("getCertificate with mismatched IP SNI succeeded; want error")
}
// A DNS name SNI has no provider to go to here.
if _, err := m.getCertificate(helloFor(t, ip4, "derp.example.com")); err == nil {
t.Error("getCertificate with DNS SNI and no next provider succeeded; want error")
}
// A second manager over the same cert directory must use the
// on-disk cache rather than creating more orders.
m2, err := newIPCertManager(dir, "test@example.com", ca.directoryURL(), nil)
if err != nil {
t.Fatal(err)
}
cachedCert, err := m2.getCertificate(helloFor(t, ip4, ""))
if err != nil {
t.Fatal(err)
}
if err := cachedCert.Leaf.VerifyHostname(ip4); err != nil {
t.Errorf("cached cert not valid for %v: %v", ip4, err)
}
if n := ca.numOrders(); n != 2 {
t.Errorf("orders created after cache reuse = %d; want 2", n)
}
// Non-challenge requests go to the fallback handler.
rec := httptest.NewRecorder()
m.HTTPHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "fallback")
})).ServeHTTP(rec, httptest.NewRequest("GET", "/other", nil))
if got := rec.Body.String(); got != "fallback" {
t.Errorf("fallback body = %q; want %q", got, "fallback")
}
}
// TestIPCertManagerNextProvider verifies that connections with a DNS
// name in the SNI are passed through to the next provider.
func TestIPCertManagerNextProvider(t *testing.T) {
dir := t.TempDir()
ca := newFakeIPACME(t)
stubCert := &tls.Certificate{}
m, err := newIPCertManager(dir, "", ca.directoryURL(), &stubCertProvider{cert: stubCert})
if err != nil {
t.Fatal(err)
}
got, err := m.getCertificate(helloFor(t, "203.0.113.7", "derp.example.com"))
if err != nil {
t.Fatal(err)
}
if got != stubCert {
t.Errorf("DNS SNI returned %p; want the next provider's cert %p", got, stubCert)
}
if n := ca.numOrders(); n != 0 {
t.Errorf("orders created = %d; want 0", n)
}
}
// TestCertModeIPCertsGating verifies the flag validation around
// --acme-ip-certs and IP address hostnames.
func TestCertModeIPCertsGating(t *testing.T) {
tests := []struct {
name string
mode string
host string
ipCerts bool
wantErr string // or empty to expect success
}{
{"letsencrypt_ip_no_flag", "letsencrypt", "1.2.3.4", false, "--acme-ip-certs"},
{"gcp_ip", "gcp", "1.2.3.4", false, "--certmode=gcp requires --hostname to be a DNS name"},
{"gcp_flag", "gcp", "1.2.3.4", true, "--acme-ip-certs requires --certmode=letsencrypt"},
{"manual_flag", "manual", "1.2.3.4", true, "--acme-ip-certs requires --certmode=letsencrypt"},
{"letsencrypt_ip_flag", "letsencrypt", "1.2.3.4", true, ""},
{"letsencrypt_hostname_flag", "letsencrypt", "derp.example.com", true, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cp, err := certProviderByCertMode(tt.mode, t.TempDir(), tt.host, tt.ipCerts, "", "", "")
if tt.wantErr == "" {
if err != nil {
t.Fatalf("certProviderByCertMode(%q, %q, ipCerts=%v) = %v; want success", tt.mode, tt.host, tt.ipCerts, err)
}
m, ok := cp.(*ipCertManager)
if !ok {
t.Fatalf("provider type = %T; want *ipCertManager", cp)
}
wantNext := net.ParseIP(tt.host) == nil
if gotNext := m.next != nil; gotNext != wantNext {
t.Errorf("has next provider = %v; want %v", gotNext, wantNext)
}
return
}
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
t.Errorf("certProviderByCertMode(%q, %q, ipCerts=%v) error = %v; want contains %q",
tt.mode, tt.host, tt.ipCerts, err, tt.wantErr)
}
})
}
}
@@ -1,7 +1,7 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package derpserver
package main
import (
"bufio"
@@ -11,20 +11,14 @@ import (
"strings"
"github.com/coder/websocket"
"tailscale.com/derp/derpserver"
"tailscale.com/net/wsconn"
)
var counterWebSocketAccepts = expvar.NewInt("derp_websocket_accepts")
// AddWebSocketSupport returns an http.Handler wrapping base that adds
// WebSocket-DERP support. WebSocket-DERP requests (those with an Upgrade:
// websocket header and a "derp" Sec-WebSocket-Protocol value) are
// handled here; all other requests pass through to base.
//
// The browser-side Tailscale client (cmd/tsconnect/wasm) can only reach DERP
// via WebSocket, so any DERP server intended to be reachable from browsers
// must wrap derpserver.Handler with this function.
func AddWebSocketSupport(s *Server, base http.Handler) http.Handler {
// addWebSocketSupport returns a Handle wrapping base that adds WebSocket server support.
func addWebSocketSupport(s *derpserver.Server, base http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
up := strings.ToLower(r.Header.Get("Upgrade"))
-821
View File
@@ -1,821 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:build linux
// fbstatus is a Linux framebuffer status display for the Tailscale
// appliance. It draws the Tailscale logo, the tailscaled backend state,
// the device's tailnet IP addresses, and (when the device needs to be
// logged in) a QR code containing the login URL so a user can enroll
// the appliance into a tailnet by pointing their phone camera at the
// screen.
//
// fbstatus accesses the framebuffer via the Linux UAPI in
// include/uapi/linux/fb.h: FBIOGET_VSCREENINFO and FBIOGET_FSCREENINFO
// ioctls plus an mmap of /dev/fb0. Only 32-bit truecolor framebuffers
// (the Raspberry Pi default) are supported.
package main
import (
"bytes"
"context"
_ "embed"
"encoding/binary"
"flag"
"fmt"
"image"
"image/color"
"image/draw"
"image/png"
"log"
"net"
"net/http"
"net/netip"
"net/url"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"strings"
"sync/atomic"
"syscall"
"time"
"unsafe"
"github.com/skip2/go-qrcode"
xdraw "golang.org/x/image/draw"
"golang.org/x/image/font"
"golang.org/x/image/font/basicfont"
"golang.org/x/image/math/fixed"
"golang.org/x/sys/unix"
"tailscale.com/client/local"
"tailscale.com/ipn"
"tailscale.com/util/cloudenv"
)
//go:embed tailscale.png
var tailscalePNG []byte
// Linux framebuffer ioctl numbers, from include/uapi/linux/fb.h.
const (
fbioGetVScreenInfo = 0x4600
fbioGetFScreenInfo = 0x4602
)
// Linux VT ioctl numbers and KD_* modes, from include/uapi/linux/kd.h
// and include/uapi/linux/vt.h.
const (
kdSetMode = 0x4B3A
kdGraphics = 1
kdText = 0
vtActivate = 0x5606
vtWaitActive = 0x5607
)
// Byte offsets into the raw fb_var_screeninfo struct returned by
// FBIOGET_VSCREENINFO. All fields we read are little-endian uint32.
const (
vsOffXres = 0
vsOffYres = 4
vsOffBitsPerPixel = 24
vsOffRedOffset = 32 // start of struct fb_bitfield red
vsOffGreenOffset = 44 // start of struct fb_bitfield green
vsOffBlueOffset = 56 // start of struct fb_bitfield blue
)
// Byte offsets into the raw fb_fix_screeninfo struct returned by
// FBIOGET_FSCREENINFO. Layout assumes a 64-bit kernel (the gokrazy
// appliance targets — arm64/amd64 — are both 64-bit). smem_start and
// mmio_start are "unsigned long", which is 8 bytes on 64-bit.
const (
fsOffSmemLen = 24
fsOffLineLength = 48
)
var flagFB = flag.String("fb", "/dev/fb0", "framebuffer device to draw to")
// noFramebufferReason reports whether this host lacks a usable Linux
// framebuffer, along with a short human-readable explanation. If the
// framebuffer device is missing, or we're running on a cloud (currently
// only AWS) whose instances don't expose one, we return true.
func noFramebufferReason(fbPath string) (string, bool) {
if cloudenv.Get() == cloudenv.AWS {
return "running on AWS (no framebuffer)", true
}
if _, err := os.Stat(fbPath); err != nil {
return fmt.Sprintf("no framebuffer at %s: %v", fbPath, err), true
}
return "", false
}
func main() {
flag.Parse()
log.SetFlags(log.LstdFlags | log.Lmicroseconds)
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
// Bail out early on cloud VMs that don't ship a framebuffer. We
// still kick off breakglass once DHCP succeeds (otherwise the
// appliance is unreachable — breakglass declares DontStartOnBoot
// and only runs when fbstatus pokes the supervisor), then exit
// 125 so the gokrazy supervisor stops respawning us. See
// https://gokrazy.org/development/process-interface/.
if reason, ok := noFramebufferReason(*flagFB); ok {
log.Printf("%s; starting breakglass after DHCP then exiting 125", reason)
startBreakglassAfterDHCP(&uiState{})
log.Printf("breakglass started; exiting 125 so gokrazy won't respawn fbstatus")
os.Exit(125)
}
if restore, err := claimVTGraphics(); err != nil {
log.Printf("could not put VT into graphics mode (fbcon may overdraw): %v", err)
} else {
defer restore()
}
fb, err := openFramebuffer(*flagFB)
if err != nil {
return fmt.Errorf("open framebuffer: %w", err)
}
defer fb.Close()
log.Printf("framebuffer %s: %dx%d, %d bpp, line=%d, RGB offsets %d/%d/%d",
*flagFB, fb.width, fb.height, fb.bpp, fb.lineLength,
fb.redShift, fb.greenShift, fb.blueShift)
logo, err := png.Decode(bytes.NewReader(tailscalePNG))
if err != nil {
return fmt.Errorf("decoding embedded logo: %w", err)
}
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel()
var lc local.Client
st := &uiState{fb: fb, logo: logo}
st.updateLAN()
st.render()
go st.pollLAN(ctx)
go startBreakglassAfterDHCP(st)
go watchKeyboardForConsole(ctx, st)
for ctx.Err() == nil {
if err := watchBusOnce(ctx, &lc, st); err != nil && ctx.Err() == nil {
log.Printf("ipn watch: %v; retrying in 2s", err)
select {
case <-ctx.Done():
case <-time.After(2 * time.Second):
}
}
}
return nil
}
func watchBusOnce(ctx context.Context, lc *local.Client, st *uiState) error {
w, err := lc.WatchIPNBus(ctx,
ipn.NotifyInitialState|ipn.NotifyInitialPrefs|ipn.NotifyInitialStatus)
if err != nil {
return err
}
defer w.Close()
loginRequested := false
for ctx.Err() == nil {
n, err := w.Next()
if err != nil {
return err
}
if n.State != nil {
st.state = *n.State
// On a fresh appliance, tailscaled enters NeedsLogin but
// does not generate a login URL until someone asks. Trigger
// an interactive login so the control server sends us a URL
// (and thus a QR code appears on the display).
if *n.State == ipn.NeedsLogin && !loginRequested {
loginRequested = true
go func() {
if err := lc.StartLoginInteractive(ctx); err != nil {
log.Printf("StartLoginInteractive: %v", err)
}
}()
}
}
if n.BrowseToURL != nil {
st.loginURL = *n.BrowseToURL
}
if n.InitialStatus != nil {
st.ips = append(st.ips[:0], n.InitialStatus.TailscaleIPs...)
}
if n.SelfChange != nil {
st.ips = st.ips[:0]
for _, p := range n.SelfChange.Addresses {
st.ips = append(st.ips, p.Addr())
}
}
st.render()
}
return ctx.Err()
}
// updateLAN scans network interfaces for a non-loopback interface with a
// hardware address, updating st.lanIP and st.lanMAC. Shows the MAC even
// if DHCP hasn't assigned an IP yet.
func (st *uiState) updateLAN() {
ifaces, err := net.Interfaces()
if err != nil {
return
}
var bestMAC string
var bestIP string
for _, iface := range ifaces {
if iface.Flags&net.FlagLoopback != 0 {
continue
}
if len(iface.HardwareAddr) == 0 {
continue
}
if bestMAC == "" {
bestMAC = iface.HardwareAddr.String()
}
if iface.Flags&net.FlagUp == 0 {
continue
}
// Prefer the first UP interface with a MAC.
if bestMAC != iface.HardwareAddr.String() && bestIP == "" {
bestMAC = iface.HardwareAddr.String()
}
addrs, err := iface.Addrs()
if err != nil {
continue
}
for _, addr := range addrs {
if ipnet, ok := addr.(*net.IPNet); ok && ipnet.IP.To4() != nil {
bestMAC = iface.HardwareAddr.String()
bestIP = ipnet.IP.String()
}
}
}
st.lanMAC = bestMAC
st.lanIP = bestIP
}
// startBreakglassAfterDHCP waits until a LAN IP is assigned (meaning DHCP
// succeeded), then restarts breakglass. This ensures breakglass sees the
// real LAN address in PrivateInterfaceAddrs and binds to it, rather than
// only binding to 127.0.0.1.
func startBreakglassAfterDHCP(st *uiState) {
for {
st.updateLAN()
if st.lanIP != "" {
break
}
time.Sleep(time.Second)
}
startBreakglass()
}
// startBreakglass asks the gokrazy init HTTP API (over its unix socket) to
// restart the breakglass service so it actually runs. By default breakglass
// calls DontStartOnBoot and exits on the first launch attempt; this poke
// tells the supervisor to try again (without GOKRAZY_FIRST_START=1).
func startBreakglass() {
const sock = "/run/gokrazy-http.sock"
hc := &http.Client{
Transport: &http.Transport{
DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, "unix", sock)
},
},
}
form := url.Values{
"path": {"/user/breakglass"},
"xsrftoken": {"1"},
}
req, err := http.NewRequest("POST", "http://gokrazy/restart", strings.NewReader(form.Encode()))
if err != nil {
log.Printf("startBreakglass: %v", err)
return
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(&http.Cookie{Name: "gokrazy_xsrf", Value: "1"})
resp, err := hc.Do(req)
if err != nil {
log.Printf("startBreakglass: %v", err)
return
}
resp.Body.Close()
if resp.StatusCode < 300 || resp.StatusCode == http.StatusSeeOther {
log.Printf("startBreakglass: restarted (status %s)", resp.Status)
} else {
log.Printf("startBreakglass: unexpected status %s", resp.Status)
}
}
// watchKeyboardForConsole monitors keyboard input devices for VT-switching
// accelerators.
// - Ctrl-Alt-F2 (or plain Esc — easier to type in NoVNC where the
// Ctrl-Alt-Fn sequence doesn't always propagate) switches to VT2, a
// text-mode busybox shell. When that shell exits, we switch back
// automatically.
// - Ctrl-Alt-F1 switches back to VT1 (fbstatus graphics mode).
//
// This mirrors standard Linux VT switching conventions plus a NoVNC-
// friendly shortcut.
func watchKeyboardForConsole(ctx context.Context, st *uiState) {
kbdPath := findKeyboard()
if kbdPath == "" {
log.Printf("no keyboard found for VT switching")
return
}
kbd, err := os.Open(kbdPath)
if err != nil {
log.Printf("open keyboard %s: %v", kbdPath, err)
return
}
defer kbd.Close()
ttyFile, err := os.OpenFile("/dev/tty0", os.O_RDWR, 0)
if err != nil {
log.Printf("open /dev/tty0 for VT switch: %v", err)
return
}
defer ttyFile.Close()
ttyFd := int(ttyFile.Fd())
log.Printf("watching %s for Ctrl-Alt-F1/F2 and Esc (VT switching)", kbdPath)
// Linux input_event has the same layout on both arm64 and amd64
// (24 bytes: two uint64 timestamps + uint16 type + uint16 code +
// int32 value), so this parser handles both the Pi and Proxmox VM.
const evSize = 24
const evKey = 1 // EV_KEY
const keyEsc = 1 // KEY_ESC
const keyF1 = 59 // KEY_F1
const keyF2 = 60 // KEY_F2
const keyLeftCtrl = 29
const keyLeftAlt = 56
const keyRightCtrl = 97
const keyRightAlt = 100
const keyPress = 1
buf := make([]byte, evSize)
var ctrlHeld, altHeld bool
switchToFbstatus := func() {
st.paused.Store(false)
syscall.Syscall(syscall.SYS_IOCTL, uintptr(ttyFd), vtActivate, 1)
syscall.Syscall(syscall.SYS_IOCTL, uintptr(ttyFd), vtWaitActive, 1)
ioctlSetInt(ttyFile, kdSetMode, kdGraphics)
st.render()
}
switchToShell := func(reason string) {
st.paused.Store(true)
ioctlSetInt(ttyFile, kdSetMode, kdText)
syscall.Syscall(syscall.SYS_IOCTL, uintptr(ttyFd), vtActivate, 2)
syscall.Syscall(syscall.SYS_IOCTL, uintptr(ttyFd), vtWaitActive, 2)
go ensureShellOnVT2(switchToFbstatus)
log.Printf("%s: switched to text console", reason)
}
for ctx.Err() == nil {
n, err := kbd.Read(buf)
if err != nil || n < evSize {
continue
}
evType := binary.LittleEndian.Uint16(buf[16:18])
evCode := binary.LittleEndian.Uint16(buf[18:20])
evValue := int32(binary.LittleEndian.Uint32(buf[20:24]))
if evType != evKey {
continue
}
pressed := evValue == keyPress
released := evValue == 0
switch evCode {
case keyLeftCtrl, keyRightCtrl:
if pressed {
ctrlHeld = true
} else if released {
ctrlHeld = false
}
case keyLeftAlt, keyRightAlt:
if pressed {
altHeld = true
} else if released {
altHeld = false
}
case keyEsc:
if pressed && !ctrlHeld && !altHeld {
// Bare Esc: NoVNC-friendly shortcut to the shell.
switchToShell("Esc")
}
case keyF1:
if pressed && ctrlHeld && altHeld {
switchToFbstatus()
log.Printf("Ctrl-Alt-F1: switched to fbstatus")
}
case keyF2:
if pressed && ctrlHeld && altHeld {
switchToShell("Ctrl-Alt-F2")
}
}
}
}
// ensureShellOnVT2 spawns a busybox ash shell on /dev/tty2 if one isn't
// already running. The shell gets the VT2 tty as its controlling terminal
// so keyboard input on VT2 goes to it. When the shell exits, onExit is
// called (typically to switch back to VT1 / fbstatus graphics mode).
var shellOnVT2Running atomic.Bool
func ensureShellOnVT2(onExit func()) {
if !shellOnVT2Running.CompareAndSwap(false, true) {
return
}
go func() {
defer shellOnVT2Running.Store(false)
defer func() {
if onExit != nil {
onExit()
}
}()
shell := "/tmp/serial-busybox/ash"
if _, err := os.Stat(shell); err != nil {
log.Printf("no shell at %s for VT2", shell)
return
}
tty, err := os.OpenFile("/dev/tty2", os.O_RDWR, 0)
if err != nil {
log.Printf("open /dev/tty2: %v", err)
return
}
defer tty.Close()
cmd := exec.Command(shell)
cmd.Stdin = tty
cmd.Stdout = tty
cmd.Stderr = tty
cmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
Setctty: true,
Ctty: 0, // index into cmd's file descriptors (stdin = tty)
}
cmd.Env = append(os.Environ(), "TERM=linux", "HOME=/tmp", "PATH=/tmp/serial-busybox:/user:/gokrazy")
log.Printf("starting shell on VT2")
if err := cmd.Run(); err != nil {
log.Printf("shell on VT2 exited: %v", err)
}
}()
}
// findKeyboard looks for a keyboard among /dev/input/event* devices by
// checking that the device's key capability bitmap has KEY_ESC set. The
// alternative "any non-zero key bitmap" check picks up the ACPI power
// button (which advertises KEY_POWER but no Esc) and misses the real
// keyboard on amd64 Proxmox VMs, where the AT keyboard is event1 but
// event0 is Power Button.
func findKeyboard() string {
matches, _ := filepath.Glob("/dev/input/event*")
for _, path := range matches {
name := filepath.Base(path)
capData, err := os.ReadFile("/sys/class/input/" + name + "/device/capabilities/key")
if err != nil {
continue
}
fields := strings.Fields(strings.TrimSpace(string(capData)))
if len(fields) == 0 {
continue
}
// The kernel prints capability bitmaps as space-separated 64-bit
// hex chunks, most-significant chunk first. The last chunk holds
// bits 0..63. KEY_ESC = 1, so its bit-mask is 1<<1 == 0x2.
low, err := strconv.ParseUint(fields[len(fields)-1], 16, 64)
if err != nil {
continue
}
const keyEscBit = 1 << 1
if low&keyEscBit != 0 {
return path
}
}
return ""
}
// pollLAN periodically refreshes LAN info and re-renders.
func (st *uiState) pollLAN(ctx context.Context) {
for {
select {
case <-ctx.Done():
return
case <-time.After(5 * time.Second):
st.updateLAN()
st.render()
}
}
}
// uiState is the most-recently-known view of the appliance state that
// gets rendered to the framebuffer on each notify.
type uiState struct {
fb *framebuffer
logo image.Image
state ipn.State
loginURL string
ips []netip.Addr
lanIP string // LAN IPv4 address (from DHCP)
lanMAC string // MAC address of the primary interface
paused atomic.Bool // when true, render() is a no-op (VT switched away)
}
var (
bgColor = color.RGBA{0x10, 0x12, 0x20, 0xff} // near-black slate
fgColor = color.RGBA{0xff, 0xff, 0xff, 0xff}
dimColor = color.RGBA{0xa0, 0xa6, 0xb8, 0xff}
stateOK = color.RGBA{0x4a, 0xc8, 0x82, 0xff} // green for Running
stateWait = color.RGBA{0xf0, 0xc8, 0x60, 0xff} // amber for NeedsLogin/Starting
)
// render composes the current state into an in-memory image and blits
// it to the framebuffer.
func (st *uiState) render() {
if st.paused.Load() {
return
}
w, h := st.fb.width, st.fb.height
img := image.NewRGBA(image.Rect(0, 0, w, h))
draw.Draw(img, img.Bounds(), &image.Uniform{C: bgColor}, image.Point{}, draw.Src)
shortSide := min(w, h)
// Logo, scaled to ~25% of the shorter dimension, centered
// horizontally near the top.
logoSize := shortSide / 4
logoRect := image.Rect(0, 0, logoSize, logoSize).Add(image.Point{
X: (w - logoSize) / 2,
Y: shortSide / 16,
})
xdraw.ApproxBiLinear.Scale(img, logoRect, st.logo, st.logo.Bounds(), xdraw.Over, nil)
lineH := basicfont.Face7x13.Metrics().Height.Ceil()
textTop := logoRect.Max.Y + shortSide/24
// Hide the state line when the QR code is visible (the "Scan to
// enroll" label is clear enough context).
showState := !(st.state == ipn.NeedsLogin && st.loginURL != "")
if showState {
stateColor := dimColor
switch st.state {
case ipn.Running:
stateColor = stateOK
case ipn.NeedsLogin, ipn.Starting, ipn.NoState:
stateColor = stateWait
}
drawCenteredScaled(img, fmt.Sprintf("State: %s", stateLabel(st.state)),
stateColor, w/2, textTop, 3)
}
y := textTop + 3*lineH + shortSide/40
if len(st.ips) > 0 {
drawCenteredScaled(img, "Tailscale IPs:", dimColor, w/2, y, 2)
y += 2 * lineH
for _, a := range st.ips {
drawCenteredScaled(img, a.String(), fgColor, w/2, y, 2)
y += 2*lineH + 4
}
}
// QR code with the login URL when enrollment is needed.
if st.state == ipn.NeedsLogin && st.loginURL != "" {
qrSize := shortSide / 2
q, err := qrcode.New(st.loginURL, qrcode.Medium)
if err != nil {
log.Printf("qr encode %q: %v", st.loginURL, err)
} else {
q.DisableBorder = false
qrImg := q.Image(qrSize)
qrRect := qrImg.Bounds().Add(image.Point{
X: (w - qrSize) / 2,
Y: h - qrSize - shortSide/16,
})
draw.Draw(img, qrRect, qrImg, qrImg.Bounds().Min, draw.Src)
drawCenteredScaled(img, "Scan to enroll this device",
fgColor, w/2, qrRect.Min.Y-lineH*2-8, 2)
}
}
// LAN status pinned to the bottom-left corner.
{
lanY := h - lineH - 4
var lanText string
if st.lanIP != "" {
lanText = "LAN IP: " + st.lanIP
} else if st.lanMAC != "" {
lanText = "Waiting for DHCP (" + st.lanMAC + ")"
}
if lanText != "" {
face := basicfont.Face7x13
textW := font.MeasureString(face, lanText).Ceil()
small := image.NewRGBA(image.Rect(0, 0, textW, lineH))
d := font.Drawer{
Dst: small,
Src: &image.Uniform{C: dimColor},
Face: face,
Dot: fixed.P(0, face.Metrics().Ascent.Ceil()),
}
d.DrawString(lanText)
dstRect := image.Rect(4, lanY, 4+textW, lanY+lineH)
draw.Draw(img, dstRect, small, image.Point{}, draw.Over)
}
}
st.fb.blit(img)
}
// drawCenteredScaled draws s with basicfont.Face7x13 at the given
// integer pixel scale, centered horizontally on x at top y, in col.
func drawCenteredScaled(dst *image.RGBA, s string, col color.Color, x, y, scale int) {
if s == "" {
return
}
face := basicfont.Face7x13
width := font.MeasureString(face, s).Ceil()
height := face.Metrics().Height.Ceil()
small := image.NewRGBA(image.Rect(0, 0, width, height))
d := font.Drawer{
Dst: small,
Src: &image.Uniform{C: col},
Face: face,
Dot: fixed.P(0, face.Metrics().Ascent.Ceil()),
}
d.DrawString(s)
scaledW, scaledH := width*scale, height*scale
dstRect := image.Rect(0, 0, scaledW, scaledH).Add(image.Point{
X: x - scaledW/2,
Y: y,
})
xdraw.NearestNeighbor.Scale(dst, dstRect, small, small.Bounds(), xdraw.Over, nil)
}
func stateLabel(s ipn.State) string {
switch s {
case ipn.NoState, ipn.Starting:
return "starting"
case ipn.NeedsLogin:
return "needs login"
case ipn.NeedsMachineAuth:
return "needs machine auth"
case ipn.Stopped:
return "stopped"
case ipn.Running:
return "running"
}
return strings.ToLower(s.String())
}
// framebuffer is an mmap'd Linux framebuffer device.
type framebuffer struct {
f *os.File
mem []byte
width int
height int
bpp int
lineLength int
// Bit offsets into a 32-bit pixel for each channel, from the
// fb_bitfield values returned by FBIOGET_VSCREENINFO.
redShift uint32
greenShift uint32
blueShift uint32
}
// openFramebuffer opens path, queries dimensions and pixel format via
// the FBIOGET_* ioctls, and mmaps the framebuffer memory.
//
// Only 32-bits-per-pixel framebuffers are supported. Raspberry Pi 3/4/5
// default to that.
func openFramebuffer(path string) (*framebuffer, error) {
f, err := os.OpenFile(path, os.O_RDWR, 0)
if err != nil {
return nil, err
}
var (
vbuf [160]byte // fb_var_screeninfo
fbuf [80]byte // fb_fix_screeninfo
)
if err := ioctlGet(f, fbioGetVScreenInfo, vbuf[:]); err != nil {
f.Close()
return nil, fmt.Errorf("FBIOGET_VSCREENINFO: %w", err)
}
if err := ioctlGet(f, fbioGetFScreenInfo, fbuf[:]); err != nil {
f.Close()
return nil, fmt.Errorf("FBIOGET_FSCREENINFO: %w", err)
}
fb := &framebuffer{
f: f,
width: int(binary.LittleEndian.Uint32(vbuf[vsOffXres:])),
height: int(binary.LittleEndian.Uint32(vbuf[vsOffYres:])),
bpp: int(binary.LittleEndian.Uint32(vbuf[vsOffBitsPerPixel:])),
lineLength: int(binary.LittleEndian.Uint32(fbuf[fsOffLineLength:])),
redShift: binary.LittleEndian.Uint32(vbuf[vsOffRedOffset:]),
greenShift: binary.LittleEndian.Uint32(vbuf[vsOffGreenOffset:]),
blueShift: binary.LittleEndian.Uint32(vbuf[vsOffBlueOffset:]),
}
if fb.bpp != 32 {
f.Close()
return nil, fmt.Errorf("unsupported framebuffer bpp %d (only 32 is supported)", fb.bpp)
}
memLen := int(binary.LittleEndian.Uint32(fbuf[fsOffSmemLen:]))
mem, err := unix.Mmap(int(f.Fd()), 0, memLen,
unix.PROT_READ|unix.PROT_WRITE, unix.MAP_SHARED)
if err != nil {
f.Close()
return nil, fmt.Errorf("mmap %s: %w", path, err)
}
fb.mem = mem
return fb, nil
}
func (fb *framebuffer) Close() error {
if fb.mem != nil {
unix.Munmap(fb.mem)
fb.mem = nil
}
return fb.f.Close()
}
// blit copies img into the mapped framebuffer, packing each
// image.RGBA pixel into the framebuffer's per-channel bit layout.
func (fb *framebuffer) blit(img *image.RGBA) {
srcStride := img.Stride
for y := 0; y < fb.height; y++ {
srcRow := img.Pix[y*srcStride : y*srcStride+fb.width*4]
dstRow := fb.mem[y*fb.lineLength:]
for x := 0; x < fb.width; x++ {
r := uint32(srcRow[x*4+0])
g := uint32(srcRow[x*4+1])
b := uint32(srcRow[x*4+2])
px := r<<fb.redShift | g<<fb.greenShift | b<<fb.blueShift
binary.LittleEndian.PutUint32(dstRow[x*4:], px)
}
}
}
// claimVTGraphics puts the active virtual terminal into KD_GRAPHICS so
// the kernel's framebuffer console (fbcon) stops drawing on /dev/fb0
// while fbstatus owns it. It returns a function that restores KD_TEXT.
//
// The Linux kernel applies VT mode to whatever VT is current; the open
// path /dev/tty0 always refers to the foreground VT, which on a
// headless gokrazy appliance is the only VT.
func claimVTGraphics() (restore func(), err error) {
f, err := os.OpenFile("/dev/tty0", os.O_RDWR, 0)
if err != nil {
return nil, err
}
if err := ioctlSetInt(f, kdSetMode, kdGraphics); err != nil {
f.Close()
return nil, fmt.Errorf("KDSETMODE KD_GRAPHICS: %w", err)
}
return func() {
if err := ioctlSetInt(f, kdSetMode, kdText); err != nil {
log.Printf("KDSETMODE KD_TEXT on shutdown: %v", err)
}
f.Close()
}, nil
}
// ioctlSetInt runs an ioctl with a single integer arg, like KDSETMODE.
func ioctlSetInt(f *os.File, req uintptr, arg uintptr) error {
_, _, errno := syscall.Syscall(syscall.SYS_IOCTL, f.Fd(), req, arg)
if errno != 0 {
return errno
}
return nil
}
// ioctlGet runs an ioctl that fills a struct of len(buf) bytes in buf.
// Used for the FBIOGET_* ioctls; on success buf holds the kernel's
// fb_*_screeninfo struct.
func ioctlGet(f *os.File, req uintptr, buf []byte) error {
_, _, errno := syscall.Syscall(syscall.SYS_IOCTL, f.Fd(), req,
uintptr(unsafe.Pointer(&buf[0])))
if errno != 0 {
return errno
}
return nil
}
-17
View File
@@ -1,17 +0,0 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
//go:build !linux
package main
import (
"fmt"
"os"
"runtime"
)
func main() {
fmt.Fprintf(os.Stderr, "fbstatus is only supported on Linux (got %s)\n", runtime.GOOS)
os.Exit(1)
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

+1 -5
View File
@@ -255,11 +255,7 @@ func getCredentials() (*http.Client, string) {
} else if idok && idToken != "" && oiok && oauthId != "" {
if exchangeJWTForToken, ok := tailscale.HookExchangeJWTForTokenViaWIF.GetOk(); ok {
var err error
apiKeyEnv, err = exchangeJWTForToken(context.Background(), tailscale.ExchangeJWTForTokenWIFArgs{
BaseURL: fmt.Sprintf("https://%s", *apiServer),
ClientID: oauthId,
IDToken: idToken,
})
apiKeyEnv, err = exchangeJWTForToken(context.Background(), fmt.Sprintf("https://%s", *apiServer), oauthId, idToken)
if err != nil {
log.Fatal(err)
}
+1 -1
View File
@@ -29,7 +29,7 @@ import (
const (
// tsNetDomain is the domain that this DNS nameserver has registered a handler for.
tsNetDomain = "ts.net"
// addr is the address that the UDP and TCP listeners will listen on.
// addr is the the address that the UDP and TCP listeners will listen on.
addr = ":1053"
// defaultTTL is the default TTL for DNS records in seconds.
// Set to 0 to disable caching. Can be increased when usage patterns are better understood.
+3 -9
View File
@@ -436,16 +436,14 @@ func exclusiveOwnerAnnotations(pg *tsapi.ProxyGroup, operatorID string, svc *tai
}
if svc == nil {
c := ownerAnnotationValue{OwnerRefs: []OwnerRef{ref}}
data, err := json.Marshal(c)
json, err := json.Marshal(c)
if err != nil {
return nil, fmt.Errorf("failed to marshal Tailscale Service's owner annotation contents: %w", err)
return nil, fmt.Errorf("[unexpected] unable to marshal Tailscale Service's owner annotation contents: %w, please report this", err)
}
return map[string]string{
ownerAnnotation: string(data),
ownerAnnotation: string(json),
}, nil
}
o, err := parseOwnerAnnotation(svc)
if err != nil {
return nil, err
@@ -453,19 +451,15 @@ func exclusiveOwnerAnnotations(pg *tsapi.ProxyGroup, operatorID string, svc *tai
if o == nil || len(o.OwnerRefs) == 0 {
return nil, fmt.Errorf("Tailscale Service %s exists, but does not contain owner annotation with owner references; not proceeding as this is likely a resource created by something other than the Tailscale Kubernetes operator", svc.Name)
}
if len(o.OwnerRefs) > 1 || o.OwnerRefs[0].OperatorID != operatorID {
return nil, fmt.Errorf("Tailscale Service %s is already owned by other operator(s) and cannot be shared across multiple clusters; configure a difference Service name to continue", svc.Name)
}
if o.OwnerRefs[0].Resource == nil {
return nil, fmt.Errorf("Tailscale Service %s exists, but does not reference an owning resource; not proceeding as this is likely a Service already owned by an Ingress", svc.Name)
}
if o.OwnerRefs[0].Resource.Kind != "ProxyGroup" || o.OwnerRefs[0].Resource.UID != string(pg.UID) {
return nil, fmt.Errorf("Tailscale Service %s is already owned by another resource: %#v; configure a difference Service name to continue", svc.Name, o.OwnerRefs[0].Resource)
}
if o.OwnerRefs[0].Resource.Name != pg.Name {
// ProxyGroup name can be updated in place.
o.OwnerRefs[0].Resource.Name = pg.Name
-7
View File
@@ -29,8 +29,6 @@ import (
tsoperator "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/kube/kubetypes"
"tailscale.com/net/netutil"
"tailscale.com/net/tsaddr"
"tailscale.com/tstime"
"tailscale.com/util/clientmetric"
"tailscale.com/util/set"
@@ -358,11 +356,6 @@ func validateRoutes(routes tsapi.Routes) error {
if pfx.Masked() != pfx {
errs = append(errs, fmt.Errorf("route %s has non-address bits set; expected %s", pfx, pfx.Masked()))
}
if tsaddr.IsViaPrefix(pfx) {
if err := netutil.ValidateViaPrefix(pfx); err != nil {
errs = append(errs, err)
}
}
}
return errors.Join(errs...)
}
-16
View File
@@ -145,22 +145,6 @@ func TestConnector(t *testing.T) {
expectReconciled(t, cr, "", "test")
expectEqual(t, fc, expectedSTS(t, fc, opts), removeResourceReqs)
// Set an invalid 4via6 route (site ID too large).
mustUpdate[tsapi.Connector](t, fc, "", "test", func(conn *tsapi.Connector) {
conn.Spec.SubnetRouter.AdvertiseRoutes = []tsapi.Route{"fd7a:115c:a1e0:b1a:1:0:a2c:0/116"}
})
expectReconciled(t, cr, "", "test")
// STS should still have the previous valid route, unchanged.
expectEqual(t, fc, expectedSTS(t, fc, opts), removeResourceReqs)
// Set a valid 4via6 route.
mustUpdate[tsapi.Connector](t, fc, "", "test", func(conn *tsapi.Connector) {
conn.Spec.SubnetRouter.AdvertiseRoutes = []tsapi.Route{"fd7a:115c:a1e0:b1a:0:1:a2c:0/116"}
})
opts.subnetRoutes = "fd7a:115c:a1e0:b1a:0:1:a2c:0/116"
expectReconciled(t, cr, "", "test")
expectEqual(t, fc, expectedSTS(t, fc, opts), removeResourceReqs)
// Delete the Connector.
if err = fc.Delete(context.Background(), cn); err != nil {
t.Fatalf("error deleting Connector: %v", err)
+29 -36
View File
@@ -12,6 +12,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
github.com/coder/websocket from tailscale.com/util/eventbus
github.com/coder/websocket/internal/errd from github.com/coder/websocket
github.com/coder/websocket/internal/util from github.com/coder/websocket
github.com/coder/websocket/internal/xsync from github.com/coder/websocket
github.com/creachadair/msync/trigger from tailscale.com/logtail
💣 github.com/davecgh/go-spew/spew from k8s.io/apimachinery/pkg/util/dump
W 💣 github.com/dblohm7/wingoes from tailscale.com/net/tshttpproxy+
@@ -41,7 +42,6 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
github.com/go-json-experiment/json/internal/jsonopts from github.com/go-json-experiment/json/jsontext+
github.com/go-json-experiment/json/internal/jsonwire from github.com/go-json-experiment/json/jsontext+
github.com/go-json-experiment/json/jsontext from tailscale.com/logtail+
github.com/go-json-experiment/json/v1 from tailscale.com/net/routecheck+
github.com/go-logr/logr from github.com/go-logr/logr/slogr+
github.com/go-logr/logr/slogr from github.com/go-logr/zapr
github.com/go-logr/zapr from sigs.k8s.io/controller-runtime/pkg/log/zap+
@@ -730,18 +730,15 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/envknob from tailscale.com/client/local+
tailscale.com/envknob/featureknob from tailscale.com/client/web+
tailscale.com/feature from tailscale.com/ipn/ipnext+
tailscale.com/feature/acme from tailscale.com/tsnet
tailscale.com/feature/buildfeatures from tailscale.com/wgengine/magicsock+
tailscale.com/feature/c2n from tailscale.com/tsnet
tailscale.com/feature/condlite/expvar from tailscale.com/wgengine/magicsock
tailscale.com/feature/condregister/netlog from tailscale.com/tsnet
tailscale.com/feature/condregister/oauthkey from tailscale.com/tsnet
tailscale.com/feature/condregister/portmapper from tailscale.com/tsnet
tailscale.com/feature/condregister/useproxy from tailscale.com/tsnet
tailscale.com/feature/netlog from tailscale.com/feature/condregister/netlog
tailscale.com/feature/oauthkey from tailscale.com/feature/condregister/oauthkey
tailscale.com/feature/portmapper from tailscale.com/feature/condregister/portmapper
tailscale.com/feature/serviceclientprefs/serviceclient from tailscale.com/client/local
tailscale.com/feature/syspolicy from tailscale.com/logpolicy
tailscale.com/feature/useproxy from tailscale.com/feature/condregister/useproxy
tailscale.com/health from tailscale.com/control/controlclient+
tailscale.com/health/healthmsg from tailscale.com/ipn/ipnlocal
@@ -755,18 +752,16 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/ipn/ipnlocal/netmapcache from tailscale.com/ipn/ipnlocal
tailscale.com/ipn/ipnstate from tailscale.com/client/local+
tailscale.com/ipn/localapi from tailscale.com/tsnet
tailscale.com/ipn/store from tailscale.com/ipn/store/kubestore+
tailscale.com/ipn/store from tailscale.com/ipn/ipnlocal+
tailscale.com/ipn/store/kubestore from tailscale.com/cmd/k8s-operator
tailscale.com/ipn/store/mem from tailscale.com/ipn/ipnlocal+
tailscale.com/k8s-operator from tailscale.com/cmd/k8s-operator+
tailscale.com/k8s-operator/api-proxy from tailscale.com/cmd/k8s-operator
tailscale.com/k8s-operator/apis from tailscale.com/k8s-operator/apis/v1alpha1
tailscale.com/k8s-operator/apis/v1alpha1 from tailscale.com/cmd/k8s-operator+
tailscale.com/k8s-operator/reconciler from tailscale.com/k8s-operator/reconciler/tailnet+
tailscale.com/k8s-operator/reconciler/peerrelay from tailscale.com/cmd/k8s-operator
tailscale.com/k8s-operator/reconciler from tailscale.com/k8s-operator/reconciler/tailnet
tailscale.com/k8s-operator/reconciler/proxygrouppolicy from tailscale.com/cmd/k8s-operator
tailscale.com/k8s-operator/reconciler/tailnet from tailscale.com/cmd/k8s-operator
tailscale.com/k8s-operator/reconciler/tailscaled from tailscale.com/k8s-operator/reconciler/peerrelay
tailscale.com/k8s-operator/sessionrecording from tailscale.com/k8s-operator/api-proxy
tailscale.com/k8s-operator/sessionrecording/spdy from tailscale.com/k8s-operator/sessionrecording
tailscale.com/k8s-operator/sessionrecording/tsrecorder from tailscale.com/k8s-operator/sessionrecording+
@@ -787,6 +782,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/metrics from tailscale.com/tsweb+
tailscale.com/net/bakedroots from tailscale.com/net/tlsdial+
💣 tailscale.com/net/batching from tailscale.com/wgengine/magicsock
tailscale.com/net/captivedetection from tailscale.com/ipn/ipnlocal+
tailscale.com/net/dns from tailscale.com/ipn/ipnlocal+
tailscale.com/net/dns/publicdns from tailscale.com/net/dns+
tailscale.com/net/dns/resolvconffile from tailscale.com/cmd/k8s-operator+
@@ -797,7 +793,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/net/ipset from tailscale.com/ipn/ipnlocal+
tailscale.com/net/memnet from tailscale.com/tsnet
tailscale.com/net/netaddr from tailscale.com/ipn+
tailscale.com/net/netcheck from tailscale.com/wgengine/magicsock
tailscale.com/net/netcheck from tailscale.com/ipn/ipnlocal+
tailscale.com/net/neterror from tailscale.com/net/dns/resolver+
tailscale.com/net/netkernelconf from tailscale.com/ipn/ipnlocal
tailscale.com/net/netknob from tailscale.com/logpolicy+
@@ -811,16 +807,12 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/net/portmapper from tailscale.com/feature/portmapper
tailscale.com/net/portmapper/portmappertype from tailscale.com/net/netcheck+
tailscale.com/net/proxymux from tailscale.com/tsnet
tailscale.com/net/routecheck from tailscale.com/client/local+
tailscale.com/net/routecheck/peernode from tailscale.com/ipn/ipnlocal+
tailscale.com/net/routemanager from tailscale.com/ipn/ipnlocal+
💣 tailscale.com/net/sockopts from tailscale.com/wgengine/magicsock
tailscale.com/net/socks5 from tailscale.com/tsnet
tailscale.com/net/sockstats from tailscale.com/control/controlclient+
tailscale.com/net/stun from tailscale.com/ipn/localapi+
tailscale.com/net/tlsdial from tailscale.com/control/controlclient+
tailscale.com/net/tlsdial/blockblame from tailscale.com/net/tlsdial
tailscale.com/net/traffic from tailscale.com/ipn/ipnlocal+
tailscale.com/net/tsaddr from tailscale.com/client/web+
tailscale.com/net/tsdial from tailscale.com/control/controlclient+
💣 tailscale.com/net/tshttpproxy from tailscale.com/feature/useproxy
@@ -834,7 +826,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/sessionrecording from tailscale.com/k8s-operator/sessionrecording+
tailscale.com/syncs from tailscale.com/control/controlknobs+
tailscale.com/tailcfg from tailscale.com/client/local+
tailscale.com/tempfork/acme from tailscale.com/feature/acme
tailscale.com/tempfork/acme from tailscale.com/ipn/ipnlocal
tailscale.com/tempfork/heap from tailscale.com/wgengine/magicsock
tailscale.com/tempfork/httprec from tailscale.com/feature/c2n
tailscale.com/tka from tailscale.com/client/local+
@@ -856,7 +848,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/types/lazy from tailscale.com/ipn/ipnlocal+
tailscale.com/types/logger from tailscale.com/appc+
tailscale.com/types/logid from tailscale.com/ipn/ipnlocal+
tailscale.com/types/mapx from tailscale.com/ipn/ipnext+
tailscale.com/types/mapx from tailscale.com/ipn/ipnext
tailscale.com/types/netlogfunc from tailscale.com/net/tstun+
tailscale.com/types/netlogtype from tailscale.com/wgengine/netlog
tailscale.com/types/netmap from tailscale.com/control/controlclient+
@@ -878,7 +870,6 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
LW tailscale.com/util/cmpver from tailscale.com/net/dns+
tailscale.com/util/ctxkey from tailscale.com/client/tailscale/apitype+
💣 tailscale.com/util/deephash from tailscale.com/util/syspolicy/setting
tailscale.com/util/def from tailscale.com/ipn/localapi
L 💣 tailscale.com/util/dirwalk from tailscale.com/metrics
tailscale.com/util/dnsname from tailscale.com/appc+
tailscale.com/util/eventbus from tailscale.com/tsd+
@@ -901,15 +892,16 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/util/set from tailscale.com/cmd/k8s-operator+
tailscale.com/util/singleflight from tailscale.com/control/controlclient+
tailscale.com/util/slicesx from tailscale.com/appc+
tailscale.com/util/syspolicy from tailscale.com/feature/syspolicy
tailscale.com/util/syspolicy/internal from tailscale.com/util/syspolicy/setting+
tailscale.com/util/syspolicy/internal/loggerx from tailscale.com/util/syspolicy/internal/metrics+
tailscale.com/util/syspolicy/internal/metrics from tailscale.com/util/syspolicy/source
tailscale.com/util/syspolicy/pkey from tailscale.com/control/controlclient+
tailscale.com/util/syspolicy/policyclient from tailscale.com/control/controlclient+
tailscale.com/util/syspolicy/ptype from tailscale.com/ipn/ipnlocal+
tailscale.com/util/syspolicy/rsop from tailscale.com/ipn/localapi
tailscale.com/util/syspolicy/setting from tailscale.com/client/local+
tailscale.com/util/syspolicy/source from tailscale.com/util/syspolicy/rsop
tailscale.com/util/syspolicy/ptype from tailscale.com/util/syspolicy+
tailscale.com/util/syspolicy/rsop from tailscale.com/util/syspolicy+
tailscale.com/util/syspolicy/setting from tailscale.com/util/syspolicy+
tailscale.com/util/syspolicy/source from tailscale.com/util/syspolicy+
tailscale.com/util/testenv from tailscale.com/control/controlclient+
tailscale.com/util/truncate from tailscale.com/logtail
tailscale.com/util/usermetric from tailscale.com/health+
@@ -926,11 +918,12 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
tailscale.com/wgengine/filter from tailscale.com/control/controlclient+
tailscale.com/wgengine/filter/filtertype from tailscale.com/types/netmap+
💣 tailscale.com/wgengine/magicsock from tailscale.com/ipn/ipnlocal+
tailscale.com/wgengine/netlog from tailscale.com/feature/netlog
tailscale.com/wgengine/netlog from tailscale.com/wgengine
tailscale.com/wgengine/netstack from tailscale.com/tsnet
tailscale.com/wgengine/netstack/gro from tailscale.com/net/tstun+
tailscale.com/wgengine/router from tailscale.com/ipn/ipnlocal+
tailscale.com/wgengine/wgcfg from tailscale.com/ipn/ipnlocal+
tailscale.com/wgengine/wgcfg/nmcfg from tailscale.com/ipn/ipnlocal
💣 tailscale.com/wgengine/wgint from tailscale.com/wgengine+
tailscale.com/wgengine/wglog from tailscale.com/wgengine
golang.org/x/crypto/argon2 from tailscale.com/tka
@@ -968,7 +961,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
D golang.org/x/net/route from tailscale.com/net/netmon+
golang.org/x/net/websocket from tailscale.com/k8s-operator/sessionrecording/ws
golang.org/x/oauth2 from golang.org/x/oauth2/clientcredentials+
golang.org/x/oauth2/clientcredentials from tailscale.com/client/tailscale/v2+
golang.org/x/oauth2/clientcredentials from tailscale.com/cmd/k8s-operator+
golang.org/x/oauth2/internal from golang.org/x/oauth2+
golang.org/x/sync/errgroup from github.com/mdlayher/socket+
golang.org/x/sys/cpu from github.com/tailscale/certstore+
@@ -1025,22 +1018,22 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
crypto/internal/boring/bbig from crypto/ecdsa+
crypto/internal/boring/sig from crypto/internal/boring
crypto/internal/constanttime from crypto/internal/fips140/edwards25519+
crypto/internal/fips140 from crypto/fips140+
crypto/internal/fips140 from crypto/internal/fips140/aes+
crypto/internal/fips140/aes from crypto/aes+
crypto/internal/fips140/aes/gcm from crypto/cipher+
crypto/internal/fips140/alias from crypto/cipher+
crypto/internal/fips140/bigmod from crypto/internal/fips140/ecdsa+
crypto/internal/fips140/check from crypto/fips140+
crypto/internal/fips140/drbg from crypto/hpke+
crypto/internal/fips140/check from crypto/internal/fips140/aes+
crypto/internal/fips140/drbg from crypto/internal/fips140/aes/gcm+
crypto/internal/fips140/ecdh from crypto/ecdh
crypto/internal/fips140/ecdsa from crypto/ecdsa
crypto/internal/fips140/ed25519 from crypto/ed25519
crypto/internal/fips140/edwards25519 from crypto/internal/fips140/ed25519
crypto/internal/fips140/edwards25519/field from crypto/ecdh+
crypto/internal/fips140/hkdf from crypto/hkdf+
crypto/internal/fips140/hkdf from crypto/internal/fips140/tls13+
crypto/internal/fips140/hmac from crypto/hmac+
crypto/internal/fips140/mlkem from crypto/mlkem
crypto/internal/fips140/nistec from crypto/ecdsa+
crypto/internal/fips140/nistec from crypto/elliptic+
crypto/internal/fips140/nistec/fiat from crypto/internal/fips140/nistec
crypto/internal/fips140/rsa from crypto/rsa
crypto/internal/fips140/sha256 from crypto/internal/fips140/check+
@@ -1105,7 +1098,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
hash from compress/zlib+
hash/adler32 from compress/zlib
hash/crc32 from compress/gzip+
hash/fnv from google.golang.org/protobuf/internal/detrand+
hash/fnv from google.golang.org/protobuf/internal/detrand
hash/maphash from go4.org/mem
html from html/template+
html/template from tailscale.com/util/eventbus
@@ -1120,14 +1113,14 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
internal/filepathlite from os+
internal/fmtsort from fmt+
internal/goarch from crypto/internal/fips140deps/cpu+
internal/godebug from crypto/ed25519+
internal/godebug from crypto/internal/fips140deps/godebug+
internal/godebugs from internal/godebug+
internal/goexperiment from net/http/pprof+
internal/goos from crypto/x509+
internal/lazyregexp from go/doc
internal/msan from internal/runtime/maps+
internal/nettrace from net+
internal/oserror from internal/syscall/windows+
internal/oserror from io/fs+
internal/poll from net+
internal/profile from net/http/pprof
internal/profilerecord from runtime+
@@ -1137,9 +1130,9 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
internal/runtime/atomic from internal/runtime/exithook+
L internal/runtime/cgroup from runtime
internal/runtime/exithook from runtime
internal/runtime/gc from internal/runtime/gc/scan+
internal/runtime/gc from runtime+
internal/runtime/gc/scan from runtime
internal/runtime/maps from hash/maphash+
internal/runtime/maps from reflect+
internal/runtime/math from internal/runtime/maps+
internal/runtime/pprof/label from runtime+
internal/runtime/sys from crypto/subtle+
@@ -1153,7 +1146,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
internal/synctest from sync
internal/syscall/execenv from os+
LD internal/syscall/unix from crypto/internal/sysrand+
W internal/syscall/windows from crypto/internal/fips140deps/time+
W internal/syscall/windows from crypto/internal/sysrand+
W internal/syscall/windows/registry from mime+
W internal/syscall/windows/sysdll from internal/syscall/windows+
internal/testlog from os
@@ -1161,7 +1154,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
internal/unsafeheader from internal/reflectlite+
io from bufio+
io/fs from crypto/x509+
io/ioutil from github.com/google/gnostic-models/compiler+
io/ioutil from github.com/godbus/dbus/v5+
iter from go/ast+
log from expvar+
log/internal from log+
@@ -1198,7 +1191,7 @@ tailscale.com/cmd/k8s-operator dependencies: (generated by github.com/tailscale/
regexp from github.com/davecgh/go-spew/spew+
regexp/syntax from regexp
runtime from crypto/internal/fips140+
runtime/debug from github.com/klauspost/compress/zstd+
runtime/debug from github.com/coder/websocket/internal/xsync+
runtime/metrics from github.com/prometheus/client_golang/prometheus+
runtime/pprof from net/http/pprof+
runtime/trace from net/http/pprof
@@ -10,4 +10,3 @@
/recorder.yaml
/tailnet.yaml
/proxygrouppolicy.yaml
/peerrelay.yaml
@@ -6,9 +6,6 @@ kind: Deployment
metadata:
name: operator
namespace: {{ .Release.Namespace }}
{{- if .Values.annotations }}
annotations: {{- toYaml .Values.annotations | nindent 4 }}
{{- end }}
spec:
replicas: 1
strategy:
@@ -81,10 +78,6 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OPERATOR_SERVICE_ACCOUNT_NAME
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
- name: OPERATOR_LOGIN_SERVER
value: {{ .Values.loginServer }}
- name: OPERATOR_INGRESS_CLASS_NAME
@@ -124,8 +117,6 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.uid
- name: OPERATOR_SHARED_ACME_ACCOUNT_KEY
value: {{ .Values.operatorConfig.sharedACMEAccountKey | quote }}
{{- with .Values.operatorConfig.extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
@@ -40,9 +40,6 @@ rules:
- apiGroups: ["tailscale.com"]
resources: ["tailnets", "tailnets/status"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["tailscale.com"]
resources: ["peerrelays", "peerrelays/status"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["tailscale.com"]
resources: ["proxygrouppolicies", "proxygrouppolicies/status"]
verbs: ["get", "list", "watch", "update"]
@@ -79,10 +76,6 @@ rules:
- apiGroups: [""]
resources: ["secrets", "serviceaccounts", "configmaps"]
verbs: ["create","delete","deletecollection","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["serviceaccounts/token"]
resourceNames: ["operator"]
verbs: ["create"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get","list","watch", "update"]
-10
View File
@@ -62,9 +62,6 @@ operatorConfig:
resources: {}
# Specifies annotations for deployment
annotations: {}
podAnnotations: {}
podLabels: {}
@@ -87,13 +84,6 @@ operatorConfig:
# - name: EXTRA_VAR2
# value: "value2"
# Default for the tailscale.com/share-acme-account annotation on new
# ProxyGroups. When true, the operator provisions a shared per-tailnet
# ACME account key Secret and configures proxies to use it, preserving
# Let's Encrypt's ARI "replaces" renewal exemption across pod restarts
# and ProxyGroup recreation. See #18251.
sharedACMEAccountKey: false
# In the case that you already have a tailscale ingressclass in your cluster (or vcluster), you can disable the creation here
ingressClass:
# Allows for customization of the ingress class name used by the operator to identify ingresses to reconcile. This does
@@ -1,264 +0,0 @@
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.0
name: peerrelays.tailscale.com
spec:
group: tailscale.com
names:
kind: PeerRelay
listKind: PeerRelayList
plural: peerrelays
shortNames:
- pr
singular: peerrelay
scope: Cluster
versions:
- additionalPrinterColumns:
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Status of the deployed PeerRelay resources.
jsonPath: .status.conditions[?(@.type == "PeerRelayReady")].reason
name: Status
type: string
- description: Public addresses the peer relay replicas are reachable on.
jsonPath: .status.endpoints[*].address
name: Endpoints
type: string
name: v1alpha1
schema:
openAPIV3Schema:
type: object
required:
- metadata
- spec
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
Spec describes the desired state of the PeerRelay.
More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
type: object
properties:
aws:
description: |-
AWS contains configuration for pinning each replica to a specific AWS Elastic IP and subnet. Only meaningful
when running on EKS with the AWS Load Balancer Controller. When set, the per-replica values override any
aws-load-balancer-eip-allocations or aws-load-balancer-subnets values supplied via spec.service.annotations.
type: object
required:
- elasticIPs
properties:
elasticIPs:
description: |-
ElasticIPs pins each replica to a specific AWS EIP allocation and subnet. Only meaningful when Network Load
Balancers are provisioned by the AWS Load Balancer Controller. ElasticIPs supplies one allocation-subnet pair
per replica: replica N uses ElasticIPs[N]. The list must be at least as long as spec.replicas so every replica
has a distinct EIP; extra entries are permitted so that scale-up doesn't immediately trip validation.
When set, the reconciler stamps
service.beta.kubernetes.io/aws-load-balancer-eip-allocations and
service.beta.kubernetes.io/aws-load-balancer-subnets on each per-replica Service, overriding any values in
spec.service.annotations.
type: array
minItems: 1
items:
description: PeerRelayAWSElasticIP pairs an EIP allocation with the subnet in the same AZ.
type: object
required:
- allocationID
- subnetID
properties:
allocationID:
description: |-
AllocationID is the AWS EIP allocation ID (e.g. eipalloc-0123abcd) whose public IP this replica is reachable
on. Stamped as service.beta.kubernetes.io/aws-load-balancer-eip-allocations on the replica's Service.
type: string
pattern: ^eipalloc-[0-9a-f]+$
subnetID:
description: |-
SubnetID is the AWS subnet in the same availability zone as AllocationID (e.g. subnet-0123abcd). Stamped as
service.beta.kubernetes.io/aws-load-balancer-subnets on the replica's Service so the NLB is provisioned in
the same AZ as the EIP.
type: string
pattern: ^subnet-[0-9a-f]+$
x-kubernetes-list-type: atomic
hostnamePrefix:
description: |-
HostnamePrefix specifies the hostname prefix for each
replica. Each device will have the integer number
from its StatefulSet pod appended to this prefix to form the full hostname.
HostnamePrefix can contain lower case letters, numbers and dashes, it
must not start with a dash and must be between 1 and 62 characters long.
type: string
pattern: ^[a-z0-9][a-z0-9-]{0,61}$
proxyClass:
description: |-
ProxyClass is the name of the ProxyClass custom resource that
contains configuration options that should be applied to the
resources created for this PeerRelay. If unset, the operator will
create resources with the default configuration.
type: string
replicas:
description: |-
Replicas specifies how many devices to create. Set this to enable
high availability for peer relays.
https://tailscale.com/kb/1115/high-availability. Defaults to 1.
type: integer
format: int32
default: 1
minimum: 0
service:
description: Service contains configuration values to modify the LoadBalancer service used to expose the peer relay.
type: object
properties:
annotations:
description: |-
Annotations to apply to the LoadBalancer service. Any annotations that conflict with those used by known
cloud providers to ensure IP addresses rather than DNS names are ignored.
type: object
additionalProperties:
type: string
tags:
description: |-
Tags that the Tailscale node will be tagged with.
Defaults to [tag:k8s].
To autoapprove the device defined by a PeerRelay,
you can configure Tailscale ACLs to give these tags the necessary
permissions.
See https://tailscale.com/kb/1337/acl-syntax#autoapprovers.
If you specify custom tags here, you must also make the operator an owner of these tags.
See https://tailscale.com/kb/1236/kubernetes-operator/#setting-up-the-kubernetes-operator.
Tags cannot be changed once a PeerRelay node has been created.
Tag values must be in form ^tag:[a-zA-Z][a-zA-Z0-9-]*$.
type: array
items:
type: string
pattern: ^tag:[a-zA-Z][a-zA-Z0-9-]*$
tailnet:
description: |-
Tailnet specifies the tailnet this PeerRelay should join. If blank, the default tailnet is used. When set, this
name must match that of a valid Tailnet resource. This field is immutable and cannot be changed once set.
type: string
x-kubernetes-validations:
- rule: self == oldSelf
message: PeerRelay tailnet is immutable
x-kubernetes-validations:
- rule: '!has(self.aws) || !has(self.aws.elasticIPs) || self.aws.elasticIPs.size() >= self.replicas'
message: spec.aws.elasticIPs must contain at least one entry per replica
status:
description: |-
Status describes the status of the PeerRelay. This is set
and managed by the Tailscale operator.
type: object
properties:
conditions:
type: array
items:
description: Condition contains details for one aspect of the current state of this API Resource.
type: object
required:
- lastTransitionTime
- message
- reason
- status
- type
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
type: string
format: date-time
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
type: string
maxLength: 32768
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
type: integer
format: int64
minimum: 0
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
type: string
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
status:
description: status of the condition, one of True, False, Unknown.
type: string
enum:
- "True"
- "False"
- Unknown
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
type: string
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
endpoints:
description: |-
Endpoints lists the public address:port pairs each peer relay replica is reachable on. There is one entry
per replica whose LoadBalancer Service has been assigned a public address; entries appear as the underlying
cloud provisions each Service.
type: array
items:
type: object
required:
- address
- port
- replica
properties:
address:
description: |-
Address is the public IP or hostname the cloud has allocated for this replica's LoadBalancer Service.
Peers reach this relay by connecting to Address:Port over UDP.
type: string
port:
description: Port is the UDP port the peer relay listens on.
type: integer
format: int32
replica:
description: Replica is the zero-based index of the peer relay replica this endpoint targets.
type: integer
format: int32
x-kubernetes-list-map-keys:
- replica
x-kubernetes-list-type: map
served: true
storage: true
subresources:
status: {}
@@ -58,18 +58,15 @@ spec:
- credentials
properties:
credentials:
description: Denotes the location of the credentials to use for authenticating with this Tailnet.
description: Denotes the location of the OAuth credentials to use for authenticating with this Tailnet.
type: object
required:
- secretName
properties:
secretName:
description: |-
The name of the secret containing the credentials used to authenticate with this Tailnet. The secret must always
contain a "client_id" field. To authenticate with a static OAuth client, also set "client_secret". To authenticate
via workload identity federation, set "audience" to the audience value expected by the Tailscale OAuth
client; the operator will mint a ServiceAccount token for itself with that audience and exchange it for an API
token. "client_secret" and "audience" are mutually exclusive.
The name of the secret containing the OAuth credentials. This secret must contain two fields "client_id" and
"client_secret".
type: string
loginUrl:
description: URL of the control plane to be used by all resources managed by the operator using this Tailnet.
+3 -295
View File
@@ -1463,271 +1463,6 @@ spec:
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.0
name: peerrelays.tailscale.com
spec:
group: tailscale.com
names:
kind: PeerRelay
listKind: PeerRelayList
plural: peerrelays
shortNames:
- pr
singular: peerrelay
scope: Cluster
versions:
- additionalPrinterColumns:
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Status of the deployed PeerRelay resources.
jsonPath: .status.conditions[?(@.type == "PeerRelayReady")].reason
name: Status
type: string
- description: Public addresses the peer relay replicas are reachable on.
jsonPath: .status.endpoints[*].address
name: Endpoints
type: string
name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
Spec describes the desired state of the PeerRelay.
More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
properties:
aws:
description: |-
AWS contains configuration for pinning each replica to a specific AWS Elastic IP and subnet. Only meaningful
when running on EKS with the AWS Load Balancer Controller. When set, the per-replica values override any
aws-load-balancer-eip-allocations or aws-load-balancer-subnets values supplied via spec.service.annotations.
properties:
elasticIPs:
description: |-
ElasticIPs pins each replica to a specific AWS EIP allocation and subnet. Only meaningful when Network Load
Balancers are provisioned by the AWS Load Balancer Controller. ElasticIPs supplies one allocation-subnet pair
per replica: replica N uses ElasticIPs[N]. The list must be at least as long as spec.replicas so every replica
has a distinct EIP; extra entries are permitted so that scale-up doesn't immediately trip validation.
When set, the reconciler stamps
service.beta.kubernetes.io/aws-load-balancer-eip-allocations and
service.beta.kubernetes.io/aws-load-balancer-subnets on each per-replica Service, overriding any values in
spec.service.annotations.
items:
description: PeerRelayAWSElasticIP pairs an EIP allocation with the subnet in the same AZ.
properties:
allocationID:
description: |-
AllocationID is the AWS EIP allocation ID (e.g. eipalloc-0123abcd) whose public IP this replica is reachable
on. Stamped as service.beta.kubernetes.io/aws-load-balancer-eip-allocations on the replica's Service.
pattern: ^eipalloc-[0-9a-f]+$
type: string
subnetID:
description: |-
SubnetID is the AWS subnet in the same availability zone as AllocationID (e.g. subnet-0123abcd). Stamped as
service.beta.kubernetes.io/aws-load-balancer-subnets on the replica's Service so the NLB is provisioned in
the same AZ as the EIP.
pattern: ^subnet-[0-9a-f]+$
type: string
required:
- allocationID
- subnetID
type: object
minItems: 1
type: array
x-kubernetes-list-type: atomic
required:
- elasticIPs
type: object
hostnamePrefix:
description: |-
HostnamePrefix specifies the hostname prefix for each
replica. Each device will have the integer number
from its StatefulSet pod appended to this prefix to form the full hostname.
HostnamePrefix can contain lower case letters, numbers and dashes, it
must not start with a dash and must be between 1 and 62 characters long.
pattern: ^[a-z0-9][a-z0-9-]{0,61}$
type: string
proxyClass:
description: |-
ProxyClass is the name of the ProxyClass custom resource that
contains configuration options that should be applied to the
resources created for this PeerRelay. If unset, the operator will
create resources with the default configuration.
type: string
replicas:
default: 1
description: |-
Replicas specifies how many devices to create. Set this to enable
high availability for peer relays.
https://tailscale.com/kb/1115/high-availability. Defaults to 1.
format: int32
minimum: 0
type: integer
service:
description: Service contains configuration values to modify the LoadBalancer service used to expose the peer relay.
properties:
annotations:
additionalProperties:
type: string
description: |-
Annotations to apply to the LoadBalancer service. Any annotations that conflict with those used by known
cloud providers to ensure IP addresses rather than DNS names are ignored.
type: object
type: object
tags:
description: |-
Tags that the Tailscale node will be tagged with.
Defaults to [tag:k8s].
To autoapprove the device defined by a PeerRelay,
you can configure Tailscale ACLs to give these tags the necessary
permissions.
See https://tailscale.com/kb/1337/acl-syntax#autoapprovers.
If you specify custom tags here, you must also make the operator an owner of these tags.
See https://tailscale.com/kb/1236/kubernetes-operator/#setting-up-the-kubernetes-operator.
Tags cannot be changed once a PeerRelay node has been created.
Tag values must be in form ^tag:[a-zA-Z][a-zA-Z0-9-]*$.
items:
pattern: ^tag:[a-zA-Z][a-zA-Z0-9-]*$
type: string
type: array
tailnet:
description: |-
Tailnet specifies the tailnet this PeerRelay should join. If blank, the default tailnet is used. When set, this
name must match that of a valid Tailnet resource. This field is immutable and cannot be changed once set.
type: string
x-kubernetes-validations:
- message: PeerRelay tailnet is immutable
rule: self == oldSelf
type: object
x-kubernetes-validations:
- message: spec.aws.elasticIPs must contain at least one entry per replica
rule: '!has(self.aws) || !has(self.aws.elasticIPs) || self.aws.elasticIPs.size() >= self.replicas'
status:
description: |-
Status describes the status of the PeerRelay. This is set
and managed by the Tailscale operator.
properties:
conditions:
items:
description: Condition contains details for one aspect of the current state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
endpoints:
description: |-
Endpoints lists the public address:port pairs each peer relay replica is reachable on. There is one entry
per replica whose LoadBalancer Service has been assigned a public address; entries appear as the underlying
cloud provisions each Service.
items:
properties:
address:
description: |-
Address is the public IP or hostname the cloud has allocated for this replica's LoadBalancer Service.
Peers reach this relay by connecting to Address:Port over UDP.
type: string
port:
description: Port is the UDP port the peer relay listens on.
format: int32
type: integer
replica:
description: Replica is the zero-based index of the peer relay replica this endpoint targets.
format: int32
type: integer
required:
- address
- port
- replica
type: object
type: array
x-kubernetes-list-map-keys:
- replica
x-kubernetes-list-type: map
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.0
@@ -6416,15 +6151,12 @@ spec:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
properties:
credentials:
description: Denotes the location of the credentials to use for authenticating with this Tailnet.
description: Denotes the location of the OAuth credentials to use for authenticating with this Tailnet.
properties:
secretName:
description: |-
The name of the secret containing the credentials used to authenticate with this Tailnet. The secret must always
contain a "client_id" field. To authenticate with a static OAuth client, also set "client_secret". To authenticate
via workload identity federation, set "audience" to the audience value expected by the Tailscale OAuth
client; the operator will mint a ServiceAccount token for itself with that audience and exchange it for an API
token. "client_secret" and "audience" are mutually exclusive.
The name of the secret containing the OAuth credentials. This secret must contain two fields "client_id" and
"client_secret".
type: string
required:
- secretName
@@ -6600,16 +6332,6 @@ rules:
- list
- watch
- update
- apiGroups:
- tailscale.com
resources:
- peerrelays
- peerrelays/status
verbs:
- get
- list
- watch
- update
- apiGroups:
- tailscale.com
resources:
@@ -6687,14 +6409,6 @@ rules:
- patch
- update
- watch
- apiGroups:
- ""
resourceNames:
- operator
resources:
- serviceaccounts/token
verbs:
- create
- apiGroups:
- ""
resources:
@@ -6846,10 +6560,6 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OPERATOR_SERVICE_ACCOUNT_NAME
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
- name: OPERATOR_LOGIN_SERVER
value: null
- name: OPERATOR_INGRESS_CLASS_NAME
@@ -6874,8 +6584,6 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.uid
- name: OPERATOR_SHARED_ACME_ACCOUNT_KEY
value: "false"
image: tailscale/k8s-operator:stable
imagePullPolicy: Always
name: operator
+13 -25
View File
@@ -22,7 +22,6 @@ import (
"k8s.io/utils/net"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
operatorutils "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/util/mak"
@@ -107,7 +106,6 @@ func (dnsRR *dnsRecordsReconciler) Reconcile(ctx context.Context, req reconcile.
if err := dnsRR.maybeProvision(ctx, proxySvc, logger); err != nil {
if strings.Contains(err.Error(), optimisticLockErrorMsg) {
logger.Infof("optimistic lock error, retrying: %s", err)
return reconcile.Result{RequeueAfter: shortRequeue}, nil
} else {
return reconcile.Result{}, err
}
@@ -283,26 +281,19 @@ func (dnsRR *dnsRecordsReconciler) fqdnForDNSRecord(ctx context.Context, proxySv
if err := dnsRR.Get(ctx, parentName, ing); err != nil {
return "", err
}
if len(ing.Status.LoadBalancer.Ingress) == 0 {
return "", nil
}
return ing.Status.LoadBalancer.Ingress[0].Hostname, nil
}
if isManagedByType(proxySvc, serviceTypeSvc) {
var svc corev1.Service
err := dnsRR.Get(ctx, parentName, &svc)
switch {
case apierrors.IsNotFound(err):
logger.Warnf("parent Service for egress proxy %q not found", proxySvc.Name)
svc := new(corev1.Service)
if err := dnsRR.Get(ctx, parentName, svc); apierrors.IsNotFound(err) {
logger.Infof("[unexpected] parent Service for egress proxy %s not found", proxySvc.Name)
return "", nil
case err != nil:
} else if err != nil {
return "", err
}
return svc.Annotations[AnnotationTailnetTargetFQDN], nil
}
return "", nil
@@ -312,31 +303,28 @@ func (dnsRR *dnsRecordsReconciler) fqdnForDNSRecord(ctx context.Context, proxySv
// ConfigMap. At this point the in-cluster ts.net nameserver is expected to be
// successfully created together with the ConfigMap.
func (dnsRR *dnsRecordsReconciler) updateDNSConfig(ctx context.Context, update func(*operatorutils.Records)) error {
var cm corev1.ConfigMap
err := dnsRR.Get(ctx, types.NamespacedName{Name: operatorutils.DNSRecordsCMName, Namespace: dnsRR.tsNamespace}, &cm)
switch {
case apierrors.IsNotFound(err):
dnsRR.logger.Warn("dnsrecords ConfigMap not found in cluster. Not updating DNS records. Please open an issue and attach operator logs.")
cm := &corev1.ConfigMap{}
err := dnsRR.Get(ctx, types.NamespacedName{Name: operatorutils.DNSRecordsCMName, Namespace: dnsRR.tsNamespace}, cm)
if apierrors.IsNotFound(err) {
dnsRR.logger.Info("[unexpected] dnsrecords ConfigMap not found in cluster. Not updating DNS records. Please open an issue and attach operator logs.")
return nil
case err != nil:
return fmt.Errorf("failed to retrieve dnsrecords ConfigMap: %w", err)
}
if err != nil {
return fmt.Errorf("error retrieving dnsrecords ConfigMap: %w", err)
}
dnsRecords := operatorutils.Records{Version: operatorutils.Alpha1Version, IP4: map[string][]string{}}
if cm.Data != nil && cm.Data[operatorutils.DNSRecordsCMKey] != "" {
if err = json.Unmarshal([]byte(cm.Data[operatorutils.DNSRecordsCMKey]), &dnsRecords); err != nil {
if err := json.Unmarshal([]byte(cm.Data[operatorutils.DNSRecordsCMKey]), &dnsRecords); err != nil {
return err
}
}
update(&dnsRecords)
dnsRecordsBs, err := json.Marshal(dnsRecords)
if err != nil {
return fmt.Errorf("error marshalling DNS records: %w", err)
}
mak.Set(&cm.Data, operatorutils.DNSRecordsCMKey, string(dnsRecordsBs))
return dnsRR.Update(ctx, &cm)
return dnsRR.Update(ctx, cm)
}
// isSvcForFQDNEgressProxy returns true if the Service is a headless Service
-85
View File
@@ -8,7 +8,6 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"testing"
@@ -22,8 +21,6 @@ import (
"k8s.io/apimachinery/pkg/util/intstr"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/controller-runtime/pkg/client/interceptor"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
operatorutils "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/kube/kubetypes"
@@ -293,88 +290,6 @@ func TestDNSRecordsReconcilerErrorCases(t *testing.T) {
}
}
func TestDNSRecordsReconcilerOptimisticLockError(t *testing.T) {
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
funcs := interceptor.Funcs{
Update: func(ctx context.Context, client client.WithWatch, obj client.Object, opts ...client.UpdateOption) error {
return errors.New(optimisticLockErrorMsg)
},
}
dnsCfg := &tsapi.DNSConfig{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
TypeMeta: metav1.TypeMeta{Kind: "DNSConfig"},
Spec: tsapi.DNSConfigSpec{Nameserver: &tsapi.Nameserver{}},
}
dnsCfg.Status.Conditions = append(dnsCfg.Status.Conditions, metav1.Condition{
Type: string(tsapi.NameserverReady),
Status: metav1.ConditionTrue,
})
egressSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "lock-service",
Namespace: "default",
Annotations: map[string]string{
AnnotationTailnetTargetFQDN: "lock-service.example.ts.net",
},
},
Spec: corev1.ServiceSpec{
Type: corev1.ServiceTypeExternalName,
ExternalName: "unused",
},
}
proxyGroupEgressSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "ts-proxygroup-egress-abcd1",
Namespace: "tailscale",
Labels: map[string]string{
kubetypes.LabelManaged: "true",
LabelParentName: "lock-service",
LabelParentNamespace: "default",
LabelParentType: "svc",
labelProxyGroup: "test-proxy-group",
labelSvcType: typeEgress,
},
},
}
f := fake.NewClientBuilder().
WithInterceptorFuncs(funcs).
WithScheme(tsapi.GlobalScheme).
WithObjects(dnsCfg, proxyGroupEgressSvc, egressSvc).
WithStatusSubresource(dnsCfg).
Build()
dnsRR := &dnsRecordsReconciler{
Client: f,
tsNamespace: "tailscale",
logger: zl.Sugar(),
}
namespacedName := types.NamespacedName{
Namespace: proxyGroupEgressSvc.GetNamespace(),
Name: proxyGroupEgressSvc.GetName(),
}
res, err := dnsRR.Reconcile(t.Context(), reconcile.Request{
NamespacedName: namespacedName,
})
if err != nil {
t.Errorf("expected requeueAfter in result, got error: %s", err)
}
if res.RequeueAfter == 0 {
t.Errorf("exptected requeueAfter in result to be > 0, got %d", res.RequeueAfter)
}
}
func TestDNSRecordsReconcilerDualStack(t *testing.T) {
// Test dual-stack (IPv4 and IPv6) scenarios
zl, err := zap.NewDevelopment()
+28 -51
View File
@@ -91,10 +91,9 @@ func (er *egressEpsReconciler) Reconcile(ctx context.Context, req reconcile.Requ
lg.Debugf("No egress config found, likely because ProxyGroup has not been created")
return res, nil
}
cfg, ok := cfgs[tailnetSvc]
if !ok {
lg.Warnf("configuration for tailnet service %q not found", tailnetSvc)
lg.Infof("[unexpected] configuration for tailnet service %s not found", tailnetSvc)
return res, nil
}
@@ -106,19 +105,16 @@ func (er *egressEpsReconciler) Reconcile(ctx context.Context, req reconcile.Requ
}
newEndpoints := make([]discoveryv1.Endpoint, 0)
for _, pod := range podList.Items {
ready, err := er.podIsReadyToRouteTraffic(ctx, pod, &cfg, tailnetSvc, eps.AddressType, lg)
ready, err := er.podIsReadyToRouteTraffic(ctx, pod, &cfg, tailnetSvc, lg)
if err != nil {
return res, fmt.Errorf("error verifying if Pod is ready to route traffic: %w", err)
}
if !ready {
continue // maybe next time
}
podIP, err := podIPForFamily(&pod, eps.AddressType)
podIP, err := podIPv4(&pod) // we currently only support IPv4
if err != nil {
return res, fmt.Errorf("error determining Pod IP for %s EndpointSlice: %w", eps.AddressType, err)
}
if podIP == "" {
continue // Pod doesn't have an IP for this address family
return res, fmt.Errorf("error determining IPv4 address for Pod: %w", err)
}
newEndpoints = append(newEndpoints, discoveryv1.Endpoint{
Hostname: (*string)(&pod.UID),
@@ -134,25 +130,21 @@ func (er *egressEpsReconciler) Reconcile(ctx context.Context, req reconcile.Requ
// run a cleanup for deleted Pods etc.
eps.Endpoints = newEndpoints
if !reflect.DeepEqual(eps, oldEps) {
lg.Info("Updating EndpointSlice to ensure traffic is routed to ready proxy Pods")
if err = er.Update(ctx, eps); err != nil {
lg.Infof("Updating EndpointSlice to ensure traffic is routed to ready proxy Pods")
if err := er.Update(ctx, eps); err != nil {
return res, fmt.Errorf("error updating EndpointSlice: %w", err)
}
}
return res, nil
}
func podIPForFamily(pod *corev1.Pod, addrType discoveryv1.AddressType) (string, error) {
func podIPv4(pod *corev1.Pod) (string, error) {
for _, ip := range pod.Status.PodIPs {
parsed, err := netip.ParseAddr(ip.IP)
if err != nil {
return "", fmt.Errorf("error parsing IP address %s: %w", ip, err)
}
switch {
case addrType == discoveryv1.AddressTypeIPv4 && parsed.Is4():
return parsed.String(), nil
case addrType == discoveryv1.AddressTypeIPv6 && parsed.Is6():
if parsed.Is4() {
return parsed.String(), nil
}
}
@@ -162,76 +154,61 @@ func podIPForFamily(pod *corev1.Pod, addrType discoveryv1.AddressType) (string,
// podIsReadyToRouteTraffic returns true if it appears that the proxy Pod has configured firewall rules to be able to
// route traffic to the given tailnet service. It retrieves the proxy's state Secret and compares the tailnet service
// status written there to the desired service configuration.
func (er *egressEpsReconciler) podIsReadyToRouteTraffic(ctx context.Context, pod corev1.Pod, cfg *egressservices.Config, tailnetSvcName string, addrType discoveryv1.AddressType, lg *zap.SugaredLogger) (bool, error) {
func (er *egressEpsReconciler) podIsReadyToRouteTraffic(ctx context.Context, pod corev1.Pod, cfg *egressservices.Config, tailnetSvcName string, lg *zap.SugaredLogger) (bool, error) {
lg = lg.With("proxy_pod", pod.Name)
lg.Debug("checking whether proxy is ready to route to egress service")
lg.Debugf("checking whether proxy is ready to route to egress service")
if !pod.DeletionTimestamp.IsZero() {
lg.Debug("proxy Pod is being deleted, ignore")
lg.Debugf("proxy Pod is being deleted, ignore")
return false, nil
}
podIP, err := podIPForFamily(&pod, addrType)
switch {
case err != nil:
podIP, err := podIPv4(&pod)
if err != nil {
return false, fmt.Errorf("error determining Pod IP address: %v", err)
case podIP == "":
lg.Debugf("Pod does not have an address for family %s", addrType)
}
if podIP == "" {
lg.Infof("[unexpected] Pod does not have an IPv4 address, and IPv6 is not currently supported")
return false, nil
}
stateS := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: pod.Name,
Namespace: pod.Namespace,
},
}
err = er.Get(ctx, client.ObjectKeyFromObject(stateS), stateS)
switch {
case apierrors.IsNotFound(err):
lg.Debug("proxy does not yet have a state Secret, waiting...")
if apierrors.IsNotFound(err) {
lg.Debugf("proxy does not have a state Secret, waiting...")
return false, nil
case err != nil:
return false, fmt.Errorf("error retrieving state Secret: %w", err)
}
if err != nil {
return false, fmt.Errorf("error getting state Secret: %w", err)
}
svcStatusBS := stateS.Data[egressservices.KeyEgressServices]
if len(svcStatusBS) == 0 {
lg.Debug("proxy's state Secret does not contain egress services status, waiting...")
lg.Debugf("proxy's state Secret does not contain egress services status, waiting...")
return false, nil
}
svcStatus := &egressservices.Status{}
if err = json.Unmarshal(svcStatusBS, svcStatus); err != nil {
if err := json.Unmarshal(svcStatusBS, svcStatus); err != nil {
return false, fmt.Errorf("error unmarshalling egress service status: %w", err)
}
var statusIP string
switch addrType {
case discoveryv1.AddressTypeIPv4:
statusIP = svcStatus.PodIPv4
case discoveryv1.AddressTypeIPv6:
statusIP = svcStatus.PodIPv6
}
if !strings.EqualFold(podIP, statusIP) {
lg.Infof("proxy's egress service status is for Pod IP %q, current proxy's Pod IP %q, waiting for the proxy to reconfigure...", statusIP, podIP)
if !strings.EqualFold(podIP, svcStatus.PodIPv4) {
lg.Infof("proxy's egress service status is for Pod IP %s, current proxy's Pod IP %s, waiting for the proxy to reconfigure...", svcStatus.PodIPv4, podIP)
return false, nil
}
st, ok := svcStatus.Services[tailnetSvcName]
st, ok := (*svcStatus).Services[tailnetSvcName]
if !ok {
lg.Infof("proxy's state Secret does not have egress service status, waiting...")
return false, nil
}
if !reflect.DeepEqual(cfg.TailnetTarget, st.TailnetTarget) {
lg.Infof("proxy has configured egress service for tailnet target %q, current target is %q, waiting for proxy to reconfigure...", st.TailnetTarget, cfg.TailnetTarget)
lg.Infof("proxy has configured egress service for tailnet target %v, current target is %v, waiting for proxy to reconfigure...", st.TailnetTarget, cfg.TailnetTarget)
return false, nil
}
if !reflect.DeepEqual(cfg.Ports, st.Ports) {
lg.Debugf("proxy has configured egress service for ports %#+v, wants ports %#+v, waiting for proxy to reconfigure", st.Ports, cfg.Ports)
return false, nil
}
lg.Debug("proxy is ready to route traffic to egress service")
lg.Debugf("proxy is ready to route traffic to egress service")
return true, nil
}
+5 -117
View File
@@ -98,7 +98,7 @@ func TestTailscaleEgressEndpointSlices(t *testing.T) {
t.Run("pods_are_ready_to_route_traffic", func(t *testing.T) {
pod, stateS := podAndSecretForProxyGroup("foo")
stBs := serviceStatusForPodIPs(t, svc, pod.Status.PodIPs[0].IP, "", port)
stBs := serviceStatusForPodIP(t, svc, pod.Status.PodIPs[0].IP, port)
mustUpdate(t, fc, "operator-ns", stateS.Name, func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
@@ -115,8 +115,8 @@ func TestTailscaleEgressEndpointSlices(t *testing.T) {
expectEqual(t, fc, eps)
})
t.Run("status_does_not_match_pod_ip", func(t *testing.T) {
_, stateS := podAndSecretForProxyGroup("foo") // replica Pod has IP 10.0.0.1
stBs := serviceStatusForPodIPs(t, svc, "10.0.0.2", "", port) // status is for a Pod with IP 10.0.0.2
_, stateS := podAndSecretForProxyGroup("foo") // replica Pod has IP 10.0.0.1
stBs := serviceStatusForPodIP(t, svc, "10.0.0.2", port) // status is for a Pod with IP 10.0.0.2
mustUpdate(t, fc, "operator-ns", stateS.Name, func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
@@ -124,117 +124,6 @@ func TestTailscaleEgressEndpointSlices(t *testing.T) {
eps.Endpoints = []discoveryv1.Endpoint{}
expectEqual(t, fc, eps)
})
// Dual-stack.
epsV6 := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: "foo-ipv6",
Namespace: "operator-ns",
Labels: map[string]string{
LabelParentName: "test",
LabelParentNamespace: "default",
labelSvcType: typeEgress,
labelProxyGroup: "foo",
},
},
AddressType: discoveryv1.AddressTypeIPv6,
}
mustCreate(t, fc, epsV6)
t.Run("dual_stack_pod_ready_to_route", func(t *testing.T) {
mustDeleteAll(t, fc, &corev1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "foo-0", Namespace: "operator-ns"}})
dualPod := &corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "foo-0",
Namespace: "operator-ns",
Labels: pgLabels("foo", nil),
UID: "foo",
},
Status: corev1.PodStatus{
PodIPs: []corev1.PodIP{{IP: "10.0.0.1"}, {IP: "fd00::1"}},
},
}
mustCreate(t, fc, dualPod)
stBs := serviceStatusForPodIPs(t, svc, "10.0.0.1", "fd00::1", port)
mustUpdate(t, fc, "operator-ns", "foo-0", func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
expectReconciled(t, er, "operator-ns", "foo")
eps.Endpoints = []discoveryv1.Endpoint{{
Addresses: []string{"10.0.0.1"},
Hostname: new("foo"),
Conditions: discoveryv1.EndpointConditions{
Serving: new(true),
Ready: new(true),
Terminating: new(false),
},
}}
expectEqual(t, fc, eps)
expectReconciled(t, er, "operator-ns", "foo-ipv6")
epsV6.Endpoints = []discoveryv1.Endpoint{{
Addresses: []string{"fd00::1"},
Hostname: new("foo"),
Conditions: discoveryv1.EndpointConditions{
Serving: new(true),
Ready: new(true),
Terminating: new(false),
},
}}
expectEqual(t, fc, epsV6)
})
// IPv6-only.
t.Run("ipv4_only_pod_skipped_for_ipv6_slice", func(t *testing.T) {
mustDeleteAll(t, fc, &corev1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "foo-0", Namespace: "operator-ns"}})
ipv4Pod, _ := podAndSecretForProxyGroup("foo")
mustCreate(t, fc, ipv4Pod)
stBs := serviceStatusForPodIPs(t, svc, "10.0.0.1", "", port)
mustUpdate(t, fc, "operator-ns", "foo-0", func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
expectReconciled(t, er, "operator-ns", "foo-ipv6")
// IPv4-only pod should not appear in the IPv6 EndpointSlice.
epsV6.Endpoints = []discoveryv1.Endpoint{}
expectEqual(t, fc, epsV6)
})
ipv6Pod := &corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "foo-0",
Namespace: "operator-ns",
Labels: pgLabels("foo", nil),
UID: "foo",
},
Status: corev1.PodStatus{
PodIPs: []corev1.PodIP{{IP: "fd00::1"}},
},
}
t.Run("ipv6_status_does_not_match_pod_ip", func(t *testing.T) {
mustDeleteAll(t, fc, &corev1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "foo-0", Namespace: "operator-ns"}})
mustCreate(t, fc, ipv6Pod)
stBs := serviceStatusForPodIPs(t, svc, "", "fd00::99", port)
mustUpdate(t, fc, "operator-ns", "foo-0", func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
expectReconciled(t, er, "operator-ns", "foo-ipv6")
epsV6.Endpoints = []discoveryv1.Endpoint{}
expectEqual(t, fc, epsV6)
})
t.Run("ipv6_pod_ready_to_route", func(t *testing.T) {
stBs := serviceStatusForPodIPs(t, svc, "", ipv6Pod.Status.PodIPs[0].IP, port)
mustUpdate(t, fc, "operator-ns", "foo-0", func(s *corev1.Secret) {
mak.Set(&s.Data, egressservices.KeyEgressServices, stBs)
})
expectReconciled(t, er, "operator-ns", "foo-ipv6")
epsV6.Endpoints = append(epsV6.Endpoints, discoveryv1.Endpoint{
Addresses: []string{"fd00::1"},
Hostname: new("foo"),
Conditions: discoveryv1.EndpointConditions{
Serving: new(true),
Ready: new(true),
Terminating: new(false),
},
})
expectEqual(t, fc, epsV6)
})
}
func configMapForSvc(t *testing.T, svc *corev1.Service, p uint16) *corev1.ConfigMap {
@@ -268,7 +157,7 @@ func configMapForSvc(t *testing.T, svc *corev1.Service, p uint16) *corev1.Config
return cm
}
func serviceStatusForPodIPs(t *testing.T, svc *corev1.Service, ipv4, ipv6 string, p uint16) []byte {
func serviceStatusForPodIP(t *testing.T, svc *corev1.Service, ip string, p uint16) []byte {
t.Helper()
ports := make(map[egressservices.PortMap]struct{})
for _, port := range svc.Spec.Ports {
@@ -283,8 +172,7 @@ func serviceStatusForPodIPs(t *testing.T, svc *corev1.Service, ipv4, ipv6 string
}
svcName := tailnetSvcName(svc)
st := egressservices.Status{
PodIPv4: ipv4,
PodIPv6: ipv6,
PodIPv4: ip,
Services: map[string]*egressservices.ServiceStatus{svcName: &svcSt},
}
bs, err := json.Marshal(st)
+7 -23
View File
@@ -10,7 +10,6 @@ import (
"errors"
"fmt"
"net/http"
"net/netip"
"slices"
"strings"
"sync/atomic"
@@ -24,7 +23,6 @@ import (
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/kube/kubetypes"
"tailscale.com/tstime"
@@ -89,9 +87,8 @@ func (er *egressPodsReconciler) Reconcile(ctx context.Context, req reconcile.Req
lg.Debugf("Pod is being deleted, do nothing")
return res, nil
}
if pod.Labels[LabelParentType] != proxyTypeProxyGroup {
lg.Warn("reconciler called for a Pod that is not a ProxyGroup Pod")
lg.Infof("[unexpected] reconciler called for a Pod that is not a ProxyGroup Pod")
return res, nil
}
@@ -109,12 +106,10 @@ func (er *egressPodsReconciler) Reconcile(ctx context.Context, req reconcile.Req
if err := er.Get(ctx, types.NamespacedName{Name: proxyGroupName}, pg); err != nil {
return res, fmt.Errorf("error getting ProxyGroup %q: %w", proxyGroupName, err)
}
if pg.Spec.Type != typeEgress {
lg.Warnf("reconciler called for %q ProxyGroup Pod", pg.Spec.Type)
lg.Infof("[unexpected] reconciler called for %q ProxyGroup Pod", pg.Spec.Type)
return res, nil
}
// Get all ClusterIP Services for all egress targets exposed to cluster via this ProxyGroup.
lbls := map[string]string{
kubetypes.LabelManaged: "true",
@@ -228,23 +223,12 @@ func (er *egressPodsReconciler) lookupPodRouteViaSvc(ctx context.Context, pod *c
lg.Debugf("Pod does not have health check enabled, unable to verify if it is currently routable via Service")
return cannotVerify, nil
}
// Use the Pod's primary IP (PodIPs[0]) to identify this Pod in the health check
// response. The primary IP family is determined by the cluster's IP family configuration.
// Note: we do not control which IP family the request uses, so on a dual-stack
// cluster either IPv4 or IPv6 could be used. In either case, a matching IP header
// comfirms the request reached this Pod.
if len(pod.Status.PodIPs) == 0 || pod.Status.PodIPs[0].IP == "" {
return podNotReady, nil
}
wantsIP := pod.Status.PodIPs[0].IP
parsed, err := netip.ParseAddr(wantsIP)
wantsIP, err := podIPv4(pod)
if err != nil {
return -1, fmt.Errorf("error parsing Pod IP %q: %w", wantsIP, err)
return -1, fmt.Errorf("error determining Pod's IP address: %w", err)
}
header := kubetypes.PodIPv4Header
if parsed.Is6() {
header = kubetypes.PodIPv6Header
if wantsIP == "" {
return podNotReady, nil
}
ctx, cancel := context.WithTimeout(ctx, time.Second*3)
@@ -262,7 +246,7 @@ func (er *egressPodsReconciler) lookupPodRouteViaSvc(ctx context.Context, pod *c
return unreachable, nil
}
defer resp.Body.Close()
gotIP := resp.Header.Get(header)
gotIP := resp.Header.Get(kubetypes.PodIPv4Header)
if gotIP == "" {
lg.Debugf("Health check does not return Pod's IP header, unable to verify if Pod is currently routable via Service")
return cannotVerify, nil
+1 -51
View File
@@ -420,44 +420,6 @@ func TestEgressPodReadiness(t *testing.T) {
expectEqual(t, fc, pod)
mustDeleteAll(t, fc, pod, svc, svc2, svc3)
})
t.Run("ipv6_only_pod_already_routed_to", func(t *testing.T) {
pod := podTemplate.DeepCopy()
pod.Status.PodIPs = []corev1.PodIP{{IP: "fd00::2"}}
svc, hep := newSvc("svc", 9002)
mustCreateAll(t, fc, svc, pod)
resp := readyRespsV6("fd00::2", 1)
httpCl := fakeHTTPClient{
t: t,
state: map[string][]fakeResponse{hep: resp},
}
rec.httpClient = &httpCl
expectReconciled(t, rec, "operator-ns", pod.Name)
podSetReady(pod, cl)
expectEqual(t, fc, pod)
mustDeleteAll(t, fc, pod, svc)
})
t.Run("dual_stack_pod", func(t *testing.T) {
pod := podTemplate.DeepCopy()
pod.Status.PodIPs = []corev1.PodIP{{IP: "10.0.0.2"}, {IP: "fd00::2"}}
svc, hep := newSvc("svc", 9002)
mustCreateAll(t, fc, svc, pod)
// Dual-stack pod: the reconciler uses PodIPs[0] (the primary IP),
// which in this case is IPv4.
resp := readyResps("10.0.0.2", 1)
httpCl := fakeHTTPClient{
t: t,
state: map[string][]fakeResponse{hep: resp},
}
rec.httpClient = &httpCl
expectReconciled(t, rec, "operator-ns", pod.Name)
podSetReady(pod, cl)
expectEqual(t, fc, pod)
mustDeleteAll(t, fc, pod, svc)
})
}
func readyResps(ip string, num int) (resps []fakeResponse) {
@@ -467,13 +429,6 @@ func readyResps(ip string, num int) (resps []fakeResponse) {
return resps
}
func readyRespsV6(ip string, num int) (resps []fakeResponse) {
for range num {
resps = append(resps, fakeResponse{statusCode: 200, podIP: ip, header: kubetypes.PodIPv6Header})
}
return resps
}
func unreadyResps(ip string, num int) (resps []fakeResponse) {
for range num {
resps = append(resps, fakeResponse{statusCode: 503, podIP: ip})
@@ -558,11 +513,7 @@ func (f *fakeHTTPClient) Do(req *http.Request) (*http.Response, error) {
Header: make(http.Header),
Body: io.NopCloser(bytes.NewReader([]byte{})),
}
h := kubetypes.PodIPv4Header
if resp.header != "" {
h = resp.header
}
r.Header.Add(h, resp.podIP)
r.Header.Add(kubetypes.PodIPv4Header, resp.podIP)
return &r, nil
}
@@ -570,5 +521,4 @@ type fakeResponse struct {
err error
statusCode int
podIP string // for the Pod IP header
header string // header key to use; defaults to PodIPv4Header
}
+21 -67
View File
@@ -9,7 +9,6 @@ import (
"context"
"errors"
"fmt"
"slices"
"strings"
"go.uber.org/zap"
@@ -21,11 +20,9 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
tsoperator "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstime"
"tailscale.com/util/set"
)
const (
@@ -74,57 +71,19 @@ func (esrr *egressSvcsReadinessReconciler) Reconcile(ctx context.Context, req re
}()
crl := egressSvcChildResourceLabels(svc)
epsList := &discoveryv1.EndpointSliceList{}
if err = esrr.List(ctx, epsList, client.InNamespace(esrr.tsNamespace), client.MatchingLabels(crl)); err != nil {
err = fmt.Errorf("error listing EndpointSlices: %w", err)
eps, err := getSingleObject[discoveryv1.EndpointSlice](ctx, esrr.Client, esrr.tsNamespace, crl)
if err != nil {
err = fmt.Errorf("error getting EndpointSlice: %w", err)
reason = reasonReadinessCheckFailed
msg = err.Error()
return res, err
}
if len(epsList.Items) == 0 {
lg.Infof("EndpointSlices for Service do not yet exist, waiting...")
if eps == nil {
lg.Infof("EndpointSlice for Service does not yet exist, waiting...")
reason, msg = reasonClusterResourcesNotReady, reasonClusterResourcesNotReady
st = metav1.ConditionFalse
return res, nil
}
// If an EndpointSlice for an expected family is missing, we mark the Service as NotReady.
//
// Setting the NotReady condition here is also used for best-effort recovery. The
// egress-svcs-reconciler does not watch EndpointSlices, so a deleted EndpointSlice is only
// recreated when this status change re-triggers a Service reconcile.
//
// TODO(beckypauley): refactor so EndpointSlice recovery is not dependent on Service status.
clusterIPSvc, err := getSingleObject[corev1.Service](ctx, esrr.Client, esrr.tsNamespace, crl)
if err != nil {
err = fmt.Errorf("error retrieving ClusterIP Service: %w", err)
reason = reasonReadinessCheckFailed
msg = err.Error()
return res, err
}
if clusterIPSvc == nil {
lg.Infof("ClusterIP Service for egress Service does not yet exist, waiting...")
reason, msg = reasonClusterResourcesNotReady, reasonClusterResourcesNotReady
st = metav1.ConditionFalse
return res, nil
}
gotAddrTypes := make(set.Set[discoveryv1.AddressType], len(epsList.Items))
for _, eps := range epsList.Items {
gotAddrTypes.Add(eps.AddressType)
}
wantAddrTypes, err := addrTypesForClusterIPSvc(clusterIPSvc)
if err != nil {
reason = reasonReadinessCheckFailed
msg = err.Error()
return res, err
}
for _, wantAddrType := range wantAddrTypes {
if !gotAddrTypes.Contains(wantAddrType) {
lg.Infof("EndpointSlice for %s is missing, waiting...", wantAddrType)
reason, msg = reasonClusterResourcesNotReady, reasonClusterResourcesNotReady
st = metav1.ConditionFalse
return res, nil
}
}
pg := &tsapi.ProxyGroup{
ObjectMeta: metav1.ObjectMeta{
Name: svc.Annotations[AnnotationProxyGroup],
@@ -159,7 +118,6 @@ func (esrr *egressSvcsReadinessReconciler) Reconcile(ctx context.Context, req re
}
podLabels := pgLabels(pg.Name, nil)
var readyReplicas int32
nextReplica:
for i := range replicas {
podLabels[appsv1.PodIndexLabel] = fmt.Sprintf("%d", i)
pod, err := getSingleObject[corev1.Pod](ctx, esrr.Client, esrr.tsNamespace, podLabels)
@@ -169,24 +127,24 @@ nextReplica:
msg = err.Error()
return res, err
}
if pod == nil {
lg.Warnf("ProxyGroup is ready, but replica %d was not found", i)
lg.Warnf("[unexpected] ProxyGroup is ready, but replica %d was not found", i)
reason, msg = reasonClusterResourcesNotReady, reasonClusterResourcesNotReady
return res, nil
}
lg.Debugf("looking at Pod with IPs %v", pod.Status.PodIPs)
for _, eps := range epsList.Items {
lg.Debugf("looking at %s EndpointSlice %s", eps.AddressType, eps.Name)
if !slices.ContainsFunc(eps.Endpoints, func(ep discoveryv1.Endpoint) bool {
return endpointReadyForPod(&ep, pod, eps.AddressType, lg)
}) {
continue nextReplica
ready := false
for _, ep := range eps.Endpoints {
lg.Debugf("looking at endpoint with addresses %v", ep.Addresses)
if endpointReadyForPod(&ep, pod, lg) {
lg.Debugf("endpoint is ready for Pod")
ready = true
break
}
}
lg.Debugf("endpoint is ready for Pod")
readyReplicas++
if ready {
readyReplicas++
}
}
msg = fmt.Sprintf(msgReadyToRouteTemplate, readyReplicas, replicas)
if readyReplicas == 0 {
@@ -203,18 +161,14 @@ nextReplica:
return res, nil
}
// endpointReadyForPod returns true if the endpoint is for the Pod's address (for the given address family)
// and is ready to serve traffic. Endpoint must not be nil.
func endpointReadyForPod(ep *discoveryv1.Endpoint, pod *corev1.Pod, addrType discoveryv1.AddressType, lg *zap.SugaredLogger) bool {
podIP, err := podIPForFamily(pod, addrType)
// endpointReadyForPod returns true if the endpoint is for the Pod's IPv4 address and is ready to serve traffic.
// Endpoint must not be nil.
func endpointReadyForPod(ep *discoveryv1.Endpoint, pod *corev1.Pod, lg *zap.SugaredLogger) bool {
podIP, err := podIPv4(pod)
if err != nil {
lg.Warnf("error retrieving Pod's %s address: %v", addrType, err)
lg.Warnf("[unexpected] error retrieving Pod's IPv4 address: %v", err)
return false
}
if podIP == "" {
return false
}
// Currently we only ever set a single address on and Endpoint and nothing else is meant to modify this.
if len(ep.Addresses) != 1 {
return false
@@ -47,14 +47,7 @@ func TestEgressServiceReadiness(t *testing.T) {
},
},
}
fakeClusterIPSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app",
Namespace: "operator-ns",
Labels: egressSvcChildResourceLabels(egressSvc),
},
Spec: corev1.ServiceSpec{ClusterIPs: []string{"10.0.0.1"}},
}
fakeClusterIPSvc := &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: "my-app", Namespace: "operator-ns"}}
labels := egressSvcEpsLabels(egressSvc, fakeClusterIPSvc)
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
@@ -70,7 +63,6 @@ func TestEgressServiceReadiness(t *testing.T) {
},
}
mustCreate(t, fc, egressSvc)
mustCreate(t, fc, fakeClusterIPSvc)
setClusterNotReady(egressSvc, cl, zl.Sugar())
t.Run("endpointslice_does_not_exist", func(t *testing.T) {
expectReconciled(t, rec, "dev", "my-app")
@@ -125,212 +117,6 @@ func TestEgressServiceReadiness(t *testing.T) {
})
}
func TestEgressServiceReadinessDualStack(t *testing.T) {
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithStatusSubresource(&tsapi.ProxyGroup{}).
Build()
zl, _ := zap.NewDevelopment()
cl := tstest.NewClock(tstest.ClockOpts{})
rec := &egressSvcsReadinessReconciler{
tsNamespace: "operator-ns",
Client: fc,
logger: zl.Sugar(),
clock: cl,
}
tailnetFQDN := "my-app.tailnetxyz.ts.net"
egressSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app",
Namespace: "dev",
Annotations: map[string]string{
AnnotationProxyGroup: "dev",
AnnotationTailnetTargetFQDN: tailnetFQDN,
},
},
}
fakeClusterIPSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app",
Namespace: "operator-ns",
Labels: egressSvcChildResourceLabels(egressSvc),
},
Spec: corev1.ServiceSpec{ClusterIPs: []string{"10.0.0.1", "fd00::1"}},
}
labels := egressSvcEpsLabels(egressSvc, fakeClusterIPSvc)
epsV4 := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app-ipv4",
Namespace: "operator-ns",
Labels: labels,
},
AddressType: discoveryv1.AddressTypeIPv4,
}
labelsV6 := egressSvcEpsLabels(egressSvc, fakeClusterIPSvc)
epsV6 := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app-ipv6",
Namespace: "operator-ns",
Labels: labelsV6,
},
AddressType: discoveryv1.AddressTypeIPv6,
}
pg := &tsapi.ProxyGroup{
ObjectMeta: metav1.ObjectMeta{
Name: "dev",
},
Spec: tsapi.ProxyGroupSpec{
Replicas: new(int32(1)),
Type: tsapi.ProxyGroupTypeEgress,
},
}
mustCreate(t, fc, egressSvc)
mustCreate(t, fc, fakeClusterIPSvc)
mustCreate(t, fc, epsV4)
mustCreate(t, fc, epsV6)
mustCreate(t, fc, pg)
setPGReady(pg, cl, zl.Sugar())
mustUpdateStatus(t, fc, pg.Namespace, pg.Name, func(p *tsapi.ProxyGroup) {
p.Status = pg.Status
})
// Create a dual-stack pod.
p := pod(pg, 0)
p.Status.PodIPs = append(p.Status.PodIPs, corev1.PodIP{IP: "fd00::0"})
mustCreate(t, fc, p)
mustUpdateStatus(t, fc, p.Namespace, p.Name, func(existing *corev1.Pod) {
existing.Status.PodIPs = p.Status.PodIPs
})
t.Run("not_ready_missing_from_ipv6_slice", func(t *testing.T) {
setEndpointForReplicaWithIP("10.0.0.0", epsV4)
mustUpdate(t, fc, epsV4.Namespace, epsV4.Name, func(e *discoveryv1.EndpointSlice) {
e.Endpoints = epsV4.Endpoints
})
expectReconciled(t, rec, "dev", "my-app")
setNotReady(egressSvc, cl, zl.Sugar(), pgReplicas(pg))
expectEqual(t, fc, egressSvc)
})
t.Run("ready_in_both_slices", func(t *testing.T) {
setEndpointForReplicaWithIP("fd00::", epsV6)
mustUpdate(t, fc, epsV6.Namespace, epsV6.Name, func(e *discoveryv1.EndpointSlice) {
e.Endpoints = epsV6.Endpoints
})
expectReconciled(t, rec, "dev", "my-app")
setReady(egressSvc, cl, zl.Sugar(), pgReplicas(pg), pgReplicas(pg))
expectEqual(t, fc, egressSvc)
})
t.Run("not_ready_when_ipv6_slice_missing", func(t *testing.T) {
// Delete the IPv6 EndpointSlice while the ClusterIP Service still
// wants an IPv6 family; the Service should report NotReady even though
// the IPv4 EndpointSlice is healthy.
if err := fc.Delete(t.Context(), epsV6); err != nil {
t.Fatalf("error deleting IPv6 EndpointSlice: %v", err)
}
expectReconciled(t, rec, "dev", "my-app")
setClusterNotReady(egressSvc, cl, zl.Sugar())
expectEqual(t, fc, egressSvc)
})
}
func TestEgressServiceReadinessIPv6Only(t *testing.T) {
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithStatusSubresource(&tsapi.ProxyGroup{}).
Build()
zl, _ := zap.NewDevelopment()
cl := tstest.NewClock(tstest.ClockOpts{})
rec := &egressSvcsReadinessReconciler{
tsNamespace: "operator-ns",
Client: fc,
logger: zl.Sugar(),
clock: cl,
}
egressSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app",
Namespace: "dev",
Annotations: map[string]string{
AnnotationProxyGroup: "dev",
AnnotationTailnetTargetFQDN: "my-app.tailnetxyz.ts.net",
},
},
}
fakeClusterIPSvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app",
Namespace: "operator-ns",
Labels: egressSvcChildResourceLabels(egressSvc),
},
Spec: corev1.ServiceSpec{ClusterIPs: []string{"fd00::1"}},
}
labels := egressSvcEpsLabels(egressSvc, fakeClusterIPSvc)
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: "my-app-ipv6",
Namespace: "operator-ns",
Labels: labels,
},
AddressType: discoveryv1.AddressTypeIPv6,
}
pg := &tsapi.ProxyGroup{
ObjectMeta: metav1.ObjectMeta{
Name: "dev",
},
}
mustCreate(t, fc, egressSvc)
mustCreate(t, fc, fakeClusterIPSvc)
mustCreate(t, fc, eps)
mustCreate(t, fc, pg)
setPGReady(pg, cl, zl.Sugar())
mustUpdateStatus(t, fc, pg.Namespace, pg.Name, func(p *tsapi.ProxyGroup) {
p.Status = pg.Status
})
// Create IPv6-only pods.
for i := range pgReplicas(pg) {
p := ipv6OnlyPod(pg, i)
mustCreate(t, fc, p)
mustUpdateStatus(t, fc, p.Namespace, p.Name, func(existing *corev1.Pod) {
existing.Status.PodIPs = p.Status.PodIPs
})
}
t.Run("no_ready_replicas", func(t *testing.T) {
expectReconciled(t, rec, "dev", "my-app")
setNotReady(egressSvc, cl, zl.Sugar(), pgReplicas(pg))
expectEqual(t, fc, egressSvc)
})
t.Run("all_replicas_ready", func(t *testing.T) {
for i := range pgReplicas(pg) {
p := ipv6OnlyPod(pg, i)
setEndpointForReplicaWithIP(p.Status.PodIPs[0].IP, eps)
}
mustUpdate(t, fc, eps.Namespace, eps.Name, func(e *discoveryv1.EndpointSlice) {
e.Endpoints = eps.Endpoints
})
setReady(egressSvc, cl, zl.Sugar(), pgReplicas(pg), pgReplicas(pg))
expectReconciled(t, rec, "dev", "my-app")
expectEqual(t, fc, egressSvc)
})
}
func ipv6OnlyPod(pg *tsapi.ProxyGroup, ordinal int32) *corev1.Pod {
labels := pgLabels(pg.Name, nil)
labels[appsv1.PodIndexLabel] = fmt.Sprintf("%d", ordinal)
ip := fmt.Sprintf("fd00::%d", ordinal+1) // +1 to avoid fd00::0 normalization issues
return &corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-%d", pg.Name, ordinal),
Namespace: "operator-ns",
Labels: labels,
},
Status: corev1.PodStatus{
PodIPs: []corev1.PodIP{{IP: ip}},
},
}
}
func setClusterNotReady(svc *corev1.Service, cl tstime.Clock, lg *zap.SugaredLogger) {
tsoperator.SetServiceCondition(svc, tsapi.EgressSvcReady, metav1.ConditionFalse, reasonClusterResourcesNotReady, reasonClusterResourcesNotReady, cl, lg)
}
@@ -380,14 +166,3 @@ func pod(pg *tsapi.ProxyGroup, ordinal int32) *corev1.Pod {
},
}
}
func setEndpointForReplicaWithIP(ip string, eps *discoveryv1.EndpointSlice) {
eps.Endpoints = append(eps.Endpoints, discoveryv1.Endpoint{
Addresses: []string{ip},
Conditions: discoveryv1.EndpointConditions{
Ready: new(true),
Serving: new(true),
Terminating: new(false),
},
})
}
+23 -59
View File
@@ -12,7 +12,6 @@ import (
"errors"
"fmt"
"math/rand/v2"
"net/netip"
"reflect"
"slices"
"strings"
@@ -203,10 +202,6 @@ func (esr *egressSvcsReconciler) maybeProvision(ctx context.Context, svc *corev1
return nil
}
if err := esr.ensureEndpointSlices(ctx, svc, clusterIPSvc, lg); err != nil {
return err
}
// Update ExternalName Service to point at the ClusterIP Service.
clusterDomain := retrieveClusterDomain(esr.tsNamespace, lg)
clusterIPSvcFQDN := fmt.Sprintf("%s.%s.svc.%s", clusterIPSvc.Name, clusterIPSvc.Namespace, clusterDomain)
@@ -223,60 +218,6 @@ func (esr *egressSvcsReconciler) maybeProvision(ctx context.Context, svc *corev1
return nil
}
// addrTypesForClusterIPSvc returns the EndpointSlice address types (IP families)
// that the given ClusterIP Service supports, derived from its ClusterIPs.
// TODO(beckypauley): this could read Spec.IPFamilies directly instead of parsing
// ClusterIPs to determine the family.
func addrTypesForClusterIPSvc(clusterIPSvc *corev1.Service) ([]discoveryv1.AddressType, error) {
addrTypes := make([]discoveryv1.AddressType, 0, len(clusterIPSvc.Spec.ClusterIPs))
for _, clusterIP := range clusterIPSvc.Spec.ClusterIPs {
ip, err := netip.ParseAddr(clusterIP)
if err != nil {
return nil, fmt.Errorf("error parsing ClusterIP %q: %w", clusterIP, err)
}
addrType := discoveryv1.AddressTypeIPv4
if ip.Is6() {
addrType = discoveryv1.AddressTypeIPv6
}
addrTypes = append(addrTypes, addrType)
}
return addrTypes, nil
}
// ensureEndpointSlices ensures that EndpointSlices exist for the egress service
// for each IP family supported by the cluster, and that their ports are up to
// date.
func (esr *egressSvcsReconciler) ensureEndpointSlices(ctx context.Context, svc, clusterIPSvc *corev1.Service, lg *zap.SugaredLogger) error {
crl := egressSvcEpsLabels(svc, clusterIPSvc)
// Only create EndpointSlices for IP families supported by the cluster.
addrTypes, err := addrTypesForClusterIPSvc(clusterIPSvc)
if err != nil {
return err
}
for _, addrType := range addrTypes {
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-%s", clusterIPSvc.Name, strings.ToLower(string(addrType))),
Namespace: esr.tsNamespace,
Labels: crl,
},
AddressType: addrType,
Ports: epsPortsFromSvc(clusterIPSvc),
}
if _, err := createOrUpdate(ctx, esr.Client, esr.tsNamespace, eps, func(e *discoveryv1.EndpointSlice) {
e.Labels = eps.Labels
e.AddressType = eps.AddressType
e.Ports = eps.Ports
for _, p := range e.Endpoints {
p.Conditions.Ready = nil
}
}); err != nil {
return fmt.Errorf("error ensuring %s EndpointSlice: %w", addrType, err)
}
}
return nil
}
func (esr *egressSvcsReconciler) provision(ctx context.Context, proxyGroupName string, svc, clusterIPSvc *corev1.Service, lg *zap.SugaredLogger) (*corev1.Service, bool, error) {
lg.Infof("updating configuration...")
usedPorts, err := esr.usedPortsForPG(ctx, proxyGroupName)
@@ -375,6 +316,29 @@ func (esr *egressSvcsReconciler) provision(ctx context.Context, proxyGroupName s
}
}
crl := egressSvcEpsLabels(svc, clusterIPSvc)
// TODO(irbekrm): support IPv6, but need to investigate how kube proxy
// sets up Service -> Pod routing when IPv6 is involved.
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-ipv4", clusterIPSvc.Name),
Namespace: esr.tsNamespace,
Labels: crl,
},
AddressType: discoveryv1.AddressTypeIPv4,
Ports: epsPortsFromSvc(clusterIPSvc),
}
if eps, err = createOrUpdate(ctx, esr.Client, esr.tsNamespace, eps, func(e *discoveryv1.EndpointSlice) {
e.Labels = eps.Labels
e.AddressType = eps.AddressType
e.Ports = eps.Ports
for _, p := range e.Endpoints {
p.Conditions.Ready = nil
}
}); err != nil {
return nil, false, fmt.Errorf("error ensuring EndpointSlice: %w", err)
}
cm, cfgs, err := egressSvcsConfigs(ctx, esr.Client, proxyGroupName, esr.tsNamespace)
if err != nil {
return nil, false, fmt.Errorf("error retrieving egress services configuration: %w", err)
+5 -172
View File
@@ -21,7 +21,6 @@ import (
"k8s.io/apimachinery/pkg/util/intstr"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/controller-runtime/pkg/client/interceptor"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/kube/egressservices"
@@ -51,9 +50,6 @@ func TestTailscaleEgressServices(t *testing.T) {
WithScheme(tsapi.GlobalScheme).
WithObjects(pg, cm).
WithStatusSubresource(pg).
WithInterceptorFuncs(interceptor.Funcs{
Create: clusterIPInterceptor("10.96.0.1"),
}).
Build()
zl, err := zap.NewDevelopment()
if err != nil {
@@ -121,23 +117,6 @@ func TestTailscaleEgressServices(t *testing.T) {
validateReadyService(t, fc, esr, svc, clock, zl, cm)
})
t.Run("endpointslice_deletion_recovery", func(t *testing.T) {
name := findGenNameForEgressSvcResources(t, fc, svc)
epsName := fmt.Sprintf("%s-ipv4", name)
// Delete the EndpointSlice and verify it is recreated.
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: epsName,
Namespace: "operator-ns",
},
}
if err := fc.Delete(t.Context(), eps); err != nil {
t.Fatalf("error deleting EndpointSlice: %v", err)
}
expectMissing[discoveryv1.EndpointSlice](t, fc, "operator-ns", epsName)
validateReadyService(t, fc, esr, svc, clock, zl, cm)
})
t.Run("delete_external_name_service", func(t *testing.T) {
name := findGenNameForEgressSvcResources(t, fc, svc)
if err := fc.Delete(context.Background(), svc); err != nil {
@@ -156,10 +135,10 @@ func validateReadyService(t *testing.T, fc client.WithWatch, esr *egressSvcsReco
expectReconciled(t, esr, "default", "test")
// Verify that a ClusterIP Service has been created.
name := findGenNameForEgressSvcResources(t, fc, svc)
expectEqual(t, fc, clusterIPSvc(name, svc), removeTargetPortsFromSvc, removeClusterIPsFromSvc)
expectEqual(t, fc, clusterIPSvc(name, svc), removeTargetPortsFromSvc)
clusterSvc := mustGetClusterIPSvc(t, fc, name)
// Verify that an EndpointSlice has been created.
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc, discoveryv1.AddressTypeIPv4))
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc))
// Verify that ConfigMap contains configuration for the new egress service.
mustHaveConfigForSvc(t, fc, svc, clusterSvc, cm, zl)
r := svcConfiguredReason(svc, true, zl.Sugar())
@@ -245,22 +224,18 @@ func mustGetClusterIPSvc(t *testing.T, cl client.Client, name string) *corev1.Se
return svc
}
func endpointSlice(name string, extNSvc, clusterIPSvc *corev1.Service, addrType discoveryv1.AddressType) *discoveryv1.EndpointSlice {
func endpointSlice(name string, extNSvc, clusterIPSvc *corev1.Service) *discoveryv1.EndpointSlice {
labels := egressSvcChildResourceLabels(extNSvc)
labels[discoveryv1.LabelManagedBy] = "tailscale.com"
labels[discoveryv1.LabelServiceName] = name
suffix := "ipv4"
if addrType == discoveryv1.AddressTypeIPv6 {
suffix = "ipv6"
}
return &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-%s", name, suffix),
Name: fmt.Sprintf("%s-ipv4", name),
Namespace: "operator-ns",
Labels: labels,
},
Ports: portsForEndpointSlice(clusterIPSvc),
AddressType: addrType,
AddressType: discoveryv1.AddressTypeIPv4,
}
}
@@ -320,145 +295,3 @@ func configFromCM(t *testing.T, cm *corev1.ConfigMap, svcName string) *egressser
}
return nil
}
func TestTailscaleEgressServicesDualStack(t *testing.T) {
pg := &tsapi.ProxyGroup{
TypeMeta: metav1.TypeMeta{Kind: "ProxyGroup", APIVersion: "tailscale.com/v1alpha1"},
ObjectMeta: metav1.ObjectMeta{
Name: "foo",
UID: types.UID("1234-UID"),
},
Spec: tsapi.ProxyGroupSpec{
Replicas: pointer.To[int32](3),
Type: tsapi.ProxyGroupTypeEgress,
},
}
cm := &corev1.ConfigMap{
ObjectMeta: metav1.ObjectMeta{
Name: pgEgressCMName("foo"),
Namespace: "operator-ns",
},
}
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithObjects(pg, cm).
WithStatusSubresource(pg).
WithInterceptorFuncs(interceptor.Funcs{
Create: clusterIPInterceptor("10.96.0.1", "fd00::1"),
}).
Build()
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
esr := &egressSvcsReconciler{
Client: fc,
logger: zl.Sugar(),
clock: clock,
tsNamespace: "operator-ns",
}
svc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
UID: types.UID("1234-UID"),
Annotations: map[string]string{
AnnotationTailnetTargetFQDN: "foo.bar.ts.net.",
AnnotationProxyGroup: "foo",
},
},
Spec: corev1.ServiceSpec{
ExternalName: "placeholder",
Type: corev1.ServiceTypeExternalName,
Selector: nil,
Ports: []corev1.ServicePort{
{
Protocol: "TCP",
Port: 80,
},
},
},
}
t.Run("dual_stack_creates_both_endpoint_slices", func(t *testing.T) {
mustCreate(t, fc, svc)
expectReconciled(t, esr, "default", "test")
validateReadyService(t, fc, esr, svc, clock, zl, cm)
// Also verify the IPv6 EndpointSlice was created.
name := findGenNameForEgressSvcResources(t, fc, svc)
clusterSvc := mustGetClusterIPSvc(t, fc, name)
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc, discoveryv1.AddressTypeIPv6))
})
t.Run("dual_stack_endpointslice_deletion_recovery", func(t *testing.T) {
name := findGenNameForEgressSvcResources(t, fc, svc)
// Delete both IPv4 and IPv6 EndpointSlices.
for _, suffix := range []string{"ipv4", "ipv6"} {
epsName := fmt.Sprintf("%s-%s", name, suffix)
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: epsName,
Namespace: "operator-ns",
},
}
if err := fc.Delete(t.Context(), eps); err != nil {
t.Fatalf("error deleting EndpointSlice %s: %v", epsName, err)
}
expectMissing[discoveryv1.EndpointSlice](t, fc, "operator-ns", epsName)
}
// Reconcile should recreate both.
validateReadyService(t, fc, esr, svc, clock, zl, cm)
clusterSvc := mustGetClusterIPSvc(t, fc, name)
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc, discoveryv1.AddressTypeIPv6))
})
t.Run("dual_stack_single_endpointslice_deletion_recovery", func(t *testing.T) {
name := findGenNameForEgressSvcResources(t, fc, svc)
// Delete only the IPv6 EndpointSlice.
epsName := fmt.Sprintf("%s-ipv6", name)
eps := &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: epsName,
Namespace: "operator-ns",
},
}
if err := fc.Delete(t.Context(), eps); err != nil {
t.Fatalf("error deleting EndpointSlice %s: %v", epsName, err)
}
expectMissing[discoveryv1.EndpointSlice](t, fc, "operator-ns", epsName)
// Reconcile should recreate the missing IPv6 EndpointSlice while leaving
// the IPv4 one untouched.
validateReadyService(t, fc, esr, svc, clock, zl, cm)
clusterSvc := mustGetClusterIPSvc(t, fc, name)
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc, discoveryv1.AddressTypeIPv6))
expectEqual(t, fc, endpointSlice(name, svc, clusterSvc, discoveryv1.AddressTypeIPv4))
})
t.Run("delete_dual_stack_service", func(t *testing.T) {
name := findGenNameForEgressSvcResources(t, fc, svc)
if err := fc.Delete(context.Background(), svc); err != nil {
t.Fatalf("error deleting ExternalName Service: %v", err)
}
expectReconciled(t, esr, "default", "test")
expectMissing[corev1.Service](t, fc, "operator-ns", name)
expectMissing[discoveryv1.EndpointSlice](t, fc, "operator-ns", fmt.Sprintf("%s-ipv4", name))
expectMissing[discoveryv1.EndpointSlice](t, fc, "operator-ns", fmt.Sprintf("%s-ipv6", name))
mustNotHaveConfigForSvc(t, fc, svc, cm)
})
}
// clusterIPInterceptor returns an interceptor.Funcs Create function that
// simulates the API server assigning ClusterIPs to ClusterIP Services.
// This is required because the reconciler iterates ClusterIPs to create
// per-family EndpointSlices but the fake client does not assign ClusterIPs.
func clusterIPInterceptor(clusterIPs ...string) func(ctx context.Context, c client.WithWatch, obj client.Object, opts ...client.CreateOption) error {
return func(ctx context.Context, c client.WithWatch, obj client.Object, opts ...client.CreateOption) error {
if svc, ok := obj.(*corev1.Service); ok && svc.Spec.Type == corev1.ServiceTypeClusterIP {
svc.Spec.ClusterIPs = clusterIPs
svc.Spec.ClusterIP = clusterIPs[0]
}
return c.Create(ctx, obj, opts...)
}
}
-4
View File
@@ -28,7 +28,6 @@ const (
proxyGroupCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_proxygroups.yaml"
tailnetCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_tailnets.yaml"
proxyGroupPolicyCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_proxygrouppolicies.yaml"
peerRelayCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_peerrelays.yaml"
helmTemplatesPath = operatorDeploymentFilesPath + "/chart/templates"
connectorCRDHelmTemplatePath = helmTemplatesPath + "/connector.yaml"
proxyClassCRDHelmTemplatePath = helmTemplatesPath + "/proxyclass.yaml"
@@ -37,7 +36,6 @@ const (
proxyGroupCRDHelmTemplatePath = helmTemplatesPath + "/proxygroup.yaml"
tailnetCRDHelmTemplatePath = helmTemplatesPath + "/tailnet.yaml"
proxyGroupPolicyCRDHelmTemplatePath = helmTemplatesPath + "/proxygrouppolicy.yaml"
peerRelayCRDHelmTemplatePath = helmTemplatesPath + "/peerrelay.yaml"
helmConditionalStart = "{{ if .Values.installCRDs -}}\n"
helmConditionalEnd = "{{- end -}}"
@@ -162,7 +160,6 @@ func generate(baseDir string) error {
{proxyGroupCRDPath, proxyGroupCRDHelmTemplatePath},
{tailnetCRDPath, tailnetCRDHelmTemplatePath},
{proxyGroupPolicyCRDPath, proxyGroupPolicyCRDHelmTemplatePath},
{peerRelayCRDPath, peerRelayCRDHelmTemplatePath},
} {
if err := addCRDToHelm(crd.crdPath, crd.templatePath); err != nil {
return fmt.Errorf("error adding %s CRD to Helm templates: %w", crd.crdPath, err)
@@ -181,7 +178,6 @@ func cleanup(baseDir string) error {
proxyGroupCRDHelmTemplatePath,
tailnetCRDHelmTemplatePath,
proxyGroupPolicyCRDHelmTemplatePath,
peerRelayCRDHelmTemplatePath,
} {
if err := os.Remove(filepath.Join(baseDir, path)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("error cleaning up %s: %w", path, err)
+67 -93
View File
@@ -173,14 +173,14 @@ func (r *HAIngressReconciler) maybeProvision(ctx context.Context, hostname strin
logger.Infof("error validating tailscale IngressClass: %v.", err)
return false, nil
}
// We only act on services that are annotated as using a proxy group.
// Get and validate ProxyGroup readiness
pgName := ing.Annotations[AnnotationProxyGroup]
if pgName == "" {
logger.Infof("[unexpected] no ProxyGroup annotation, skipping Tailscale Service provisioning")
return false, nil
}
logger = logger.With("ProxyGroup", pgName)
if !tsoperator.ProxyGroupAvailable(pg) {
logger.Infof("ProxyGroup is not (yet) ready")
return false, nil
@@ -455,10 +455,8 @@ func (r *HAIngressReconciler) maybeCleanupProxyGroup(ctx context.Context, logger
if err := r.List(ctx, ingList); err != nil {
return false, fmt.Errorf("listing Ingresses: %w", err)
}
// Collect orphans first so we are not mutating cfg.Services during
// iteration.
var orphans []tailcfg.ServiceName
serveConfigChanged := false
// For each Tailscale Service in serve config...
for tsSvcName := range cfg.Services {
// ...check if there is currently an Ingress with this hostname
found := false
@@ -471,23 +469,40 @@ func (r *HAIngressReconciler) maybeCleanupProxyGroup(ctx context.Context, logger
}
if !found {
orphans = append(orphans, tsSvcName)
logger.Infof("Tailscale Service %q is not owned by any Ingress, cleaning up", tsSvcName)
tsService, err := tsClient.VIPServices().Get(ctx, tsSvcName.String())
switch {
case tailscale.IsNotFound(err):
return false, nil
case err != nil:
return false, fmt.Errorf("getting Tailscale Service %q: %w", tsSvcName, err)
}
// Delete the Tailscale Service from control if necessary.
svcsChanged, err = r.cleanupTailscaleService(ctx, tsService, logger, tsClient)
if err != nil {
return false, fmt.Errorf("deleting Tailscale Service %q: %w", tsSvcName, err)
}
// Make sure the Tailscale Service is not advertised in tailscaled or serve config.
if err = r.maybeUpdateAdvertiseServicesConfig(ctx, tsSvcName, serviceAdvertisementOff, pg); err != nil {
return false, fmt.Errorf("failed to update tailscaled config services: %w", err)
}
_, ok := cfg.Services[tsSvcName]
if ok {
logger.Infof("Removing Tailscale Service %q from serve config", tsSvcName)
delete(cfg.Services, tsSvcName)
serveConfigChanged = true
}
if err = cleanupCertResources(ctx, r.Client, r.tsNamespace, tsSvcName, pg); err != nil {
return false, fmt.Errorf("failed to clean up cert resources: %w", err)
}
}
}
// 1. Remove all orphans from serve config in a single ConfigMap Update
// so the proxy cancels every cert loop before we start deleting
// VIPServices, and we only pay one fsnotify propagation window.
updated := false
for _, tsSvcName := range orphans {
logger.Infof("Tailscale Service %q is not owned by any Ingress, cleaning up", tsSvcName)
_, ok := cfg.Services[tsSvcName]
if ok {
delete(cfg.Services, tsSvcName)
updated = true
}
}
if updated {
if serveConfigChanged {
cfgBytes, err := json.Marshal(cfg)
if err != nil {
return false, fmt.Errorf("marshaling serve config: %w", err)
@@ -496,37 +511,7 @@ func (r *HAIngressReconciler) maybeCleanupProxyGroup(ctx context.Context, logger
if err := r.Update(ctx, cm); err != nil {
return false, fmt.Errorf("updating serve config: %w", err)
}
logger.Infof("Removed Tailscale Services from serve config: %v", orphans)
}
for _, tsSvcName := range orphans {
// 2. Unadvertise the Tailscale Service in tailscaled config.
if err := r.maybeUpdateAdvertiseServicesConfig(ctx, tsSvcName, serviceAdvertisementOff, pg); err != nil {
return svcsChanged, fmt.Errorf("failed to update tailscaled config services: %w", err)
}
// 3. Delete the Tailscale Service from the control plane.
tsService, err := tsClient.VIPServices().Get(ctx, tsSvcName.String())
switch {
case tailscale.IsNotFound(err):
// Already gone at the control plane; continue with cluster
// cleanup rather than aborting the sweep.
case err != nil:
return svcsChanged, fmt.Errorf("getting Tailscale Service %q: %w", tsSvcName, err)
default:
updated, err := r.cleanupTailscaleService(ctx, tsService, logger, tsClient)
if err != nil {
return svcsChanged, fmt.Errorf("deleting Tailscale Service %q: %w", tsSvcName, err)
}
svcsChanged = svcsChanged || updated
}
// 4. Clean up cluster cert resources.
if err := cleanupCertResources(ctx, r.Client, r.tsNamespace, tsSvcName, pg); err != nil {
return svcsChanged, fmt.Errorf("failed to clean up cert resources: %w", err)
}
}
return svcsChanged, nil
}
@@ -534,10 +519,6 @@ func (r *HAIngressReconciler) maybeCleanupProxyGroup(ctx context.Context, logger
// Ingress is being deleted or is unexposed. The cleanup is safe for a multi-cluster setup- the Tailscale Service is only
// deleted if it does not contain any other owner references. If it does the cleanup only removes the owner reference
// corresponding to this Ingress.
//
// Steps are ordered so the proxy cancels its cert loop (via serve config
// removal) before the VIPService is deleted; otherwise the loop retries
// against a domain the control plane no longer recognises.
func (r *HAIngressReconciler) maybeCleanup(ctx context.Context, hostname string, ing *networkingv1.Ingress, logger *zap.SugaredLogger, tsClient tsclient.Client, pg *tsapi.ProxyGroup) (svcChanged bool, err error) {
logger.Debugf("Ensuring any resources for Ingress are cleaned up")
ix := slices.Index(ing.Finalizers, FinalizerNamePG)
@@ -562,53 +543,49 @@ func (r *HAIngressReconciler) maybeCleanup(ctx context.Context, hostname string,
err = r.deleteFinalizer(ctx, ing, logger)
}()
// 1. Check if there is a Tailscale Service associated with this Ingress.
cm, cfg, err := r.proxyGroupServeConfig(ctx, pg.Name)
if err != nil {
return false, fmt.Errorf("error getting ProxyGroup serve config: %w", err)
}
// 1. Remove the Tailscale Service from the proxy's serve config. The proxy
// picks up the change via fsnotify on the mounted ConfigMap and cancels
// its cert loop for this domain before we proceed to delete the
// VIPService.
if cfg != nil && cfg.Services != nil {
if _, ok := cfg.Services[serviceName]; ok {
logger.Infof("Removing TailscaleService %q from serve config for ProxyGroup %q", hostname, pg.Name)
delete(cfg.Services, serviceName)
cfgBytes, err := json.Marshal(cfg)
if err != nil {
return false, fmt.Errorf("error marshaling serve config: %w", err)
}
mak.Set(&cm.BinaryData, serveConfigKey, cfgBytes)
if err := r.Update(ctx, cm); err != nil {
return false, fmt.Errorf("error updating serve config: %w", err)
}
}
// Tailscale Service is always first added to serve config and only then created in the Tailscale API, so if it is not
// found in the serve config, we can assume that there is no Tailscale Service. (If the serve config does not exist at
// all, it is possible that the ProxyGroup has been deleted before cleaning up the Ingress, so carry on with
// cleanup).
if cfg != nil && cfg.Services != nil && cfg.Services[serviceName] == nil {
return false, nil
}
// 2. Unadvertise the Tailscale Service in each proxy's tailscaled config.
// Skipped if the ProxyGroup itself has been deleted (no config Secrets to
// update).
if cfg != nil {
if err = r.maybeUpdateAdvertiseServicesConfig(ctx, serviceName, serviceAdvertisementOff, pg); err != nil {
return false, fmt.Errorf("failed to update tailscaled config services: %w", err)
}
}
// 3. Delete the Tailscale Service from the control plane. By now the
// proxy has stopped serving HTTPS for the domain and stopped trying to
// renew its cert.
// 2. Clean up the Tailscale Service resources.
svcChanged, err = r.cleanupTailscaleService(ctx, svc, logger, tsClient)
if err != nil {
return false, fmt.Errorf("error deleting Tailscale Service: %w", err)
}
// 4. Clean up cluster cert resources (TLS Secret + RBAC).
// 3. Clean up any cluster resources
if err = cleanupCertResources(ctx, r.Client, r.tsNamespace, serviceName, pg); err != nil {
return false, fmt.Errorf("failed to clean up cert resources: %w", err)
}
return svcChanged, nil
if cfg == nil || cfg.Services == nil { // user probably deleted the ProxyGroup
return svcChanged, nil
}
// 4. Unadvertise the Tailscale Service in tailscaled config.
if err = r.maybeUpdateAdvertiseServicesConfig(ctx, serviceName, serviceAdvertisementOff, pg); err != nil {
return false, fmt.Errorf("failed to update tailscaled config services: %w", err)
}
// 5. Remove the Tailscale Service from the serve config for the ProxyGroup.
logger.Infof("Removing TailscaleService %q from serve config for ProxyGroup %q", hostname, pg.Name)
delete(cfg.Services, serviceName)
cfgBytes, err := json.Marshal(cfg)
if err != nil {
return false, fmt.Errorf("error marshaling serve config: %w", err)
}
mak.Set(&cm.BinaryData, serveConfigKey, cfgBytes)
return svcChanged, r.Update(ctx, cm)
}
func (r *HAIngressReconciler) deleteFinalizer(ctx context.Context, ing *networkingv1.Ingress, logger *zap.SugaredLogger) error {
@@ -708,10 +685,9 @@ func (r *HAIngressReconciler) validateIngress(ctx context.Context, ing *networki
// It is invalid to have multiple Ingress resources for the same Tailscale Service in one cluster.
ingList := &networkingv1.IngressList{}
if err := r.List(ctx, ingList); err != nil {
errs = append(errs, fmt.Errorf("failed to list ingresses: %w", err))
errs = append(errs, fmt.Errorf("[unexpected] error listing Ingresses: %w", err))
return errors.Join(errs...)
}
for _, i := range ingList.Items {
if r.shouldExpose(&i) && hostnameForIngress(&i) == hostname && i.UID != ing.UID {
errs = append(errs, fmt.Errorf("found duplicate Ingress %q for hostname %q - multiple Ingresses for the same hostname in the same cluster are not allowed", client.ObjectKeyFromObject(&i), hostname))
@@ -900,16 +876,14 @@ func ownerAnnotations(operatorID string, svc *tailscale.VIPService) (map[string]
}
if svc == nil {
c := ownerAnnotationValue{OwnerRefs: []OwnerRef{ref}}
data, err := json.Marshal(c)
json, err := json.Marshal(c)
if err != nil {
return nil, fmt.Errorf("failed to marshal Tailscale Service's owner annotation contents: %w", err)
return nil, fmt.Errorf("[unexpected] unable to marshal Tailscale Service's owner annotation contents: %w, please report this", err)
}
return map[string]string{
ownerAnnotation: string(data),
ownerAnnotation: string(json),
}, nil
}
o, err := parseOwnerAnnotation(svc)
if err != nil {
return nil, err
+1 -2
View File
@@ -8,7 +8,6 @@ package main
import (
"context"
"fmt"
"net"
"slices"
"strings"
"sync"
@@ -365,7 +364,7 @@ func handlersForIngress(ctx context.Context, ing *networkingv1.Ingress, cl clien
proto = "https+insecure://"
}
mak.Set(&handlers, path, &ipn.HTTPHandler{
Proxy: proto + net.JoinHostPort(svc.Spec.ClusterIP, fmt.Sprint(port)) + path,
Proxy: proto + svc.Spec.ClusterIP + ":" + fmt.Sprint(port) + path,
})
}
addIngressBackend(ing.Spec.DefaultBackend, "/")
-88
View File
@@ -942,91 +942,3 @@ func TestTailscaleIngressWithHTTPRedirect(t *testing.T) {
t.Errorf("incorrect status ports after removing redirect: got %v, want %v", ing.Status.LoadBalancer.Ingress[0].Ports, wantPorts)
}
}
func TestTailscaleIngressIPv6(t *testing.T) {
fc := fake.NewFakeClient(ingressClass())
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
// Create a Service with an IPv6 ClusterIP
ipv6Svc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test-ipv6",
Namespace: "default",
},
Spec: corev1.ServiceSpec{
ClusterIP: "fda9:e575:6e22:2::25",
Ports: []corev1.ServicePort{
{
Port: 2283,
Name: "http",
},
},
},
}
mustCreate(t, fc, ipv6Svc)
// Create an Ingress that routes to the IPv6 service
ing := &networkingv1.Ingress{
TypeMeta: metav1.TypeMeta{Kind: "Ingress", APIVersion: "networking.k8s.io/v1"},
ObjectMeta: metav1.ObjectMeta{
Name: "test-ipv6",
Namespace: "default",
UID: "1234-UID-IPV6",
},
Spec: networkingv1.IngressSpec{
IngressClassName: new("tailscale"),
DefaultBackend: &networkingv1.IngressBackend{
Service: &networkingv1.IngressServiceBackend{
Name: "test-ipv6",
Port: networkingv1.ServiceBackendPort{
Number: 2283,
},
},
},
},
}
mustCreate(t, fc, ing)
ingR := &IngressReconciler{
Client: fc,
ingressClassName: "tailscale",
ssr: &tailscaleSTSReconciler{
Client: fc,
clients: tsclient.NewProvider(&fakeTSClient{}),
tsnetServer: &fakeTSNetServer{certDomains: []string{"test-host"}},
defaultTags: []string{"tag:test"},
operatorNamespace: "operator-ns",
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
}
expectReconciled(t, ingR, "default", "test-ipv6")
// Verify the generated serveConfig has properly bracketed IPv6 address
fullName, _ := findGenName(t, fc, "default", "test-ipv6", "ingress")
opts := configOpts{
replicas: new(int32(1)),
stsName: "tailscale-ipv6-ingress-test-ipv6",
secretName: fullName,
namespace: "default",
parentType: "ingress",
hostname: "default-test-ipv6-ingress",
app: kubetypes.AppIngressResource,
serveConfig: &ipn.ServeConfig{
TCP: map[uint16]*ipn.TCPPortHandler{443: {HTTPS: true}},
Web: map[ipn.HostPort]*ipn.WebServerConfig{
"${TS_CERT_DOMAIN}:443": {Handlers: map[string]*ipn.HTTPHandler{
"/": {Proxy: "http://[fda9:e575:6e22:2::25]:2283/"},
}},
},
},
}
// expectedSecret hardcodes the parent-resource label to "test", so fix it for our IPv6 test
secret := expectedSecret(t, fc, opts)
secret.Labels[LabelParentName] = "test-ipv6"
expectEqual(t, fc, secret)
}
+1 -1
View File
@@ -55,7 +55,7 @@ type ServiceMonitorSpec struct {
JobLabel string `json:"jobLabel"`
// NamespaceSelector selects the namespace of Service(s) that this ServiceMonitor allows to scrape.
// https://github.com/prometheus-operator/prometheus-operator/blob/bb4514e0d5d69f20270e29cfd4ad39b87865ccdf/pkg/apis/monitoring/v1/servicemonitor_types.go#L88
NamespaceSelector ServiceMonitorNamespaceSelector `json:"namespaceSelector"`
NamespaceSelector ServiceMonitorNamespaceSelector `json:"namespaceSelector,omitempty"`
// Selector is the label selector for Service(s) that this ServiceMonitor allows to scrape.
// https://github.com/prometheus-operator/prometheus-operator/blob/bb4514e0d5d69f20270e29cfd4ad39b87865ccdf/pkg/apis/monitoring/v1/servicemonitor_types.go#L85
Selector metav1.LabelSelector `json:"selector"`
+12 -84
View File
@@ -55,7 +55,6 @@ import (
"tailscale.com/ipn/store/kubestore"
apiproxy "tailscale.com/k8s-operator/api-proxy"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/k8s-operator/reconciler/peerrelay"
"tailscale.com/k8s-operator/reconciler/proxygrouppolicy"
"tailscale.com/k8s-operator/reconciler/tailnet"
"tailscale.com/k8s-operator/tsclient"
@@ -96,10 +95,8 @@ func main() {
tsFirewallMode = defaultEnv("PROXY_FIREWALL_MODE", "")
defaultProxyClass = defaultEnv("PROXY_DEFAULT_CLASS", "")
isDefaultLoadBalancer = defaultBool("OPERATOR_DEFAULT_LOAD_BALANCER", false)
sharedACMEAccountKey = defaultBool("OPERATOR_SHARED_ACME_ACCOUNT_KEY", false)
loginServer = strings.TrimSuffix(defaultEnv("OPERATOR_LOGIN_SERVER", ""), "/")
ingressClassName = defaultEnv("OPERATOR_INGRESS_CLASS_NAME", "tailscale")
operatorSAName = defaultEnv("OPERATOR_SERVICE_ACCOUNT_NAME", "operator")
)
var opts []kzap.Opts
@@ -160,7 +157,6 @@ func main() {
tsServer: s,
tsClient: tsc,
tailscaleNamespace: tsNamespace,
operatorSAName: operatorSAName,
restConfig: restConfig,
proxyImage: image,
k8sProxyImage: k8sProxyImage,
@@ -171,7 +167,6 @@ func main() {
defaultProxyClass: defaultProxyClass,
loginServer: loginServer,
ingressClassName: ingressClassName,
sharedACMEAccountKey: sharedACMEAccountKey,
})
}
@@ -354,7 +349,6 @@ func runReconcilers(opts reconcilerOpts) {
tailnetOptions := tailnet.ReconcilerOptions{
Client: mgr.GetClient(),
TailscaleNamespace: opts.tailscaleNamespace,
OperatorSAName: opts.operatorSAName,
Clock: tstime.DefaultClock{},
Logger: opts.log,
Registry: clients,
@@ -372,19 +366,6 @@ func runReconcilers(opts reconcilerOpts) {
startlog.Fatalf("could not register proxygrouppolicy reconciler: %v", err)
}
peerRelayOptions := peerrelay.ReconcilerOptions{
Client: mgr.GetClient(),
TailscaleNamespace: opts.tailscaleNamespace,
ProxyImage: opts.proxyImage,
DefaultTags: strings.Split(opts.proxyTags, ","),
Clients: clients,
Logger: opts.log,
}
if err = peerrelay.NewReconciler(peerRelayOptions).Register(mgr); err != nil {
startlog.Fatalf("could not register peerrelay reconciler: %v", err)
}
svcFilter := handler.EnqueueRequestsFromMapFunc(serviceHandler)
svcChildFilter := handler.EnqueueRequestsFromMapFunc(managedResourceHandlerForType("svc"))
// If a ProxyClass changes, enqueue all Services labeled with that
@@ -754,7 +735,6 @@ func runReconcilers(opts reconcilerOpts) {
proxyClassFilterForProxyGroup := handler.EnqueueRequestsFromMapFunc(proxyClassHandlerForProxyGroup(mgr.GetClient(), startlog))
nodeFilterForProxyGroup := handler.EnqueueRequestsFromMapFunc(nodeHandlerForProxyGroup(mgr.GetClient(), opts.defaultProxyClass, startlog))
saFilterForProxyGroup := handler.EnqueueRequestsFromMapFunc(serviceAccountHandlerForProxyGroup(mgr.GetClient(), startlog))
acmeSecretFilterForProxyGroup := handler.EnqueueRequestsFromMapFunc(acmeAccountsSecretHandlerForProxyGroup(mgr.GetClient(), opts.tailscaleNamespace, opts.sharedACMEAccountKey, startlog))
err = builder.ControllerManagedBy(mgr).
For(&tsapi.ProxyGroup{}).
Named("proxygroup-reconciler").
@@ -763,9 +743,6 @@ func runReconcilers(opts reconcilerOpts) {
Watches(&corev1.ConfigMap{}, ownedByProxyGroupFilter).
Watches(&corev1.ServiceAccount{}, saFilterForProxyGroup).
Watches(&corev1.Secret{}, ownedByProxyGroupFilter).
// The shared ACME accounts Secret has no ProxyGroup owner ref, so
// watch it by name to react to its deletion/recreation.
Watches(&corev1.Secret{}, acmeSecretFilterForProxyGroup).
Watches(&rbacv1.Role{}, ownedByProxyGroupFilter).
Watches(&rbacv1.RoleBinding{}, ownedByProxyGroupFilter).
Watches(&tsapi.ProxyClass{}, proxyClassFilterForProxyGroup).
@@ -786,8 +763,6 @@ func runReconcilers(opts reconcilerOpts) {
loginServer: opts.tsServer.ControlURL,
authKeyRateLimits: make(map[string]*rate.Limiter),
authKeyReissuing: make(map[string]bool),
sharedACMEAccountKey: opts.sharedACMEAccountKey,
})
if err != nil {
startlog.Fatalf("could not create ProxyGroup reconciler: %v", err)
@@ -843,17 +818,6 @@ type reconcilerOpts struct {
// ingressClassName is the name of the ingress class used by reconcilers of Ingress resources. This defaults
// to "tailscale" but can be customised.
ingressClassName string
// sharedACMEAccountKey is the operator-wide default for the
// shared-ACME-account feature. When true, every ProxyGroup uses the
// shared per-tailnet account key unless the ProxyGroup explicitly
// opts out via tailscale.com/share-acme-account=false. When false,
// ProxyGroups opt in individually via
// tailscale.com/share-acme-account=true.
sharedACMEAccountKey bool
// operatorSAName is the name of the ServiceAccount that the operator pod runs as. It is used as the target
// ServiceAccount when minting tokens via the Kubernetes TokenRequest API for Tailnets that authenticate using
// workload identity federation.
operatorSAName string
}
// enqueueAllIngressEgressProxySvcsinNS returns a reconcile request for each
@@ -1245,30 +1209,6 @@ func serviceAccountHandlerForProxyGroup(cl client.Client, logger *zap.SugaredLog
}
}
// acmeAccountsSecretHandlerForProxyGroup enqueues ProxyGroups that use the
// shared ACME account when the shared ACME accounts Secret changes. The
// Secret carries no owner reference, so the owner-based Secret watch never
// matches it.
func acmeAccountsSecretHandlerForProxyGroup(cl client.Client, tsNamespace string, sharedACMEAccountDefault bool, logger *zap.SugaredLogger) handler.MapFunc {
return func(ctx context.Context, o client.Object) []reconcile.Request {
if o.GetName() != kubetypes.ACMEAccountsSecretName || o.GetNamespace() != tsNamespace {
return nil
}
pgList := new(tsapi.ProxyGroupList)
if err := cl.List(ctx, pgList); err != nil {
logger.Debugf("error listing ProxyGroups for shared ACME accounts Secret: %v", err)
return nil
}
reqs := make([]reconcile.Request, 0, len(pgList.Items))
for _, pg := range pgList.Items {
if sharedACMEAccountEnabled(&pg, sharedACMEAccountDefault) {
reqs = append(reqs, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(&pg)})
}
}
return reqs
}
}
// serviceHandlerForIngress returns a handler for Service events for ingress
// reconciler that ensures that if the Service associated with an event is of
// interest to the reconciler, the associated Ingress(es) gets be reconciled.
@@ -1530,10 +1470,9 @@ func HAIngressesFromSecret(cl client.Client, logger *zap.SugaredLogger) handler.
return func(ctx context.Context, o client.Object) []reconcile.Request {
secret, ok := o.(*corev1.Secret)
if !ok {
logger.Warn("Secret handler triggered for an object that is not a Secret")
logger.Infof("[unexpected] Secret handler triggered for an object that is not a Secret")
return nil
}
if isTLSSecret(secret) {
return []reconcile.Request{
{
@@ -1570,16 +1509,15 @@ func HAIngressesFromSecret(cl client.Client, logger *zap.SugaredLogger) handler.
}
}
// HAServicesFromSecret returns a handler that returns reconcile requests for
// HAServiceFromSecret returns a handler that returns reconcile requests for
// all HA Services that should be reconciled in response to a Secret event.
func HAServicesFromSecret(cl client.Client, logger *zap.SugaredLogger) handler.MapFunc {
return func(ctx context.Context, o client.Object) []reconcile.Request {
secret, ok := o.(*corev1.Secret)
if !ok {
logger.Warn("Secret handler triggered for an object that is not a Secret")
logger.Infof("[unexpected] Secret handler triggered for an object that is not a Secret")
return nil
}
if !isPGStateSecret(secret) {
return nil
}
@@ -1611,10 +1549,9 @@ func kubeAPIServerPGsFromSecret(cl client.Client, logger *zap.SugaredLogger) han
return func(ctx context.Context, o client.Object) []reconcile.Request {
secret, ok := o.(*corev1.Secret)
if !ok {
logger.Warn("Secret handler triggered for an object that is not a Secret")
logger.Infof("[unexpected] Secret handler triggered for an object that is not a Secret")
return nil
}
if secret.ObjectMeta.Labels[kubetypes.LabelManaged] != "true" ||
secret.ObjectMeta.Labels[LabelParentType] != "proxygroup" {
return nil
@@ -1650,10 +1587,9 @@ func egressSvcsFromEgressProxyGroup(cl client.Client, logger *zap.SugaredLogger)
return func(ctx context.Context, o client.Object) []reconcile.Request {
pg, ok := o.(*tsapi.ProxyGroup)
if !ok {
logger.Warn("ProxyGroup handler triggered for an object that is not a ProxyGroup")
logger.Infof("[unexpected] ProxyGroup handler triggered for an object that is not a ProxyGroup")
return nil
}
if pg.Spec.Type != tsapi.ProxyGroupTypeEgress {
return nil
}
@@ -1681,10 +1617,9 @@ func ingressesFromIngressProxyGroup(cl client.Client, logger *zap.SugaredLogger)
return func(ctx context.Context, o client.Object) []reconcile.Request {
pg, ok := o.(*tsapi.ProxyGroup)
if !ok {
logger.Warn("ProxyGroup handler triggered for an object that is not a ProxyGroup")
logger.Infof("[unexpected] ProxyGroup handler triggered for an object that is not a ProxyGroup")
return nil
}
if pg.Spec.Type != tsapi.ProxyGroupTypeIngress {
return nil
}
@@ -1712,10 +1647,9 @@ func epsFromExternalNameService(cl client.Client, logger *zap.SugaredLogger, ns
return func(ctx context.Context, o client.Object) []reconcile.Request {
svc, ok := o.(*corev1.Service)
if !ok {
logger.Warn("Service handler triggered for an object that is not a Service")
logger.Infof("[unexpected] Service handler triggered for an object that is not a Service")
return nil
}
if !isEgressSvcForProxyGroup(svc) {
return nil
}
@@ -1742,10 +1676,9 @@ func podsFromEgressEps(cl client.Client, logger *zap.SugaredLogger, ns string) h
return func(ctx context.Context, o client.Object) []reconcile.Request {
eps, ok := o.(*discoveryv1.EndpointSlice)
if !ok {
logger.Warn("EndpointSlice handler triggered for an object that is not a EndpointSlice")
logger.Infof("[unexpected] EndpointSlice handler triggered for an object that is not a EndpointSlice")
return nil
}
if eps.Labels[labelProxyGroup] == "" {
return nil
}
@@ -1782,21 +1715,18 @@ func proxyClassesWithServiceMonitor(cl client.Client, logger *zap.SugaredLogger)
return func(ctx context.Context, o client.Object) []reconcile.Request {
crd, ok := o.(*apiextensionsv1.CustomResourceDefinition)
if !ok {
logger.Warn("ServiceMonitor CRD handler received an object that is not a CustomResourceDefinition")
logger.Debugf("[unexpected] ServiceMonitor CRD handler received an object that is not a CustomResourceDefinition")
return nil
}
if crd.Name != serviceMonitorCRD {
logger.Warnf("ServiceMonitor CRD handler received an unexpected CRD %q", crd.Name)
logger.Debugf("[unexpected] ServiceMonitor CRD handler received an unexpected CRD %q", crd.Name)
return nil
}
pcl := &tsapi.ProxyClassList{}
if err := cl.List(ctx, pcl); err != nil {
logger.Errorf("failed to list ProxyClass resources: %v", err)
logger.Debugf("[unexpected] error listing ProxyClasses: %v", err)
return nil
}
reqs := make([]reconcile.Request, 0)
for _, pc := range pcl.Items {
if pc.Spec.Metrics != nil && pc.Spec.Metrics.ServiceMonitor != nil && pc.Spec.Metrics.ServiceMonitor.Enable {
@@ -1805,7 +1735,6 @@ func proxyClassesWithServiceMonitor(cl client.Client, logger *zap.SugaredLogger)
})
}
}
return reqs
}
}
@@ -1815,10 +1744,9 @@ func crdTransformer(log *zap.SugaredLogger) toolscache.TransformFunc {
return func(o any) (any, error) {
crd, ok := o.(*apiextensionsv1.CustomResourceDefinition)
if !ok {
log.Warn("CRD transformer called for a non-CRD type")
log.Infof("[unexpected] CRD transformer called for a non-CRD type")
return crd, nil
}
crd.Spec = apiextensionsv1.CustomResourceDefinitionSpec{}
return crd, nil
}
+1 -3
View File
@@ -26,7 +26,6 @@ import (
"k8s.io/client-go/tools/record"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
tsoperator "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstime"
@@ -171,11 +170,10 @@ func (pcr *ProxyClassReconciler) validate(ctx context.Context, pc *tsapi.ProxyCl
}
}
}
if pc.Spec.Metrics != nil && pc.Spec.Metrics.ServiceMonitor != nil && pc.Spec.Metrics.ServiceMonitor.Enable {
found, err := hasServiceMonitorCRD(ctx, pcr.Client)
if err != nil {
pcr.logger.Errorf("error retrieving %q CRD: %v", serviceMonitorCRD, err)
pcr.logger.Infof("[unexpected]: error retrieving %q CRD: %v", serviceMonitorCRD, err)
// best effort validation - don't error out here
} else if !found {
msg := fmt.Sprintf("ProxyClass defines that a ServiceMonitor custom resource should be created, but %q CRD was not found", serviceMonitorCRD)
+2 -79
View File
@@ -56,7 +56,6 @@ const (
reasonProxyGroupCreating = "ProxyGroupCreating"
reasonProxyGroupInvalid = "ProxyGroupInvalid"
reasonProxyGroupTailnetUnavailable = "ProxyGroupTailnetUnavailable"
reasonACMEAccountsPendingDeletion = "ACMEAccountsPendingDeletion"
// Copied from k8s.io/apiserver/pkg/registry/generic/registry/store.go@cccad306d649184bf2a0e319ba830c53f65c445c
optimisticLockErrorMsg = "the object has been modified; please apply your changes to the latest version and try again"
@@ -103,14 +102,6 @@ type ProxyGroupReconciler struct {
apiServerProxyGroups set.Slice[types.UID] // for kube-apiserver proxygroups gauge
authKeyRateLimits map[string]*rate.Limiter // per-ProxyGroup rate limiters for auth key re-issuance.
authKeyReissuing map[string]bool
// sharedACMEAccountKey is the operator-wide default for the
// shared-ACME-account feature. When true, every ProxyGroup uses the
// shared per-tailnet account key unless the ProxyGroup explicitly
// opts out via tailscale.com/share-acme-account=false. When false,
// only ProxyGroups annotated with tailscale.com/share-acme-account=true
// use it.
sharedACMEAccountKey bool
}
func (r *ProxyGroupReconciler) logger(name string) *zap.SugaredLogger {
@@ -363,7 +354,7 @@ func (r *ProxyGroupReconciler) maybeProvision(ctx context.Context, tsClient tscl
}
}
role := pgRole(pg, r.tsNamespace, r.sharedACMEAccountEnabledFor(pg))
role := pgRole(pg, r.tsNamespace)
if _, err := createOrUpdate(ctx, r.Client, r.tsNamespace, role, func(r *rbacv1.Role) {
r.ObjectMeta.Labels = role.ObjectMeta.Labels
r.ObjectMeta.Annotations = role.ObjectMeta.Annotations
@@ -403,36 +394,13 @@ func (r *ProxyGroupReconciler) maybeProvision(ctx context.Context, tsClient tscl
}); err != nil {
return r.notReadyErrf(pg, logger, "error provisioning ingress ConfigMap %q: %w", cm.Name, err)
}
// Ensure the shared ACME accounts Secret exists (with finalizer)
// when this ProxyGroup opts into the feature. Proxy pods
// populate its fields on first cert issuance. See #18251.
if r.sharedACMEAccountEnabledFor(pg) {
acmeSecret := pgACMEAccountSecret(r.tsNamespace)
if _, err := createOrUpdate(ctx, r.Client, r.tsNamespace, acmeSecret, func(existing *corev1.Secret) {
if !existing.DeletionTimestamp.IsZero() {
// Deletion can't be undone; warn so the account keys
// get backed up before the finalizer is removed.
msg := fmt.Sprintf("shared ACME accounts Secret %q is marked for deletion but retained by the %q finalizer. Its data remains readable until the finalizer is removed - back it up first to preserve the ACME account keys.", existing.Name, kubetypes.ACMEAccountsFinalizer)
r.recorder.Event(existing, corev1.EventTypeWarning, reasonACMEAccountsPendingDeletion, msg)
logger.Warn(msg)
return
}
existing.Labels = acmeSecret.Labels
if !slices.Contains(existing.Finalizers, kubetypes.ACMEAccountsFinalizer) {
existing.Finalizers = append(existing.Finalizers, kubetypes.ACMEAccountsFinalizer)
}
}); err != nil {
return r.notReadyErrf(pg, logger, "error provisioning shared ACME accounts Secret %q: %w", acmeSecret.Name, err)
}
}
}
defaultImage := r.tsProxyImage
if pg.Spec.Type == tsapi.ProxyGroupTypeKubernetesAPIServer {
defaultImage = r.k8sProxyImage
}
ss, err := pgStatefulSet(pg, r.tsNamespace, defaultImage, r.tsFirewallMode, tailscaledPort, proxyClass, r.sharedACMEAccountEnabledFor(pg))
ss, err := pgStatefulSet(pg, r.tsNamespace, defaultImage, r.tsFirewallMode, tailscaledPort, proxyClass)
if err != nil {
return r.notReadyErrf(pg, logger, "error generating StatefulSet spec: %w", err)
}
@@ -1136,35 +1104,9 @@ func (r *ProxyGroupReconciler) findStaticEndpoints(ctx context.Context, existing
return nil, &FindStaticEndpointErr{msg: fmt.Sprintf("failed to find any `status.addresses` of type %q on nodes using configured Selectors on `spec.staticEndpoints.nodePort.selectors` for ProxyClass %q", corev1.NodeExternalIP, proxyClass.Name)}
}
// If we ended up selecting the same set of addresses already in use, keep
// the existing order. nodes.Items from r.List is not guaranteed to be in
// a stable order across calls, so without this the slice can permute on
// each reconcile, making the marshalled config Secret differ byte-for-byte
// even though nothing has effectively changed. That trips the DeepEqual
// check on the config Secret, which writes the Secret, which fires a
// watch event, which re-enqueues the ProxyGroup, and so on.
if len(currAddrs) > 0 && sameAddrPortSet(endpoints, currAddrs) {
return currAddrs, nil
}
return endpoints, nil
}
// sameAddrPortSet reports whether a and b contain the same AddrPorts,
// ignoring order. Both slices are assumed to be free of duplicates, which
// holds for callers in this package.
func sameAddrPortSet(a, b []netip.AddrPort) bool {
if len(a) != len(b) {
return false
}
for _, x := range a {
if !slices.Contains(b, x) {
return false
}
}
return true
}
func getStaticEndpointAddress(a *corev1.NodeAddress, port uint16) *netip.AddrPort {
addr, err := netip.ParseAddr(a.Address)
if err != nil {
@@ -1379,25 +1321,6 @@ func notReady(reason, msg string) (map[string][]netip.AddrPort, *notReadyReason,
}, nil
}
// sharedACMEAccountEnabledFor reports whether the shared-ACME-account
// feature should be applied to pg. The per-PG
// tailscale.com/share-acme-account annotation wins when set; otherwise
// the operator's OPERATOR_SHARED_ACME_ACCOUNT_KEY setting is the default
// for every ProxyGroup.
func (r *ProxyGroupReconciler) sharedACMEAccountEnabledFor(pg *tsapi.ProxyGroup) bool {
return sharedACMEAccountEnabled(pg, r.sharedACMEAccountKey)
}
// sharedACMEAccountEnabled reports whether pg should use the shared ACME
// account, with the tailscale.com/share-acme-account annotation overriding
// the operator-wide default.
func sharedACMEAccountEnabled(pg *tsapi.ProxyGroup, operatorDefault bool) bool {
if v, ok := pg.Annotations[AnnotationShareACMEAccount]; ok {
return v == "true"
}
return operatorDefault
}
func (r *ProxyGroupReconciler) notReadyErrf(pg *tsapi.ProxyGroup, logger *zap.SugaredLogger, format string, a ...any) (map[string][]netip.AddrPort, *notReadyReason, error) {
err := fmt.Errorf(format, a...)
if strings.Contains(err.Error(), optimisticLockErrorMsg) {
+13 -68
View File
@@ -19,7 +19,6 @@ import (
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/intstr"
"sigs.k8s.io/yaml"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/kube/egressservices"
"tailscale.com/kube/ingressservices"
@@ -64,12 +63,8 @@ func pgNodePortService(pg *tsapi.ProxyGroup, name string, namespace string) *cor
}
// Returns the base StatefulSet definition for a ProxyGroup. A ProxyClass may be
// applied over the top after. shareACMEAccount, when true, injects the env
// vars that route the pod's ACME account key to the shared per-tailnet
// Secret and drops TS_DEBUG_ACME_FORCE_RENEWAL so ARI-based renewals are
// attempted; the caller is responsible for checking the operator setting
// and the PG opt-in annotation.
func pgStatefulSet(pg *tsapi.ProxyGroup, namespace, image, tsFirewallMode string, port *uint16, proxyClass *tsapi.ProxyClass, shareACMEAccount bool) (*appsv1.StatefulSet, error) {
// applied over the top after.
func pgStatefulSet(pg *tsapi.ProxyGroup, namespace, image, tsFirewallMode string, port *uint16, proxyClass *tsapi.ProxyClass) (*appsv1.StatefulSet, error) {
if pg.Spec.Type == tsapi.ProxyGroupTypeKubernetesAPIServer {
return kubeAPIServerStatefulSet(pg, namespace, image, port)
}
@@ -79,10 +74,10 @@ func pgStatefulSet(pg *tsapi.ProxyGroup, namespace, image, tsFirewallMode string
}
// Validate some base assumptions.
if len(ss.Spec.Template.Spec.InitContainers) != 1 {
return nil, fmt.Errorf("base proxy config had %d init containers instead of 1", len(ss.Spec.Template.Spec.InitContainers))
return nil, fmt.Errorf("[unexpected] base proxy config had %d init containers instead of 1", len(ss.Spec.Template.Spec.InitContainers))
}
if len(ss.Spec.Template.Spec.Containers) != 1 {
return nil, fmt.Errorf("base proxy config had %d containers instead of 1", len(ss.Spec.Template.Spec.Containers))
return nil, fmt.Errorf("[unexpected] base proxy config had %d containers instead of 1", len(ss.Spec.Template.Spec.Containers))
}
// StatefulSet config.
@@ -191,6 +186,14 @@ func pgStatefulSet(pg *tsapi.ProxyGroup, namespace, image, tsFirewallMode string
Name: "TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR",
Value: "/etc/tsconfig/$(POD_NAME)",
},
{
// This ensures that cert renewals can succeed if ACME account
// keys have changed since issuance. We cannot guarantee or
// validate that the account key has not changed, see
// https://github.com/tailscale/tailscale/issues/18251
Name: "TS_DEBUG_ACME_FORCE_RENEWAL",
Value: "true",
},
}
if port != nil {
@@ -248,29 +251,6 @@ func pgStatefulSet(pg *tsapi.ProxyGroup, namespace, image, tsFirewallMode string
Value: "true",
},
)
if shareACMEAccount {
envs = append(envs,
corev1.EnvVar{
Name: "TS_ACME_ACCOUNT_SECRET_NAME",
Value: kubetypes.ACMEAccountsSecretName,
},
corev1.EnvVar{
Name: "TS_ACME_ACCOUNT_FIELD",
Value: pgACMEAccountField(pg),
},
)
} else {
// Without a shared account key we cannot guarantee that
// the account key that issued the previous cert is the
// same one attempting renewal. Force plain new-order flow
// so renewals do not silently fail on rejected ARI
// "replaces" claims. See
// https://github.com/tailscale/tailscale/issues/18251.
envs = append(envs, corev1.EnvVar{
Name: "TS_DEBUG_ACME_FORCE_RENEWAL",
Value: "true",
})
}
}
return append(c.Env, envs...)
}()
@@ -426,7 +406,7 @@ func pgServiceAccount(pg *tsapi.ProxyGroup, namespace string) *corev1.ServiceAcc
}
}
func pgRole(pg *tsapi.ProxyGroup, namespace string, shareACMEAccount bool) *rbacv1.Role {
func pgRole(pg *tsapi.ProxyGroup, namespace string) *rbacv1.Role {
return &rbacv1.Role{
ObjectMeta: metav1.ObjectMeta{
Name: pg.Name,
@@ -458,12 +438,6 @@ func pgRole(pg *tsapi.ProxyGroup, namespace string, shareACMEAccount bool) *rbac
pgPodName(pg.Name, i), // State.
)
}
// Ingress ProxyGroup write replicas need access to the
// shared ACME account Secret so they can read the
// per-tailnet account key and write it on first use.
if pg.Spec.Type == tsapi.ProxyGroupTypeIngress && shareACMEAccount {
secrets = append(secrets, kubetypes.ACMEAccountsSecretName)
}
return secrets
}(),
},
@@ -502,35 +476,6 @@ func pgRoleBinding(pg *tsapi.ProxyGroup, namespace string) *rbacv1.RoleBinding {
}
}
// pgACMEAccountField returns the field name used inside the shared
// tailscale-acme-accounts Secret for this ProxyGroup's tailnet. The blank
// tailnet (operator-default credentials) is represented by a reserved
// identifier so it gets a stable, unique field.
func pgACMEAccountField(pg *tsapi.ProxyGroup) string {
tn := pg.Spec.Tailnet
if tn == "" {
tn = kubetypes.ACMEAccountDefaultKey
}
return tn + kubetypes.ACMEAccountKeySuffix
}
// pgACMEAccountSecret returns the shared per-tailnet ACME account key
// Secret, keyed by tailnet inside its data. Not owned by any ProxyGroup
// so it outlives ProxyGroup deletion.
func pgACMEAccountSecret(namespace string) *corev1.Secret {
return &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: kubetypes.ACMEAccountsSecretName,
Namespace: namespace,
Labels: map[string]string{
kubetypes.LabelManaged: "true",
},
// Block accidental deletion.
Finalizers: []string{kubetypes.ACMEAccountsFinalizer},
},
}
}
// kube-apiserver proxies in auth mode use a static ServiceAccount. Everything
// else uses a per-ProxyGroup ServiceAccount.
func pgServiceAccountName(pg *tsapi.ProxyGroup) string {
+11 -192
View File
@@ -811,90 +811,6 @@ func TestProxyGroupWithStaticEndpoints(t *testing.T) {
}
}
// TestFindStaticEndpointsStableOrder verifies that findStaticEndpoints returns
// the existing endpoint order from the config Secret when the resulting set of
// addresses is unchanged. nodes.Items from r.List is not order-stable across
// calls, so without this guarantee the slice can permute on each reconcile,
// triggering a spurious config Secret rewrite which fires a watch event that
// re-enqueues the ProxyGroup, looping forever (issue #19700).
func TestFindStaticEndpointsStableOrder(t *testing.T) {
const (
addrA = "10.0.0.1"
addrB = "10.0.0.2"
port = uint16(30001)
)
pc := &tsapi.ProxyClass{
ObjectMeta: metav1.ObjectMeta{Name: "test-pc"},
Spec: tsapi.ProxyClassSpec{
StaticEndpoints: &tsapi.StaticEndpointsConfig{
NodePort: &tsapi.NodePortConfig{
Ports: []tsapi.PortRange{{Port: port}},
Selector: map[string]string{"foo/bar": "baz"},
},
},
},
}
// Existing config Secret already pins the order [B, A]. The fake client
// lists nodes in name order ([node-a, node-b]) so without the stable-order
// guard findStaticEndpoints would return [A, B], differing from currAddrs
// and causing a spurious Secret rewrite.
currAddrs := []netip.AddrPort{
netip.MustParseAddrPort(addrB + ":30001"),
netip.MustParseAddrPort(addrA + ":30001"),
}
cfg := ipn.ConfigVAlpha{StaticEndpoints: currAddrs}
cfgJSON, err := json.Marshal(cfg)
if err != nil {
t.Fatalf("marshal config: %v", err)
}
existingSecret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{Name: "test-0-config", Namespace: tsNamespace},
Data: map[string][]byte{tsoperator.TailscaledConfigFileName(106): cfgJSON},
}
nodes := []*corev1.Node{
{
ObjectMeta: metav1.ObjectMeta{Name: "node-a", Labels: map[string]string{"foo/bar": "baz"}},
Status: corev1.NodeStatus{Addresses: []corev1.NodeAddress{
{Type: corev1.NodeExternalIP, Address: addrA},
}},
},
{
ObjectMeta: metav1.ObjectMeta{Name: "node-b", Labels: map[string]string{"foo/bar": "baz"}},
Status: corev1.NodeStatus{Addresses: []corev1.NodeAddress{
{Type: corev1.NodeExternalIP, Address: addrB},
}},
},
}
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithObjects(pc, nodes[0], nodes[1], existingSecret).
Build()
zl, _ := zap.NewDevelopment()
r := &ProxyGroupReconciler{Client: fc}
got, err := r.findStaticEndpoints(t.Context(), existingSecret, pc, port, zl.Sugar())
if err != nil {
t.Fatalf("findStaticEndpoints: %v", err)
}
if !slices.Equal(got, currAddrs) {
t.Errorf("findStaticEndpoints returned %v, want %v (order must match currAddrs to avoid reconcile churn)", got, currAddrs)
}
// Repeat to confirm the result is stable across calls.
got2, err := r.findStaticEndpoints(t.Context(), existingSecret, pc, port, zl.Sugar())
if err != nil {
t.Fatalf("findStaticEndpoints (2nd call): %v", err)
}
if !slices.Equal(got, got2) {
t.Errorf("findStaticEndpoints not stable across calls: first=%v second=%v", got, got2)
}
}
func TestProxyGroup(t *testing.T) {
pc := &tsapi.ProxyClass{
ObjectMeta: metav1.ObjectMeta{
@@ -1136,15 +1052,14 @@ func TestProxyGroupTypes(t *testing.T) {
zl, _ := zap.NewDevelopment()
reconciler := &ProxyGroupReconciler{
tsNamespace: tsNamespace,
tsProxyImage: testProxyImage,
Client: fc,
log: zl.Sugar(),
clients: tsclient.NewProvider(&fakeTSClient{}),
clock: tstest.NewClock(tstest.ClockOpts{}),
authKeyRateLimits: make(map[string]*rate.Limiter),
authKeyReissuing: make(map[string]bool),
sharedACMEAccountKey: true,
tsNamespace: tsNamespace,
tsProxyImage: testProxyImage,
Client: fc,
log: zl.Sugar(),
clients: tsclient.NewProvider(&fakeTSClient{}),
clock: tstest.NewClock(tstest.ClockOpts{}),
authKeyRateLimits: make(map[string]*rate.Limiter),
authKeyReissuing: make(map[string]bool),
}
t.Run("egress_type", func(t *testing.T) {
@@ -1264,9 +1179,6 @@ func TestProxyGroupTypes(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{
Name: "test-ingress",
UID: "test-ingress-uid",
Annotations: map[string]string{
AnnotationShareACMEAccount: "true",
},
},
Spec: tsapi.ProxyGroupSpec{
Type: tsapi.ProxyGroupTypeIngress,
@@ -1287,44 +1199,6 @@ func TestProxyGroupTypes(t *testing.T) {
verifyEnvVar(t, sts, "TS_INTERNAL_APP", kubetypes.AppProxyGroupIngress)
verifyEnvVar(t, sts, "TS_SERVE_CONFIG", "/etc/proxies/serve-config.json")
verifyEnvVar(t, sts, "TS_EXPERIMENTAL_CERT_SHARE", "true")
verifyEnvVar(t, sts, "TS_ACME_ACCOUNT_SECRET_NAME", kubetypes.ACMEAccountsSecretName)
// pg.Spec.Tailnet is empty here so the default tailnet field is used.
verifyEnvVar(t, sts, "TS_ACME_ACCOUNT_FIELD", kubetypes.ACMEAccountDefaultKey+kubetypes.ACMEAccountKeySuffix)
// TS_DEBUG_ACME_FORCE_RENEWAL must NOT be set when the PG is
// opted in to the shared ACME account.
for _, e := range sts.Spec.Template.Spec.Containers[0].Env {
if e.Name == "TS_DEBUG_ACME_FORCE_RENEWAL" {
t.Errorf("TS_DEBUG_ACME_FORCE_RENEWAL must not be set on ingress ProxyGroup pods that share an ACME account")
}
}
// Verify the shared ACME accounts Secret exists and has the
// deletion finalizer (see tailscale/tailscale#18251).
acmeSecret := &corev1.Secret{}
if err := fc.Get(t.Context(), client.ObjectKey{Namespace: tsNamespace, Name: kubetypes.ACMEAccountsSecretName}, acmeSecret); err != nil {
t.Errorf("failed to get shared ACME accounts Secret: %v", err)
}
if !slices.Contains(acmeSecret.Finalizers, kubetypes.ACMEAccountsFinalizer) {
t.Errorf("shared ACME accounts Secret missing finalizer %q (got %v)", kubetypes.ACMEAccountsFinalizer, acmeSecret.Finalizers)
}
// Verify the per-ProxyGroup Role grants access to the shared
// ACME accounts Secret (write replicas need it to read/write the
// per-tailnet account key).
role := &rbacv1.Role{}
if err := fc.Get(t.Context(), client.ObjectKey{Namespace: tsNamespace, Name: pg.Name}, role); err != nil {
t.Fatalf("failed to get ProxyGroup Role: %v", err)
}
var sawACMEAccess bool
for _, rule := range role.Rules {
if slices.Contains(rule.Verbs, "patch") && slices.Contains(rule.ResourceNames, kubetypes.ACMEAccountsSecretName) {
sawACMEAccess = true
break
}
}
if !sawACMEAccess {
t.Errorf("ProxyGroup Role does not grant patch access to %q", kubetypes.ACMEAccountsSecretName)
}
// Verify ConfigMap volume mount
cmName := fmt.Sprintf("%s-ingress-config", pg.Name)
@@ -1354,60 +1228,6 @@ func TestProxyGroupTypes(t *testing.T) {
}
})
t.Run("ingress_type_shared_acme_opt_out", func(t *testing.T) {
// The reconciler has sharedACMEAccountKey=true, so ingress PGs
// default to shared. Explicit tailscale.com/share-acme-account=false
// must opt this PG out: no shared-Secret env vars, no Role
// access to the shared Secret, and TS_DEBUG_ACME_FORCE_RENEWAL
// must still be set so ARI "replaces" doesn't silently fail.
pg := &tsapi.ProxyGroup{
ObjectMeta: metav1.ObjectMeta{
Name: "test-ingress-optout",
UID: "test-ingress-optout-uid",
Annotations: map[string]string{
AnnotationShareACMEAccount: "false",
},
},
Spec: tsapi.ProxyGroupSpec{
Type: tsapi.ProxyGroupTypeIngress,
Replicas: new(int32(0)),
},
}
if err := fc.Create(t.Context(), pg); err != nil {
t.Fatal(err)
}
expectReconciled(t, reconciler, "", pg.Name)
sts := &appsv1.StatefulSet{}
if err := fc.Get(t.Context(), client.ObjectKey{Namespace: tsNamespace, Name: pg.Name}, sts); err != nil {
t.Fatalf("failed to get StatefulSet: %v", err)
}
for _, e := range sts.Spec.Template.Spec.Containers[0].Env {
switch e.Name {
case "TS_ACME_ACCOUNT_SECRET_NAME", "TS_ACME_ACCOUNT_FIELD":
t.Errorf("env %q unexpectedly present on opt-out PG", e.Name)
}
}
var sawForceRenewal bool
for _, e := range sts.Spec.Template.Spec.Containers[0].Env {
if e.Name == "TS_DEBUG_ACME_FORCE_RENEWAL" {
sawForceRenewal = true
}
}
if !sawForceRenewal {
t.Errorf("TS_DEBUG_ACME_FORCE_RENEWAL must be set on opt-out PG (avoids silent ARI \"replaces\" rejection)")
}
role := &rbacv1.Role{}
if err := fc.Get(t.Context(), client.ObjectKey{Namespace: tsNamespace, Name: pg.Name}, role); err != nil {
t.Fatalf("failed to get ProxyGroup Role: %v", err)
}
for _, rule := range role.Rules {
if slices.Contains(rule.ResourceNames, kubetypes.ACMEAccountsSecretName) {
t.Errorf("opt-out PG Role must not grant access to %q", kubetypes.ACMEAccountsSecretName)
}
}
})
t.Run("kubernetes_api_server_type", func(t *testing.T) {
pg := &tsapi.ProxyGroup{
ObjectMeta: metav1.ObjectMeta{
@@ -1427,7 +1247,7 @@ func TestProxyGroupTypes(t *testing.T) {
}
expectReconciled(t, reconciler, "", pg.Name)
verifyProxyGroupCounts(t, reconciler, 2, 2, 1)
verifyProxyGroupCounts(t, reconciler, 1, 2, 1)
sts := &appsv1.StatefulSet{}
if err := fc.Get(t.Context(), client.ObjectKey{Namespace: tsNamespace, Name: pg.Name}, sts); err != nil {
@@ -2132,11 +1952,10 @@ func verifyEnvVarNotPresent(t *testing.T, sts *appsv1.StatefulSet, name string)
func expectProxyGroupResources(t *testing.T, fc client.WithWatch, pg *tsapi.ProxyGroup, shouldExist bool, proxyClass *tsapi.ProxyClass) {
t.Helper()
shareACMEAccount := pg.Annotations[AnnotationShareACMEAccount] == "true"
role := pgRole(pg, tsNamespace, shareACMEAccount)
role := pgRole(pg, tsNamespace)
roleBinding := pgRoleBinding(pg, tsNamespace)
serviceAccount := pgServiceAccount(pg, tsNamespace)
statefulSet, err := pgStatefulSet(pg, tsNamespace, testProxyImage, "auto", nil, proxyClass, shareACMEAccount)
statefulSet, err := pgStatefulSet(pg, tsNamespace, testProxyImage, "auto", nil, proxyClass)
if err != nil {
t.Fatal(err)
}

Some files were not shown because too many files have changed in this diff Show More