It is possible that upon a cold-start, we enqueue a partial file
for deletion that is resumed shortly after startup.
If the file transfer happens to last longer than deleteDelay,
we will delete the partial file, which is unfortunate.
The client spent a long time uploading a file,
only for it to be accidentally deleted.
It's a very rare race, but also a frustrating one
if it happens to manifest.
Fix the code to only delete partial files that
do not have an active puts against it.
We also fix a minor bug in ResumeReader
where we read b[:blockSize] instead of b[:cs.Size].
The former is the fixed size of 64KiB,
while the latter is usually 64KiB,
but may be less for the last block.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
This change refactors & moves the bulk of the app connector logic from
./cmd/sniproxy.
A future change will delete the delta in sniproxy and wire it to this type.
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: https://github.com/tailscale/corp/issues/15038
The progress printer was buggy where it would not print correctly
and some of the truncation logic was faulty.
The progress printer now prints something like:
go1.21.3.linux-amd64.tar.gz 21.53MiB 13.83MiB/s 33.88% ETA 00:00:03
where it shows
* the number of bytes transferred so far
* the rate of bytes transferred
(using a 1-second half-life for an exponentially weighted average)
* the progress made as a percentage
* the estimated time
(as calculated from the rate of bytes transferred)
Other changes:
* It now correctly prints the progress for very small files
* It prints at a faster rate (4Hz instead of 1Hz)
* It uses IEC units for byte quantities
(to avoid ambiguities of "kb" being kilobits or kilobytes)
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The `serve` command for TCP has always required the scheme of the target to be specified. However, when it's omitted the error message reported is misleading
```
error: failed to apply TCP serve: invalid TCP target "localhost:5900": missing port in address
```
Since we know the target is TCP, we shouldn't require it to be specified. This aligns with the changes for HTTP proxies in https://github.com/tailscale/tailscale/issues/8489closes#9855
Signed-off-by: Tyler Smalley <tyler@tailscale.com>
The `off` subcommand removes a serve/funnel for the corresponding type and port. Previously, we were not providing this which would result in an error if someone was using something than the default https=443.
closes#9858
Signed-off-by: Tyler Smalley <tyler@tailscale.com>
Initiates http/h2c transport for userspace proxy
backend lazily and at most once.
Updates tailscale/tailscale#9725
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Connects serveTailscaleAuth to the localapi webclient endpoint
and pipes auth URLs and session cookies back to the browser to
redirect users from the frontend.
All behind debug flags for now.
Updates tailscale/corp#14335
Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
Updates userspace proxy to detect plaintext grpc requests
using the preconfigured host prefix and request's content
type header and ensure that these will be proxied over h2c.
Updates tailscale/tailscale#9725
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Don't assume Linux lacks UDP_GRO support if it lacks UDP_SEGMENT
support. This mirrors a similar change in wireguard/wireguard-go@177caa7
for consistency sake. We haven't found any issues here, just being
overly paranoid.
Updates #cleanup
Signed-off-by: Jordan Whited <jordan@tailscale.com>
Previously, the test simply relied on:
defer close()
to cleanup file handles.
This works fine on Unix-based systems,
but not on Windows, which dislikes deleting files
where an open file handle continues to exist.
Fix the test by explicitly closing the file handle
after we are done with the resource.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
It's required as of the recent 5297bd2cff.
Updates #7894
Updates #9394 (sure would be nice)
Change-Id: Id6672408dd8a6c82dba71022c8763e589d789fcd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The LoadFunc loads a value and calls a user-provided function.
The utility of this method is to ensure that the map lock is held
while executing user-provided logic.
This allows us to solve TOCTOU bugs that would be nearly imposible
to the solve without this API.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
We were previously using the netlink API to see if there are chains/rules that
already exist. This works fine in environments where there is either full
nftable support or no support at all. However, we have identified certain
environments which have partial nftable support and the only feasible way of
detecting such an environment is to try to create some of the chains that we
need.
This adds a check to create a dummy postrouting chain which is immediately
deleted. The goal of the check is to ensure we are able to use nftables and
that it won't error out later. This check is only done in the path where we
detected that the system has no preexisting nftable rules.
Updates #5621
Updates #8555
Updates #8762
Signed-off-by: Maisem Ali <maisem@tailscale.com>
These tests were broken at HEAD. CI currently does not run these
as root, will figure out how to do that in a followup.
Updates #5621
Updates #8555
Updates #8762
Signed-off-by: Maisem Ali <maisem@tailscale.com>
WaitGroup.Wait should not be concurrently called WaitGroup.Add.
In other words, we should not start new goroutines after shutodwn is called.
Thus, add a conditional to check that shutdown has not been called
before starting off a new waitAndDelete goroutine.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
While the previous logic was correct, it did not perform well.
Resuming is a dance between the client and server, where
1. the client requests hashes for a partial file,
2. the server then computes those hashes,
3. the client computes hashes locally and compares them.
4. goto 1 while the partial file still has data
While step 2 is running, the client is sitting idle.
While step 3 is running, the server is sitting idle.
By streaming over the block hash immediately after the server
computes it, the client can start checking the hash,
while the server works on the next hash (in a pipelined manner).
This performs dramatically better and also uses less memory
as we don't need to hold a list of hashes, but only need to
handle one hash at a time.
There are two detriments to this approach:
* The HTTP API relies on a JSON stream,
which is not a standard REST-like pattern.
However, since we implement both client and server,
this is fine.
* While the stream is on-going, we hold an open file handle
on the server side while the file is being hashed.
On really slow streams, this could hold a file open forever.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Minor fixes:
* The branch for listing or hashing partial files was inverted.
* The host for peerapi call needs to be real (rather than bogus).
* Handle remote peers that don't support resuming.
* Make resume failures non-fatal (since we can still continue).
This was tested locally, end-to-end system test is future work.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Two bug fixes:
1. when tailscale update is executed as root, `os.UserCacheDir` may
return an error because `$XDG_CACHE_HOME` and `$HOME` are not set;
fallback to `os.TempDir` in those cases
2. on some weird distros (like my EndeavourOS), `/usr/sbin` is just a
symlink to `/usr/bin`; when we resolve `tailscale` binary path from
`tailscaled`, allow `tailscaled` to be in either directory
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
Moves request authorization back into Server.serve to be run at
the start of any request. Fixes Synology unstable track bug where
client would get stuck unable to auth due to not rendering the
Synology redirect auth html on index.html load.
Updates tailscale/corp#14335
Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
File resumption requires keeping partial files around for some time,
but we must still eventually delete them if never resumed.
Thus, we implement asynchronous file deletion, which could
spawn a background goroutine to delete the files.
We also use the same mechanism for deleting files on Windows,
where a file can't be deleted if there is still an open file handle.
We can enqueue those with the asynchronous file deleter as well.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
* cmd/k8s-operator: users can configure operator to set firewall mode for proxies
Users can now pass PROXY_FIREWALL_MODE={nftables,auto,iptables} to operator to make it create ingress/egress proxies with that firewall mode
Also makes sure that if an invalid firewall mode gets configured, the operator will not start provisioning proxy resources, but will instead log an error and write an error event to the related Service.
Updates tailscale/tailscale#9310
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
The change is being kept to a minimum to make a revert easy if necessary. After the release, we will go back for a final cleanup.
updates #8489
Signed-off-by: Tyler Smalley <tyler@tailscale.com>
Previously we were just smushing together args and not trying
to parse the values at all. This resulted in the args to testwrapper
being limited and confusing.
This makes it so that testwrapper parses flags in the exact format as `go test`
command and passes them down in the provided order. It uses tesing.Init to
register flags that `go test` understands, however those are not the only
flags understood by `go test` (such as `-exec`) so we register these separately.
Updates tailscale/corp#14975
Signed-off-by: Maisem Ali <maisem@tailscale.com>
This partially reverts commits a61a9ab087
and 7538f38671 and fully reverts
4823a7e591.
The goal of that commit was to reapply known config whenever the
container restarts. However, that already happens when TS_AUTH_ONCE was
false (the default back then). So we only had to selectively reapply the
config if TS_AUTH_ONCE is true, this does exactly that.
This is a little sad that we have to revert to `tailscale up`, but it
fixes the backwards incompatibility problem.
Updates tailscale/tailscale#9539
Signed-off-by: Maisem Ali <maisem@tailscale.com>
The old code would always retain value `true` if it was set once, even
if you then change `prefs.AutoUpdate.Apply` to `false`.
Instead of using the previous value, use the default (envknob) value to
OR with.
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
This change:
* adds a partial files peerAPI endpoint to get a list of partial files
* adds a helper function to extract the basename of a file
* updates the peer put peerAPI endpoint
* updates the file put localapi endpoint to allow resume functionality
Updates #14772
Signed-off-by: Rhea Ghosh <rhea@tailscale.com>
It seems to be implicated in a CPU consumption bug that's not yet
understood. Disable it until we understand.
Updates tailscale/corp#15261
Change-Id: Ia6d0c310da6464dda79a70fc3c18be0782812d3f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The Sparkle-based update is not quite working yet. Make `NewUpdater`
return `ErrUnsupported` for it to avoid the proliferation of exceptions
up the stack.
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
Debug endpoint for the web client's auth flow to talk back to the
control server. Restricted behind a feature flag on control.
We will either be removing this debug endpoint, or renaming it
before launching the web client updates.
Updates tailscale/corp#14335
Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
This commit changes the PostureChecking syspolicy key to be a
PreferenceOption(user-defined, always, never) instead of Bool.
This aligns better with the defaults implementation on macOS allowing
CLI arguments to be read when user-defined or no defaults is set.
Updates #tailscale/tailscale/5902
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Endeavour OS, at least, uses NetworkManager 1.44.2 and does
not use systemd-resolved behind the scenes at all. If we
find ourselves in that situation, return "direct" not
"systemd-resolved"
Fixes https://github.com/tailscale/tailscale/issues/9687
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
Move the compilation of everything to its own job too, separate
from test execution.
Updates #7894
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
They're slow. Make them their own job that can run in parallel.
Also, only run them in race mode. No need to run them on 386
or non-race amd64.
Updates #7894
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Joe Tsai
Andrew Lytvynov
Sonia Appasamy
Tom DNetto
Tyler Smalley
Irbe Krumina
Jordan Whited
Brad Fitzpatrick
James Tucker
Maisem Ali
License Updater
Tyler Smalley
Rhea Ghosh
Flakes Updater
Kristoffer Dalby
Denton Gentry