Removes the avoidFinalRename logic and all associated code as it is no longer required by the Apple clients.
Enables resume logic to be usable for Apple clients.
Fixestailscale/corp#14772
Signed-off-by: Rhea Ghosh <rhea@tailscale.com>
It is possible that upon a cold-start, we enqueue a partial file
for deletion that is resumed shortly after startup.
If the file transfer happens to last longer than deleteDelay,
we will delete the partial file, which is unfortunate.
The client spent a long time uploading a file,
only for it to be accidentally deleted.
It's a very rare race, but also a frustrating one
if it happens to manifest.
Fix the code to only delete partial files that
do not have an active puts against it.
We also fix a minor bug in ResumeReader
where we read b[:blockSize] instead of b[:cs.Size].
The former is the fixed size of 64KiB,
while the latter is usually 64KiB,
but may be less for the last block.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
While the previous logic was correct, it did not perform well.
Resuming is a dance between the client and server, where
1. the client requests hashes for a partial file,
2. the server then computes those hashes,
3. the client computes hashes locally and compares them.
4. goto 1 while the partial file still has data
While step 2 is running, the client is sitting idle.
While step 3 is running, the server is sitting idle.
By streaming over the block hash immediately after the server
computes it, the client can start checking the hash,
while the server works on the next hash (in a pipelined manner).
This performs dramatically better and also uses less memory
as we don't need to hold a list of hashes, but only need to
handle one hash at a time.
There are two detriments to this approach:
* The HTTP API relies on a JSON stream,
which is not a standard REST-like pattern.
However, since we implement both client and server,
this is fine.
* While the stream is on-going, we hold an open file handle
on the server side while the file is being hashed.
On really slow streams, this could hold a file open forever.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
File resumption requires keeping partial files around for some time,
but we must still eventually delete them if never resumed.
Thus, we implement asynchronous file deletion, which could
spawn a background goroutine to delete the files.
We also use the same mechanism for deleting files on Windows,
where a file can't be deleted if there is still an open file handle.
We can enqueue those with the asynchronous file deleter as well.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Perform the same m==nil check in Manager.{PartialFiles,HashPartialFile}
as we do in the other methods.
Fix HashPartialFile is properly handle a length of -1.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
We add the following API:
* type FileChecksums
* type Checksum
* func Manager.PartialFiles
* func Manager.HashPartialFile
* func ResumeReader
The Manager methods provide the ability to query for partial files
and retrieve a list of checksums for a given partial file.
The ResumeReader function is a helper that wraps an io.Reader
to discard content that is identical locally and remotely.
The FileChecksums type represents the checksums of a file
and is safe to JSON marshal and send over the wire.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Rhea Ghosh
Joe Tsai