I wrote this function two hours ago, tried to use it in corp, and
immediately found myself confused about the meaning of the arguments.
Time for named parameters!
Updates tailscale/corp#40404
Change-Id: Ic2866e052ccc9f6361b8d529233df54d63abbaa1
Signed-off-by: Alex Chan <alexc@tailscale.com>
We previously identified sync failures that occur when a node falls behind
the remote, and compacts away most its local state. We fixed the underlying
issue in #19444, but that PR only tested the basic scenario where the
local chain is a direct ancestor of the remote chain.
This patch adds an explicit regression test for the case where a node is on
a fork (that is, its HEAD is not part of the remote's active chain).
Although #19444 happened to cover this case, other proposed patches did not
handle the forked state. Adding this test locks in the behaviour and prevents
future sync regressions in this area.
Also, add a shared helper for writing this sort of TKA sync test.
Updates tailscale/corp#40404
Change-Id: I78fdc6beaf71392edf11806197f126db48886f93
Signed-off-by: Alex Chan <alexc@tailscale.com>
Previously there was a mismatch between how nodes store AUMs and what
the control plane would offer during sync:
- Client compaction: Nodes aggressively compact their TKA state -- they
keep the last 24 AUMs, every AUM received in the last two weeks, and
then everything from there back to the last checkpoint. Depending on
when it compacts, a node may only have ~50 AUMs.
- Exponential sampling: To save bandwidth, the control plane would send
a SyncOffer containing ancestors at exponentially increasing intervals
(4th, 16th, 64th, 256th...).
If a node has been offline for too long, the exponential sampling skips
the node's smaller window. When the SyncOffer and local state are disjoint,
the node cannot find a common ancestor to use for synchronisation.
It enters a failure loop where it keeps polling for new TKA state, but
it cannot catch up and has an increasingly-outdated view of the tailnet.
This patch replaces the exponential sampling with a SyncOffer that sends
every checkpoint ancestor of the current HEAD. Since every node is
guaranteed to keep at least one checkpoint after compaction, we're more
likely to have an intersection for the sync process.
This patch also increases `maxSyncHeadIntersectionIter`, which in
practice means the control plane will send every checkpoint in the
current chain. This means all affected nodes will be able to find an
intersection and catch up immediately, without requiring a client update.
It's still possible for a node to be unable to sync, but these edge cases
become less likely with this change. (For example, if a node is 1000+ AUMs
behind, or if it creates a local branch and then compacts away the
intersection with the main chain.)
This patch includes a regression test with synthetic data, and I
verified the fix with customer data.
Updates https://github.com/tailscale/corp/issues/40404
Change-Id: I2174011bb23a2b5972f6d1591aadcc016e3cba35
Signed-off-by: Alex Chan <alexc@tailscale.com>
This avoids putting "DisablementSecrets" in the JSON output from
`tailscale lock log`, which is potentially scary to somebody who doesn't
understand the distinction.
AUMs are stored and transmitted in CBOR-encoded format, which uses an
integer rather than a string key, so this doesn't break already-created
TKAs.
Fixes#19189
Change-Id: I15b4e81a7cef724a450bafcfa0b938da223c78c9
Signed-off-by: Alex Chan <alexc@tailscale.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
Previously a TKA compaction would only run when a node starts, which means a long-running node could use unbounded storage as it accumulates ever-increasing amounts of TKA state. This patch changes TKA so it runs a compaction after every sync.
Updates https://github.com/tailscale/corp/issues/33537
Change-Id: I91df887ea0c5a5b00cb6caced85aeffa2a4b24ee
Signed-off-by: Alex Chan <alexc@tailscale.com>
This updates all source files to use a new standard header for copyright
and license declaration. Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.
This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.
Updates #6865
Signed-off-by: Will Norris <will@tailscale.com>
Updates #5435
Based on the discussion in #5435, we can better support transactional data models
by making the underlying storage layer a parameter (which can be specialized for
the request) rather than a long-lived member of Authority.
Now that Authority is just an instantaneous snapshot of state, we can do things
like provide idempotent methods and make it cloneable, too.
Signed-off-by: Tom DNetto <tom@tailscale.com>
This PR implements the synchronization mechanics for TKA: generating a SyncOffer, processing a SyncOffer to find an intersection,
and computing the set of AUMs that should be transmitted to effect convergence.
This is the final PR implementing core mechanics for TKA.
Signed-off-by: Tom DNetto <tom@tailscale.com>
Alex ChanandAlex Chan
Will NorrisandWill Norris
Tom DNettoandTom