I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
Our own WaitGroup wrapper type was a prototype implementation
for the Go method on the standard sync.WaitGroup type.
Now that there is first-class support for Go,
we should migrate over to using it and delete syncs.WaitGroup.
Updates #cleanup
Updates tailscale/tailscale#16330
Change-Id: Ib52b10f9847341ce29b4ca0da927dc9321691235
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Fix CompareAndSwap in the edge-case where
the underlying sync.AtomicValue is uninitialized
(i.e., Store was never called) and
the oldV is the zero value,
then perform CompareAndSwap with any(nil).
Also, document that T must be comparable.
This is a pre-existing restriction.
Fixes#16135
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
ShardedInt provides an int type expvar.Var that supports more efficient
writes at high frequencies (one order of magnigude on an M1 Max, much
more on NUMA systems).
There are two implementations of ShardValue, one that abuses sync.Pool
that will work on current public Go versions, and one that takes a
dependency on a runtime.TailscaleP function exposed in Tailscale's Go
fork. The sync.Pool variant has about 10x the throughput of a single
atomic integer on an M1 Max, and the runtime.TailscaleP variant is about
10x faster than the sync.Pool variant.
Neither variant have perfect distribution, or perfectly always avoid
cross-CPU sharing, as there is no locking or affinity to ensure that the
time of yield is on the same core as the time of core biasing, but in
the average case the distributions are enough to provide substantially
better performance.
See golang/go#18802 for a related upstream proposal.
Updates tailscale/go#109
Updates tailscale/corp#25450
Signed-off-by: James Tucker <james@tailscale.com>
MutexValue is simply a value guarded by a mutex.
For any type that is not pointer-sized,
MutexValue will perform much better than AtomicValue
since it will not incur an allocation boxing the value
into an interface value (which is how Go's atomic.Value
is implemented under-the-hood).
Updates #cleanup
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
One primary purpose of WithLock is to mutate the underlying map.
However, this can lead to a panic if it happens to be nil.
Thus, always allocate a map before passing it to f.
Updates tailscale/corp#11038
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Add Keys, Values, and All to iterate over
all keys, values, and entries, respectively.
Updates #11038
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Some operations cannot be implemented with the prior API:
* Iterating over the map and deleting keys
* Iterating over the map and replacing items
* Calling APIs that expect a native Go map
Add a Map.WithLock method that acquires a write-lock on the map
and then calls a user-provided closure with the underlying Go map.
This allows users to interact with the Map as a regular Go map,
but with the gaurantees that it is concurrent safe.
Updates tailscale/corp#9115
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
If AtomicValue[T] is used with a T that is an interface kind,
then Store may panic if different concret types are ever stored.
Fix this by always wrapping in a concrete type.
Technically, this is only needed if T is an interface kind,
but there is no harm in doing it also for non-interface kinds.
Updates #cleanup
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
To mimic sync.Map.Swap, sync/atomic.Value.Swap, etc.
Updates tailscale/corp#1297
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: If7627da1bce8b552873b21d7e5ebb98904e9a650
Depending on how the preemption will occur, in some scenarios sendc
would have blocked indefinitely even after cancelling the context.
Fixes#10315
Signed-off-by: Uri Gorelik <uri.gore@gmail.com>
The LoadFunc loads a value and calls a user-provided function.
The utility of this method is to ensure that the map lock is held
while executing user-provided logic.
This allows us to solve TOCTOU bugs that would be nearly imposible
to the solve without this API.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
I was reviewing some code that was performing this by hand, and wanted
to suggest using syncs.Map, however as the code in question was
allocating a non-trivial structure this would be necessary to meet the
target.
Updates #cleanup
Signed-off-by: James Tucker <james@tailscale.com>
I need this for a corp change where I have a set as a queue, and make a
different decisison if the set is empty.
Updates tailscale/corp#10344
Signed-off-by: James Tucker <james@tailscale.com>
The test is re-enabled for Windows with a relaxed time assertion.
On Windows the runtime poller currently does not have sufficient
resolution to meet the normal requirements for this test.
See https://github.com/golang/go/issues/44343 for background.
Updates #7876
Signed-off-by: James Tucker <jftucker@gmail.com>
We accidentally switched to ./tool/go in
4022796484 which resulted in no longer
running Windows builds, as this is attempting to run a bash script.
I was unable to quickly fix the various tests that have regressed, so
instead I've added skips referencing #7876, which we need to back and
fix.
Updates #7262
Updates #7876
Signed-off-by: James Tucker <james@tailscale.com>
The addition of WaitGroup.Go in the standard library has been
repeatedly proposed and rejected.
See golang/go#18022, golang/go#23538, and golang/go#39863
In summary, the argument for WaitGroup.Go is that it avoids bugs like:
go func() {
wg.Add(1)
defer wg.Done()
...
}()
where the increment happens after execution (not before)
and also (to a lesser degree) because:
wg.Go(func() {
...
})
is shorter and more readble.
The argument against WaitGroup.Go is that the provided function
takes no arguments and so inputs and outputs must closed over
by the provided function. The most common race bug for goroutines
is that the caller forgot to capture the loop iteration variable,
so this pattern may make it easier to be accidentally racy.
However, that is changing with golang/go#57969.
In my experience the probability of race bugs due to the former
still outwighs the latter, but I have no concrete evidence to prove it.
The existence of errgroup.Group.Go and frequent utility of the method
at least proves that this is a workable pattern and
the possibility of accidental races do not appear to
manifest as frequently as feared.
A reason *not* to use errgroup.Group everywhere is that there are many
situations where it doesn't make sense for the goroutine to return an error
since the error is handled in a different mechanism
(e.g., logged and ignored, formatted and printed to the frontend, etc.).
While you can use errgroup.Group by always returning nil,
the fact that you *can* return nil makes it easy to accidentally return
an error when nothing is checking the return of group.Wait.
This is not a hypothetical problem, but something that has bitten us
in usages that was only using errgroup.Group without intending to use
the error reporting part of it.
Thus, add a (yet another) variant of WaitGroup here that
is identical to sync.WaitGroup, but with an extra method.
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
This updates all source files to use a new standard header for copyright
and license declaration. Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.
This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.
Updates #6865
Signed-off-by: Will Norris <will@tailscale.com>
Map is a concurrent safe map that is a trivial wrapper
over a Go map and a sync.RWMutex.
It is optimized for use-cases where the entries change often,
which is the opposite use-case of what sync.Map is optimized for.
The API is patterned off of sync.Map, but made generic.
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The //go:build syntax was introduced in Go 1.17:
https://go.dev/doc/go1.17#build-lines
gofmt has kept the +build and go:build lines in sync since
then, but enough time has passed. Time to remove them.
Done with:
perl -i -npe 's,^// \+build.*\n,,' $(git grep -l -F '+build')
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The docs say:
Note that while correct uses of TryLock do exist, they are rare,
and use of TryLock is often a sign of a deeper problem in a particular use of mutexes.
Rare code! Or bad code! Who can tell!
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
The Windows CI machine experiences significant random execution delays.
For example, in this code from watchdog.go:
done := make(chan bool)
go func() {
start := time.Now()
mu.Lock()
There was a 500ms delay from initializing done to locking mu.
This test checks that we receive a sufficient number of events quickly enough.
In the face of random 500ms delays, unsurprisingly, the test fails.
There's not much principled we can do about it.
We could build a system of retries or attempt to detect these random delays,
but that game isn't worth the candle.
Skip the test.
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
This allows us to check lock invariants.
It was proposed upstream and rejected in:
https://github.com/golang/go/issues/1366
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
Brad Fitzpatrick
Will Norris
Joe Tsai
James Tucker
Andrew Dunham
Uri Gorelik
Denton Gentry
James Tucker
Maisem Ali
Josh Bleecher Snyder
David Crawshaw
Josh Bleecher Snyder
Dmytro Tananayskiy