Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
This PR cleans up a bunch of things in ./tstest/integration/vms:
- Bumps version of Ubuntu that's actually run from CI 20.04 -> 24.04
- Removes Ubuntu 18.04 test
- Bumps NixOS 21.05 -> 25.05
Updates#cleanup
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Previously all tests shared their tailscale+tailscaled binaries in
system /tmp directories, which often leaked, and required TestMain to
clean up (which feature/taildrop didn't use).
This makes it use testing.T.TempDir for the binaries, but still only
builds them once and efficiently as possible depending on the OS
copies them around between each test's temp dir.
Updates #15812
Change-Id: I0e2585613f272c3d798a423b8ad1737f8916f527
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I'm not saying it works, but it compiles.
Updates #5794
Change-Id: I2f3c99732e67fe57a05edb25b758d083417f083e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This updates all source files to use a new standard header for copyright
and license declaration. Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.
This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.
Updates #6865
Signed-off-by: Will Norris <will@tailscale.com>
The //go:build syntax was introduced in Go 1.17:
https://go.dev/doc/go1.17#build-lines
gofmt has kept the +build and go:build lines in sync since
then, but enough time has passed. Time to remove them.
Done with:
perl -i -npe 's,^// \+build.*\n,,' $(git grep -l -F '+build')
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Updates #4377
Very smoky/high-level test to ensure that derphttp internals play well
with an agressive (stare + bump) meddler-in-the-middle proxy.
Signed-off-by: Tom DNetto <tom@tailscale.com>
Also fix a somewhat related printing bug in the process where
some paths would print "Success." inconsistently even
when there otherwise was no output (in the EditPrefs path)
Fixes#3830
Updates #3702 (which broke it once while trying to fix it)
Change-Id: Ic51e14526ad75be61ba00084670aa6a98221daa5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The existing code relied on the Go build cache to avoid
needless work when obtaining the tailscale binaries.
For non-obvious reasons, the binaries were getting re-linked
every time, which added 600ms or so on my machine to every test.
Instead, build the binaries exactly once, on demand.
This reduces the time to run 'go test -count=5' from 34s to 10s
on my machine.
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
The tests build fine on other Unix's, they just can't run there.
But there is already a t.Skip by default, so `go test` ends up
working fine elsewhere and checks the code compiles.
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
This uses a neat little tool to dump the output of DNS queries to
standard out. This is the first end-to-end test of DNS that runs against
actual linux systems. The /etc/resolv.conf test may look superflous,
however this will help for correlating system state if one of the DNS
tests fails.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Fix a few test printing issues when tests fail.
Qemu console output is super useful when something is wrong in the
harness and we cannot even bring up the tests.
Also useful for figuring out where all the time goes in tests.
A little noisy, but not too noisy as long as you're only running one VM
as part of the tests, which is my plan.
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
This moves the distribution definitions into a maintainable hujson file
instead of just existing as constants in `distros.go`. Comments are
maintained from the inline definitions.
This uses jennifer[1] for hygenic source tree creation. This allows us
to generate a unique top-level test for each VM run. This should
hopefully help make the output of `go test` easier to read.
This also separates each test out into its own top-level test so that we
can better track the time that each distro takes. I really wish there
was a way to have the `test_codegen.go` file _always_ run as a part of
the compile process instead of having to rely on people remembering to
run `go generate`, but I am limited by my tools.
This will let us remove the `-distro-regex` flag and use `go test -run`
to pick which distros are run.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Apparently this test was flaking because I critically misunderstood how
the kernel buffers UDP packets for senders. I'm trying to send more UDP
packets and will see if that helps.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This test used to try to run this only once, but this variant of the
test attempts to run `tailscale status` up to 6 times in a loop with
exponential backoff.
This fixes the flakiness found in previous instances of this test.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
My spatial memory functions poorly with large files and the vms_test.go
file recently surpassed the point where it functions adequately. This
patch splits up vms_test.go into more files to make my spatial memory
function like I need it to.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Split from https://github.com/tailscale/tailscale/pull/2376.
This adds IPv6 support to testcontrol so each member of the tailscale
network gets an IPv6 address too.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
To avoid the generated nixos disk images from becoming immune from the
GC, I delete the symlink to the nix store at the end of tests.
`t.Cleanup` runs at the end of a test. I changed this part of the code
to have a separate timer for how long it takes to run NixOS builds, but
I did that by using a subtest. This means that it was creating the NixOS
image, deleting its symlink and then trying to use that symlink to find
the resulting disk image, making the whole thing ineffectual.
This was a mistake. I am reverting this change made in
https://github.com/tailscale/tailscale/pull/2360 to remove this layer of
subtesting.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This tests incoming and outgoing UDP traffic. It would test incoming UDP
traffic however our socks server doesn't seem to allow for connecting to
destinations over UDP. When the socks server gets that support the
incoming test should pass without issue.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This adapts the existing in-process logcatcher from tstest/integration
into a public type and uses it on the side of testcontrol. This also
fixes a bug in the Alpine Linux OpenRC unit that makes every value in
`/etc/default/tailscaled` exported into tailscaled's environment, a-la
systemd [Service].EnviromentFile.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This does a few things:
1. Rewrites the tests so that we get a log of what individual tests
failed at the end of a test run.
2. Adds a test that runs an HTTP server via the tester tailscale node and
then has the VMs connect to that over Tailscale.
3. Dials the VM over Tailscale and ensures it answers SSH requests.
4. Other minor framework refactoring.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Oracle Linux[1] is a CentOS fork. It is not very special. I am adding it
to the integration jungle because I am adding it to pkgs and the website
directions.
[1]: https://www.oracle.com/linux/
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This makes sure `tailscale status` and `tailscale ping` works. It also
switches goexpect to use a batch instead of manually banging out each
line, which makes the tests so much easier to read.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This puts nix build logs on the filesystem so that we can debug them
later. This also disables nixos unstable until
https://github.com/NixOS/nixpkgs/issues/128783 is fixed.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Okay, so, at a high level testing NixOS is a lot different than
other distros due to NixOS' determinism. Normally NixOS wants packages to
be defined in either an overlay, a custom packageOverrides or even
yolo-inline as a part of the system configuration. This is going to have
us take a different approach compared to other distributions. The overall
plan here is as following:
1. make the binaries as normal
2. template in their paths as raw strings to the nixos system module
3. run `nixos-generators -f qcow -o $CACHE_DIR/tailscale/nixos/version -c generated-config.nix`
4. pass that to the steps that make the virtual machine
It doesn't really make sense for us to use a premade virtual machine image
for this as that will make it harder to deterministically create the image.
Nix commands generate a lot of output, so their output is hidden behind the
`-verbose-nix-output` flag.
This unfortunately makes this test suite have a hard dependency on
Nix/NixOS, however the test suite has only ever been run on NixOS (and I
am not sure if it runs on other distros at all), so this probably isn't too
big of an issue.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Previously this test would reach out to the public DERP servers in order
to help machines connect with eachother. This is not ideal given our
plans to run these tests completely disconnected from the internet. This
patch introduces an in-process DERP server running on its own randomly
assigned HTTP port.
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Occasionally the test framework would fail with a timeout due to a
virtual machine not phoning home in time. This seems to be happen
whenever qemu can't bind the VNC or SSH ports for a virtual machine.
This was fixed by taking the following actions:
1. Don't listen on VNC unless the `-use-vnc` flag is passed, this
removes the need to listen on VNC at all in most cases. The option to
use VNC is still left in for debugging virtual machines, but removing
this makes it easier to deal with (VNC uses this odd system of
"displays" that are mapped to ports above 5900, and qemu doesn't
offer a decent way to use a normal port number, so we just disable
VNC by default as a compromise).
2. Use a (hopefully) inactive port for SSH. In an ideal world I'd just
have the VM's SSH port be exposed via a Unix socket, however the QEMU
documentation doesn't really say if you can do this or not. While I
do more research, this stopgap will have to make do.
3. Strictly tie more VM resource lifetimes to the tests themselves.
Previously the disk image layers for virtual machines were only
cleaned up at the end of the test and existed in the parent
test-scoped temporary folder. This can make your tmpfs run out of
space, which is not ideal. This should minimize the use of temporary
storage as much as I know how to.
4. Strictly tie the qemu process lifetime to the lifetime of the test
using testing.T#Cleanup. Previously it used a defer statement to
clean up the qemu process, however if the tests timed out this defer
was not run. This left around an orphaned qemu process that had to be
killed manually. This change ensures that all qemu processes exit
when their relevant tests finish.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Previously we used t.Logf indirectly via package log. This worked, but
it was not ideal for our needs. It could cause the streams of output to
get crossed. This change uses a logger.FuncWriter every place log.Output
was previously used, which will more correctly write log information to
the right test output stream.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Alpine Linux[1] is a minimal Linux distribution built around musl libc.
It boots very quickly, requires very little ram and is as close as you
can get to an ideal citizen for testing Tailscale on musl. Alpine has a
Tailscale package already[2], but this patch also makes it easier for us
to provide an Alpine Linux package off of pkgs in the future.
Alpine only offers Tailscale on the rolling-release edge branch.
[1]: https://alpinelinux.org/
[2]: https://pkgs.alpinelinux.org/packages?name=tailscale&branch=edge
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This makes integration tests pull pristine VM images from Amazon S3 if
they don't exist on disk. If the S3 fetch fails, it will fall back to
grabbing the image from the public internet. The VM images on the public
internet are known to be updated without warning and thusly change their
SHA256 checksum. This is not ideal for a test that we want to be able to
fire and forget, then run reliably for a very long time.
This requires an AWS profile to be configured at the default path. The
S3 bucket is rigged so that the requester pays. The VM images are
currently about 6.9 gigabytes. Please keep this in mind when running
these tests on your machine.
Documentation was added to the integration test folder to aid others in
running these tests on their machine.
Some wording in the logs of the tests was altered.
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
The image downloads can take a significant amount of time for the tests.
This creates a new test that will download every distro image into the
local cache in parallel, optionally matching the distribution regex.
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
I've run into a couple issues where the tests time out while a VM image
is being downloaded, making the cache poisoned for the next run. This
moves the hash checking into its own function and calls it much sooner
in the testing chain. If the hash check fails, the OS is redownloaded.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Most of the time qemu will output nothing when it is running. This is
expected behavior. However when qemu is unable to start due to some
problem, it prints that to either stdout or stderr. Previously this
output wasn't being captured. This patch captures that output to aid in
debugging qemu issues.
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Brad Fitzpatrick
Will Norris
Alex Chan
Irbe Krumina
Tom DNetto
Josh Bleecher Snyder
David Crawshaw
Christine Dodrill