This change adds an HTTP handler with a table showing a list of all
probes, their status, and a button that allows triggering a specific
probe.
Updates tailscale/corp#20583
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
- Keep track of the last 10 probe results and successful probe
latencies;
- Add an HTTP handler that triggers a given probe by name and returns it
result as a plaintext HTML page, showing recent probe results as a
baseline
Updates tailscale/corp#20583
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
To make it easier for people to monitor their custom DERP fleet.
Updates tailscale/corp#20654
Change-Id: Id8af22936a6d893cc7b6186d298ab794a2672524
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The goal is to move more network state accessors to netmon.Monitor
where they can be cheaper/cached. But first (this change and others)
we need to make sure the one netmon.Monitor is plumbed everywhere.
Some notable bits:
* tsdial.NewDialer is added, taking a now-required netmon
* because a tsdial.Dialer always has a netmon, anything taking both
a Dialer and a NetMon is now redundant; take only the Dialer and
get the NetMon from that if/when needed.
* netmon.NewStatic is added, primarily for tests
Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299
Change-Id: I877f9cb87618c4eb037cee098241d18da9c01691
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
- Wrap each prober function into a probe class that allows associating
metric labels and custom metrics with a given probe;
- Make sure all existing probe classes set a `class` metric label;
- Move bandwidth probe size from being a metric label to a separate
gauge metric; this will make it possible to use it to calculate
average used bandwidth using a PromQL query;
- Also export transfer time for the bandwidth prober (more accurate than
the total probe time, since it excludes connection establishment
time).
Updates tailscale/corp#17912
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
So that we can e.g. check TLS on multiple ports for a given IP.
Updates tailscale/corp#16367
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I81d840a4c88138de1cbb2032b917741c009470e6
This allows us to check all IP addresses (and address families) for a
given DNS hostname while dynamically discovering new IPs and removing
old ones as they're no longer valid.
Also add a testable example that demonstrates how to use it.
Alternative to #11610
Updates tailscale/corp#16367
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I6d6f39bafc30e6dfcf6708185d09faee2a374599
So we can probe load balancers by their unique DNS name but without
asking for that cert name.
Updates tailscale/corp#13050
Change-Id: Ie4c0a2f951328df64281ed1602b4e624e3c8cf2e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
prober uses NewRegionClient() to connect to a derper using a faked up
single-node region, but NewRegionClient() fails to connect if there is
no non-STUN only client in the region. Set the STUN only flag to false
before we call NewRegionClient() so we can monitor nodes marked as
STUN only in the default derpmap.
Updates #11492
Signed-off-by: Val <valerie@tailscale.com>
On some platforms (notably macOS and iOS) we look up the default
interface to bind outgoing connections to. This is both duplicated
work and results in logspam when the default interface is not available
(i.e. when a phone has no connectivity, we log an error and thus cause
more things that we will try to upload and fail).
Fixed by passing around a netmon.Monitor to more places, so that we can
use its cached interface state.
Fixes#7850
Updates #7621
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
This adds an initial and intentionally minimal configuration for
golang-ci, fixes the issues reported, and adds a GitHub Action to check
new pull requests against this linter configuration.
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I8f38fbc315836a19a094d0d3e986758b9313f163
This provides an example of using native Prometheus metrics with tsweb.
Prober library seems to be the only user of PrometheusVar, so I am
removing support for it in tsweb.
Updates https://github.com/tailscale/corp/issues/10205
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
`prober.DERP` was created in #5988 based on derpprobe. Having used it
instead of derpprobe for a few months, I think we have enough confidence
that it works and can now migrate derpprobe to use the prober framework
and get rid of code duplication.
A few notable changes in behaviour:
- results of STUN probes over IPv4 and IPv6 are now reported separately;
- TLS probing now includes OCSP verification;
- probe names in the output have changed;
- ability to send Slack notification from the prober has been removed.
Instead, the prober now exports metrics in Expvar (/debug/vars) and
Prometheus (/debug/varz) formats.
Fixes https://github.com/tailscale/corp/issues/8497
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
This updates all source files to use a new standard header for copyright
and license declaration. Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.
This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.
Updates #6865
Signed-off-by: Will Norris <will@tailscale.com>
This was tested by running 10000 test iterations and observing no flakes
after this change was made.
Change-Id: Ib036fd03a3a17800132c53c838cc32bfe2961306
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
By default all probes with the same probe interval that have been added
together will run on a synchronized schedule, which results in spiky
resource usage and potential throttling by third-party systems (for
example, OCSP servers used by the TLS probes).
To address this, prober can now run in "spread" mode that will
introduce a random delay before the first run of each probe.
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
This ensures that each DERP server is probed individually (TLS and STUN)
and also manages per-region mesh probing. Actual probing code has been
copied from cmd/derpprobe.
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
sendAlert will trigger the Incident Response system.
sendWarning will post to Slack.
Co-authored-by: M. J. Fromberger <fromberger@tailscale.com>
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
TLS prober now checks validity period for all server certificates
and verifies OCSP revocation status for the leaf cert.
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
prober: add labels to Probe instances.
This allows especially dynamically-registered probes to have a bunch
more dimensions along which they can be sliced in Prometheus.
Signed-off-by: David Anderson <danderson@tailscale.com>
Turns out, it's annoying to have to wait the entire interval
before getting any monitorable data, especially for very long
interval probes like hourly/daily checks.
Signed-off-by: David Anderson <danderson@tailscale.com>
Anton Tolchanov
Brad Fitzpatrick
Andrew Dunham
Thomas Kosiewski
valscale
Mihai Parparita
Andrew Dunham
Will Norris
Denton Gentry
Josh Soref
David Anderson