cmd/testwrapper: surface race reports and skip retries when detected
A data race in a package matters more than any individual test
result. Two related problems:
1. Where go test's race detector text ("WARNING: DATA RACE" plus
the goroutine stack traces) lands in JSON output is timing-
dependent: it can be attributed to a test that ends up reporting
PASS (e.g. when the racing goroutines outlive the test that
spawned them and TSan prints during a different test's window).
testwrapper's main loop only flushes the logs of failed tests,
so the race report ends up stuck in a passing test's buffer and
is silently dropped. The race builders just see a bare
"FAIL\nFAIL\tpkg\ttime".
2. If the failing test in such a package happens to be marked flaky,
testwrapper retries it. That is the worst possible response to a
race: the flaky test might not even be the racy code, and a
second run without the racy goroutines could "succeed" while
hiding the real bug.
Address both: scan every output line for the race detector's first-
line marker. Track whether the package observed a race at all, on
the pkgFinished testAttempt. When a race was seen, fold every per-
test log buffer into the package-level logs (so the full report
surfaces from the existing pkg-fail flush path), and drop any
flaky-test retry plans for that package so we fail immediately
instead of running another attempt.
Two new tests:
- TestRaceSuppressesFlakyRetry verifies that a flaky test alongside
a racy test does NOT get retried.
- TestRaceAttributedToPassingTest verifies that a race attributed by
test2json to a passing test still surfaces in the output.
Also add a corpus of captured raw test binary outputs under
cmd/testwrapper/testdata/, with one subdirectory per scenario,
documenting the six representative shapes that go test -race can
emit (race in test body, race in goroutines that outlive a test,
race forced into a later test, race in TestMain post-m.Run, and a
parallel-tests split-attribution case via a "=== NAME" redirect
line). See its README.md for details.
Fixes #19603
Change-Id: Ifbfcd67fb3b1882c4907bd9cb2d68a8b5a91dd54
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit is contained in:
committed by
Brad Fitzpatrick
parent
ee0a03b140
commit
95d874e9b4
+32
@@ -0,0 +1,32 @@
|
||||
=== RUN TestA
|
||||
--- PASS: TestA (0.00s)
|
||||
=== RUN TestSleep
|
||||
==================
|
||||
WARNING: DATA RACE
|
||||
Read at 0x0000007f12a8 by goroutine 9:
|
||||
racesurvey/D_postterminal.TestA.func2()
|
||||
/tmp/racesurvey/D_postterminal/d_test.go:16 +0x8a
|
||||
|
||||
Previous write at 0x0000007f12a8 by goroutine 8:
|
||||
racesurvey/D_postterminal.TestA.func1()
|
||||
/tmp/racesurvey/D_postterminal/d_test.go:15 +0xa4
|
||||
|
||||
Goroutine 9 (running) created at:
|
||||
racesurvey/D_postterminal.TestA()
|
||||
/tmp/racesurvey/D_postterminal/d_test.go:16 +0x44
|
||||
testing.tRunner()
|
||||
/home/ubuntu/sdk/go1.26.3/src/testing/testing.go:2036 +0x21c
|
||||
testing.(*T).Run.gowrap1()
|
||||
/home/ubuntu/sdk/go1.26.3/src/testing/testing.go:2101 +0x38
|
||||
|
||||
Goroutine 8 (finished) created at:
|
||||
racesurvey/D_postterminal.TestA()
|
||||
/tmp/racesurvey/D_postterminal/d_test.go:15 +0x34
|
||||
testing.tRunner()
|
||||
/home/ubuntu/sdk/go1.26.3/src/testing/testing.go:2036 +0x21c
|
||||
testing.(*T).Run.gowrap1()
|
||||
/home/ubuntu/sdk/go1.26.3/src/testing/testing.go:2101 +0x38
|
||||
==================
|
||||
testing.go:1712: race detected during execution of test
|
||||
--- FAIL: TestSleep (0.05s)
|
||||
FAIL
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
// Copyright (c) Tailscale Inc & contributors
|
||||
// SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
package delayed
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var counter int
|
||||
var wg sync.WaitGroup
|
||||
var trigger = make(chan struct{})
|
||||
|
||||
func TestA(t *testing.T) {
|
||||
wg.Add(2)
|
||||
go func() { defer wg.Done(); <-trigger; counter++ }()
|
||||
go func() { defer wg.Done(); <-trigger; counter++ }()
|
||||
}
|
||||
|
||||
func TestSleep(t *testing.T) {
|
||||
close(trigger)
|
||||
// Sleep long enough that the goroutines race during this sleep,
|
||||
// but TestSleep itself doesn't write to counter.
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
wg.Wait()
|
||||
}
|
||||
Reference in New Issue
Block a user