tstest/natlab/vmtest: add web UI for watching VM tests live

Add an optional --vmtest-web flag that starts an HTTP server showing a
live dashboard for vmtest runs. The dashboard includes:

- Step progress tracker showing all test phases (compile, image prep,
  QEMU launch, agent connect, tailscale up, test-specific steps)
  with status icons and elapsed times
- Per-VM "virtual monitor" cards showing serial console output
  streamed in realtime via WebSocket
- Per-NIC DHCP status (supporting multi-homed VMs like subnet routers)
- Per-node Tailscale status (hidden for non-tailnet VMs)
- Test status badge (Running/Passed/Failed) with live elapsed timer
- Event log showing all lifecycle events chronologically

Architecture follows the existing util/eventbus HTMX+WebSocket pattern:
the server pushes HTML fragments with hx-swap-oob attributes over a
WebSocket, and HTMX routes them to the correct DOM elements by ID.

Key components:
- vmstatus.go: Step tracker (Begin/End lifecycle), EventBus (pub/sub
  with history for late joiners), VMEvent types, NodeStatus tracking
- web.go: HTTP server, WebSocket handler, template loading, ANSI-to-HTML
  conversion via robert-nix/ansihtml, deterministic port selection
- assets/: HTML templates, CSS, HTMX library (copied from eventbus)
- vnet/vnet.go: DHCP event callback on Server for observing DHCP lifecycle
- qemu.go: Console log file tailing with manual offset-based reading

Usage:
  go test ./tstest/natlab/vmtest/ --run-vm-tests --vmtest-web=:0 -v

When using :0, a deterministic port based on the test name is tried
first so re-runs get the same URL, falling back to OS-assigned on
conflict.

Updates #13038

Change-Id: I45281347b3d7af78ed9f4ff896033984f84dcb4d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit is contained in:
Brad Fitzpatrick
2026-04-11 04:33:48 +00:00
committed by Brad Fitzpatrick
parent 0ac09721df
commit b9eac14ef9
14 changed files with 1322 additions and 39 deletions
+73
View File
@@ -5,6 +5,7 @@ package vmtest
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net"
@@ -13,6 +14,7 @@ import (
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
"tailscale.com/tstest/natlab/vnet"
@@ -163,6 +165,11 @@ func (e *Env) launchQEMU(name, logPath string, args []string) error {
}
e.t.Logf("launched QEMU for %s (pid %d), log: %s", name, cmd.Process.Pid, logPath)
e.qemuProcs = append(e.qemuProcs, cmd)
// Start tailing the VM console log for the web UI.
if e.ctx != nil {
go e.tailLogFile(e.ctx, name, logPath)
}
e.t.Cleanup(func() {
cmd.Process.Kill()
cmd.Wait()
@@ -237,3 +244,69 @@ func qmpQueryHostFwd(sockPath string) (int, error) {
}
return strconv.Atoi(m[1])
}
// tailLogFile tails a VM's serial console log file and publishes each line
// as an EventConsoleOutput to the event bus for the web UI.
func (e *Env) tailLogFile(ctx context.Context, name, logPath string) {
// Wait for the file to appear (QEMU may not have created it yet).
var f *os.File
for {
var err error
f, err = os.Open(logPath)
if err == nil {
break
}
select {
case <-ctx.Done():
return
case <-time.After(100 * time.Millisecond):
}
}
defer f.Close()
// Read the file in a loop, tracking our position manually.
// We can't use bufio.Scanner because it caches EOF and won't
// pick up new data appended by QEMU after the first EOF.
var buf []byte
var partial string // incomplete line (no trailing newline yet)
readBuf := make([]byte, 4096)
for {
n, err := f.Read(readBuf)
if n > 0 {
buf = append(buf, readBuf[:n]...)
// Split into complete lines.
for {
idx := bytes.IndexByte(buf, '\n')
if idx < 0 {
break
}
line := partial + string(buf[:idx])
partial = ""
buf = buf[idx+1:]
// Strip trailing \r from serial consoles.
line = strings.TrimRight(line, "\r")
if line == "" {
continue
}
e.appendConsoleLine(name, line)
e.eventBus.Publish(VMEvent{
NodeName: name,
Type: EventConsoleOutput,
Message: line,
})
}
if len(buf) > 0 {
partial = string(buf)
buf = buf[:0]
}
}
if err != nil || n == 0 {
// EOF or error — wait for more data.
select {
case <-ctx.Done():
return
case <-time.After(100 * time.Millisecond):
}
}
}
}