drive/driveimpl: handle Unicode normalization mismatches in filenames

Files whose names contain characters with Unicode decompositions (such
as umlauts or voiced kana) could not be opened or written over
Taildrive.

Background: keyboards and IMEs emit NFC (precomposed) characters on
every platform, so filenames on Linux (ext4 etc) and Windows (NTFS)
disks are usually NFC bytes. NFD (decomposed) names mostly come from
Apple software: HFS+ forced a variant of NFD on write, and Apple's
frameworks still decompose paths via fileSystemRepresentation. APFS
preserves whatever bytes it is given but does normalization-insensitive
lookups (it stores a hash of the normalized name), so canonically
equivalent names find the same file. ext4 and NTFS lookups, by
contrast, are byte-exact.

On the wire, the macOS WebDAV client sends paths in NFD form (they
pass through the decomposing file system representation, and unlike
Apple's NFS client there is no "nfc" mount option). Windows and Linux
WebDAV clients pass names through as the application provided them,
typically NFC. WebDAV itself mandates no normalization, and PROPFIND
hrefs reflect the server's on-disk bytes.

The two forms are canonically equivalent but byte-wise different, so a
macOS client requesting the NFD form of an NFC-named file on a Linux
or Windows host got a 404 from the exact-byte lookup. Even against an
APFS host, where the filesystem absorbs the mismatch, the client-side
StatCache could still infer a 404: a cached directory listing in one
form caused depth 0 PROPFINDs in the other form to be treated as not
found without ever reaching the server. The inverse direction (NFD
bytes on a Linux disk, copied there from a Mac, requested in NFC form
by a Windows or Linux client) was broken too.

Alternative regimes considered: normalizing names at storage time (as
Nextcloud and Syncthing's autoNormalize do) would rename user files in
shared directories as a side effect of serving them; normalizing
request paths to a fixed form on the wire is unsound because the
on-disk form is unknowable a priori (ext4 can hold either form, or
both). Instead, adopt the APFS model: preserve bytes, but make lookups
normalization-insensitive.

Concretely, wrap the remote file server's webdav.Dir in a
normalizingFS that, when an exact path lookup fails, rescans the
parent directory for an entry whose name is canonically equivalent,
comparing the NFC form of both sides (which also sidesteps Apple's
nonstandard decomposition tables). Exact matches always win, and newly
created files keep the exact bytes the client sent. Also NFC-normalize
StatCache keys so canonically equivalent names share a cache entry.

The change is covered at three levels: unit tests for the StatCache,
an in-process two-node test in drive/driveimpl, and a new TestTaildrive
VM integration test in tstest/natlab/vmtest that shares a directory
between two Ubuntu VMs and exercises the NFC/NFD cases over the real
stack with curl playing the part of a macOS WebDAV client.

Fixes #15020

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I9c2f157e604efc629828581e08d5b3191dbb7d4e
This commit is contained in:
Brad Fitzpatrick
2026-07-27 06:35:54 -07:00
committed by Brad Fitzpatrick
parent b93d9ba1ff
commit 420a8e5a1a
9 changed files with 520 additions and 7 deletions
+1 -1
View File
@@ -573,7 +573,7 @@ tailscale.com/cmd/tailscaled dependencies: (generated by github.com/tailscale/de
golang.org/x/text/secure/bidirule from golang.org/x/net/idna golang.org/x/text/secure/bidirule from golang.org/x/net/idna
golang.org/x/text/transform from golang.org/x/text/secure/bidirule+ golang.org/x/text/transform from golang.org/x/text/secure/bidirule+
golang.org/x/text/unicode/bidi from golang.org/x/net/idna+ golang.org/x/text/unicode/bidi from golang.org/x/net/idna+
golang.org/x/text/unicode/norm from golang.org/x/net/idna golang.org/x/text/unicode/norm from golang.org/x/net/idna+
golang.org/x/time/rate from gvisor.dev/gvisor/pkg/log+ golang.org/x/time/rate from gvisor.dev/gvisor/pkg/log+
vendor/golang.org/x/crypto/chacha20 from vendor/golang.org/x/crypto/chacha20poly1305 vendor/golang.org/x/crypto/chacha20 from vendor/golang.org/x/crypto/chacha20poly1305
vendor/golang.org/x/crypto/chacha20poly1305 from crypto/hpke+ vendor/golang.org/x/crypto/chacha20poly1305 from crypto/hpke+
+15 -3
View File
@@ -13,6 +13,7 @@ import (
"time" "time"
"github.com/jellydator/ttlcache/v3" "github.com/jellydator/ttlcache/v3"
"golang.org/x/text/unicode/norm"
"tailscale.com/drive/driveimpl/shared" "tailscale.com/drive/driveimpl/shared"
) )
@@ -20,6 +21,17 @@ var (
notFound = newCacheEntry(http.StatusNotFound, nil) notFound = newCacheEntry(http.StatusNotFound, nil)
) )
// normalize converts the given path into a canonical form for use as a cache
// key. In addition to path cleanup, it applies Unicode NFC normalization so
// that canonically equivalent names (e.g. the NFC name on disk and the NFD
// name requested by a macOS WebDAV client) share a single cache entry.
// Without this, a depth 0 lookup for the NFD form of a name would miss the
// entry cached from the NFC href in the parent directory's listing, and get
// would wrongly infer that the file doesn't exist.
func normalize(p string) string {
return norm.NFC.String(shared.Normalize(p))
}
// StatCache provides a cache for directory listings and file metadata. // StatCache provides a cache for directory listings and file metadata.
// Especially when used from the command-line, mapped WebDAV drives can // Especially when used from the command-line, mapped WebDAV drives can
// generate repetitive requests for the same file metadata. This cache helps // generate repetitive requests for the same file metadata. This cache helps
@@ -89,7 +101,7 @@ func (c *StatCache) get(name string, depth int) *cacheEntry {
return nil return nil
} }
name = shared.Normalize(name) name = normalize(name)
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
@@ -142,7 +154,7 @@ func (c *StatCache) set(name string, depth int, ce *cacheEntry) {
return return
} }
name = shared.Normalize(name) name = normalize(name)
var self *cacheEntry var self *cacheEntry
var children map[string]*cacheEntry var children map[string]*cacheEntry
@@ -171,7 +183,7 @@ func (c *StatCache) set(name string, depth int, ce *cacheEntry) {
log.Printf("statcache.set child parse error: %s", err) log.Printf("statcache.set child parse error: %s", err)
return return
} }
name = shared.Normalize(name) name = normalize(name)
raw := marshalMultiStatus(response) raw := marshalMultiStatus(response)
entry := newCacheEntry(ce.Status, raw) entry := newCacheEntry(ce.Status, raw)
if i == 0 { if i == 0 {
@@ -7,6 +7,7 @@ import (
"fmt" "fmt"
"log" "log"
"net/http" "net/http"
"net/url"
"path" "path"
"strings" "strings"
"testing" "testing"
@@ -211,3 +212,51 @@ func TestParentChildRelationship(t *testing.T) {
}) })
} }
} }
// TestUnicodeNormalizationInsensitivity verifies that cache lookups treat
// canonically equivalent names as the same key. A parent directory listing
// caches children under the names from the server's hrefs (often NFC), while
// macOS WebDAV clients request the same files using NFD names. Without
// normalization, the depth 0 lookup would miss and get would wrongly infer
// notFound from the cached parent.
func TestUnicodeNormalizationInsensitivity(t *testing.T) {
// Make sure we don't leak goroutines
tstest.ResourceCheck(t)
c := &StatCache{TTL: 24 * time.Hour} // don't expire
defer c.stop()
const (
nfcParent = "\u30ae\u30bf\u30fc" // ギター in NFC: ギ is the single code point U+30AE
nfdParent = "\u30ad\u3099\u30bf\u30fc" // same name in NFD: キ U+30AD plus combining voiced mark U+3099
nfcChild = "\u30c6\u30ba\u30c8.wav" // テズト.wav in NFC: ズ is the single code point U+30BA
nfdChild = "\u30c6\u30b9\u3099\u30c8.wav" // same name in NFD: ス U+30B9 plus combining voiced mark U+3099
)
nfcParentPath := "/" + nfcParent
nfdParentPath := "/" + nfdParent
nfcChildPath := nfcParentPath + "/" + nfcChild
nfdChildPath := nfdParentPath + "/" + nfdChild
// The hrefs in a real PROPFIND response contain the percent-encoded UTF-8
// bytes of the names as they appear on the server's disk, here NFC.
unicodeParentResponse := strings.ReplaceAll(parentResponse, "/parent%20with%20spaces/", "/"+url.PathEscape(nfcParent)+"/")
unicodeChildResponse := strings.ReplaceAll(childResponse, "/parent%20with%20spaces/child.txt", "/"+url.PathEscape(nfcParent)+"/"+url.PathEscape(nfcChild))
unicodeFullParent := []byte(
strings.ReplaceAll(
fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><D:multistatus xmlns:D="DAV:">%s%s</D:multistatus>`, unicodeParentResponse, unicodeChildResponse),
"\n", ""))
unicodeFullChild := []byte(
strings.ReplaceAll(
fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><D:multistatus xmlns:D="DAV:">%s</D:multistatus>`, unicodeChildResponse),
"\n", ""))
c.set(nfcParentPath, 1, newCacheEntry(http.StatusMultiStatus, unicodeFullParent))
want := newCacheEntry(http.StatusMultiStatus, unicodeFullChild)
for _, childPath := range []string{nfcChildPath, nfdChildPath} {
got := c.get(childPath, 0)
if diff := cmp.Diff(got, want); diff != "" {
t.Errorf("get(%q): unexpected cached value; (-got+want):%v", childPath, diff)
}
}
}
+1 -1
View File
@@ -102,7 +102,7 @@ func (s *FileServer) ClearSharesLocked() {
// has been called first. // has been called first.
func (s *FileServer) AddShareLocked(share, path string) { func (s *FileServer) AddShareLocked(share, path string) {
s.shareHandlers[share] = &webdav.Handler{ s.shareHandlers[share] = &webdav.Handler{
FileSystem: &birthTimingFS{webdav.Dir(path)}, FileSystem: &birthTimingFS{&normalizingFS{webdav.Dir(path)}},
LockSystem: webdav.NewMemLS(), LockSystem: webdav.NewMemLS(),
} }
} }
+113
View File
@@ -0,0 +1,113 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package driveimpl
import (
"context"
"errors"
iofs "io/fs"
"os"
"path"
"strings"
"github.com/tailscale/xnet/webdav"
"golang.org/x/text/unicode/norm"
)
// normalizingFS extends a webdav.FileSystem to resolve paths in a Unicode
// normalization-insensitive way. Different clients encode non-ASCII filenames
// differently: for example, macOS WebDAV clients request paths in NFD form,
// while files on disk are often named in NFC form. On normalization-sensitive
// filesystems like ext4, opening the NFD name of an NFC file fails. When an
// exact lookup fails, normalizingFS rescans the parent directory for an entry
// whose name is canonically equivalent to the requested one and uses that
// instead. See https://github.com/tailscale/tailscale/issues/15020.
type normalizingFS struct {
webdav.FileSystem
}
func (fs *normalizingFS) Mkdir(ctx context.Context, name string, perm os.FileMode) error {
return fs.FileSystem.Mkdir(ctx, fs.resolve(ctx, name), perm)
}
func (fs *normalizingFS) OpenFile(ctx context.Context, name string, flag int, perm os.FileMode) (webdav.File, error) {
return fs.FileSystem.OpenFile(ctx, fs.resolve(ctx, name), flag, perm)
}
func (fs *normalizingFS) RemoveAll(ctx context.Context, name string) error {
return fs.FileSystem.RemoveAll(ctx, fs.resolve(ctx, name))
}
func (fs *normalizingFS) Rename(ctx context.Context, oldName, newName string) error {
return fs.FileSystem.Rename(ctx, fs.resolve(ctx, oldName), fs.resolve(ctx, newName))
}
func (fs *normalizingFS) Stat(ctx context.Context, name string) (os.FileInfo, error) {
return fs.FileSystem.Stat(ctx, fs.resolve(ctx, name))
}
// resolve maps name to the path of an existing file whose name is canonically
// equivalent to name under Unicode NFC normalization. Exact matches always
// win. Path components with no existing equivalent are left as given, so that
// newly created files keep the exact name that the client requested.
func (fs *normalizingFS) resolve(ctx context.Context, name string) string {
if isASCII(name) {
// ASCII strings are canonically equivalent only to themselves.
return name
}
if _, err := fs.FileSystem.Stat(ctx, name); err == nil || !errors.Is(err, iofs.ErrNotExist) {
return name
}
resolved := "/"
parts := strings.Split(strings.Trim(path.Clean("/"+name), "/"), "/")
for i, part := range parts {
candidate := path.Join(resolved, part)
if isASCII(part) {
resolved = candidate
continue
}
if _, err := fs.FileSystem.Stat(ctx, candidate); err == nil {
resolved = candidate
continue
}
match, ok := fs.findEquivalent(ctx, resolved, part)
if !ok {
// No equivalent entry exists. Keep the remaining components
// as given; deeper lookups would fail anyway.
return path.Join(append([]string{resolved}, parts[i:]...)...)
}
resolved = path.Join(resolved, match)
}
return resolved
}
// findEquivalent scans the directory at dir for an entry whose name is
// canonically equivalent to name, returning the on-disk name if found.
func (fs *normalizingFS) findEquivalent(ctx context.Context, dir, name string) (string, bool) {
d, err := fs.FileSystem.OpenFile(ctx, dir, os.O_RDONLY, 0)
if err != nil {
return "", false
}
defer d.Close()
fis, err := d.Readdir(0)
if err != nil {
return "", false
}
want := norm.NFC.String(name)
for _, fi := range fis {
if norm.NFC.String(fi.Name()) == want {
return fi.Name(), true
}
}
return "", false
}
func isASCII(s string) bool {
for i := 0; i < len(s); i++ {
if s[i] >= 0x80 {
return false
}
}
return true
}
+114
View File
@@ -0,0 +1,114 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package driveimpl
import (
"os"
"path/filepath"
"testing"
"tailscale.com/drive"
)
// Unicode filenames from https://github.com/tailscale/tailscale/issues/15020,
// in both NFC (precomposed) and NFD (decomposed) forms. The two forms are
// canonically equivalent but byte-wise different, so on
// normalization-sensitive filesystems like ext4 they name different files.
const (
nfcName = "\u30c6\u30ba\u30c8 \u00e4.wav" // テズト ä.wav, precomposed
nfdName = "\u30c6\u30b9\u3099\u30c8 a\u0308.wav" // same name, decomposed
nfcDir = "\u30ae\u30bf\u30fc" // ギター, precomposed
nfdDir = "\u30ad\u3099\u30bf\u30fc" // same name, decomposed
)
// TestUnicodeRoundTrip verifies that a file written via WebDAV with a
// non-ASCII name can be statted and read back with the identical name.
func TestUnicodeRoundTrip(t *testing.T) {
s := newSystem(t)
s.addRemote(remote1)
s.addShare(remote1, share11, drive.PermissionReadWrite)
s.writeFile("writing unicode file should succeed", remote1, share11, nfcName, "hello", true)
s.checkFileStatus(remote1, share11, nfcName)
s.checkFileContents(remote1, share11, nfcName)
}
// TestUnicodeNormalizationMismatch verifies that a file whose on-disk name is
// in one Unicode normalization form can be read, statted and overwritten via
// WebDAV using a canonically equivalent name in the other form. This is what
// happens when a macOS WebDAV client (which sends NFD paths) accesses a share
// with NFC filenames on disk, and vice versa.
func TestUnicodeNormalizationMismatch(t *testing.T) {
tests := []struct {
name string
onDisk, viaRequest string
}{
{"nfc-on-disk-nfd-request", nfcName, nfdName},
{"nfd-on-disk-nfc-request", nfdName, nfcName},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := newSystem(t)
s.addRemote(remote1)
s.addShare(remote1, share11, drive.PermissionReadWrite)
// Write the file directly to disk with one form, then read and
// stat it via WebDAV using the other form.
s.write(remote1, share11, tt.onDisk, "hello world")
if got := s.readViaWebDAV(remote1, share11, tt.viaRequest); got != "hello world" {
t.Errorf("read: got %q, want %q", got, "hello world")
}
s.statViaWebDAV(remote1, share11, tt.viaRequest)
// Overwriting via the other form must update the existing file
// rather than creating a second one.
s.writeFile("overwrite with equivalent name should succeed", remote1, share11, tt.viaRequest, "updated", true)
if got := s.read(remote1, share11, tt.onDisk); got != "updated" {
t.Errorf("read from disk after overwrite: got %q, want %q", got, "updated")
}
shareDir := s.remotes[remote1].shares[share11]
entries, err := os.ReadDir(shareDir)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 {
t.Errorf("got %d files on disk, want 1: %q", len(entries), entries)
}
})
}
}
// TestUnicodeNormalizationMismatchInDir is like
// TestUnicodeNormalizationMismatch, but with the mismatch in a directory
// component of the path rather than in the filename.
func TestUnicodeNormalizationMismatchInDir(t *testing.T) {
s := newSystem(t)
s.addRemote(remote1)
s.addShare(remote1, share11, drive.PermissionReadWrite)
shareDir := s.remotes[remote1].shares[share11]
if err := os.Mkdir(filepath.Join(shareDir, nfcDir), 0755); err != nil {
t.Fatal(err)
}
s.write(remote1, share11, filepath.Join(nfcDir, nfcName), "hello world")
got := s.readViaWebDAV(remote1, share11, nfdDir+"/"+nfdName)
if got != "hello world" {
t.Errorf("read: got %q, want %q", got, "hello world")
}
}
// TestUnicodeNewFileKeepsRequestedName verifies that creating a new file
// keeps the exact bytes of the name that the client sent when no equivalent
// file exists yet.
func TestUnicodeNewFileKeepsRequestedName(t *testing.T) {
s := newSystem(t)
s.addRemote(remote1)
s.addShare(remote1, share11, drive.PermissionReadWrite)
s.writeFile("writing new NFD file should succeed", remote1, share11, nfdName, "hello", true)
if got := s.read(remote1, share11, nfdName); got != "hello" {
t.Errorf("read from disk: got %q, want %q", got, "hello")
}
}
+1 -1
View File
@@ -4,7 +4,7 @@
"sri": "sha256-TwIaPmGVHO9ZwdaO/jDStgWGQtKa4smS3er1i5aTNTw=" "sri": "sha256-TwIaPmGVHO9ZwdaO/jDStgWGQtKa4smS3er1i5aTNTw="
}, },
"vendor": { "vendor": {
"goModSum": "sha256-7j0GzgLJIFwoNkWuJaxiujq+cCnIJVs/z0cvhNek66U=", "goModSum": "sha256-eB5NyqpDoiGHKrFlobSbkZLdjfA3lSSqpPoIMhFAurc=",
"sri": "sha256-5ClQ5fSyEHUlhPtZI0ir8ddQRXSnqOG5VIJ3KjWtXmw=" "sri": "sha256-5ClQ5fSyEHUlhPtZI0ir8ddQRXSnqOG5VIJ3KjWtXmw="
} }
} }
+1 -1
View File
@@ -511,7 +511,7 @@ require (
go.uber.org/multierr v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f // indirect golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f // indirect
golang.org/x/image v0.41.0 golang.org/x/image v0.41.0
golang.org/x/text v0.40.0 // indirect golang.org/x/text v0.40.0
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/inf.v0 v0.9.1 // indirect
+225
View File
@@ -0,0 +1,225 @@
// Copyright (c) Tailscale Inc & contributors
// SPDX-License-Identifier: BSD-3-Clause
package vmtest_test
import (
"fmt"
"net/url"
"regexp"
"strconv"
"strings"
"testing"
"time"
"github.com/creachadair/mds/shell"
"tailscale.com/tailcfg"
"tailscale.com/tstest"
"tailscale.com/tstest/natlab/vmtest"
"tailscale.com/tstest/natlab/vnet"
)
const (
driveShareName = "docs"
driveShareDir = "/srv/taildrive-share"
driveASCIIName = "hello.txt"
driveASCIIContent = "hello world"
// driveKanaNFC and driveKanaNFD are the same filename (テズト ä.wav) in
// NFC (precomposed) and NFD (decomposed) Unicode normalization forms,
// from tailscale/tailscale#15020. They are canonically equivalent but
// byte-wise different: in NFC, ズ is the single code point U+30BA and ä
// is U+00E4; in NFD they are ス U+30B9 plus the combining voiced sound
// mark U+3099, and "a" plus the combining diaeresis U+0308.
driveKanaNFC = "\u30c6\u30ba\u30c8 \u00e4.wav"
driveKanaNFD = "\u30c6\u30b9\u3099\u30c8 a\u0308.wav"
driveKanaContent = "kana content"
)
// TestTaildrive shares a directory from one node and accesses it from another
// via the accessing node's local Taildrive WebDAV proxy at
// 100.100.100.100:8080, first with a plain ASCII filename and then with
// filenames whose Unicode normalization form differs between the request and
// the on-disk name.
//
// The Unicode cases simulate a macOS WebDAV client, which requests paths in
// NFD form, accessing a share on a Linux disk whose filenames are NFC bytes
// (tailscale/tailscale#15020). Requesting a directory listing immediately
// before the NFD PROPFIND also exercises the accessing node's StatCache,
// which caches children under their NFC hrefs and previously inferred 404
// for the NFD name without contacting the share host.
func TestTaildrive(t *testing.T) {
env := vmtest.New(t, vmtest.AllOnline())
hostNet := env.AddNetwork("1.0.0.1", "192.168.1.1/24", vnet.EasyNAT)
clientNet := env.AddNetwork("2.0.0.1", "192.168.2.1/24", vnet.EasyNAT)
// Taildrive is disabled unless control sends these node attributes.
driveCaps := tailcfg.NodeCapMap{
tailcfg.NodeAttrsTaildriveShare: nil,
tailcfg.NodeAttrsTaildriveAccess: nil,
}
host := env.AddNode("drivehost", hostNet,
vmtest.OS(vmtest.Ubuntu2404),
driveCaps)
client := env.AddNode("driveclient", clientNet,
vmtest.OS(vmtest.Ubuntu2404),
driveCaps)
// Declare test-specific steps for the web UI.
setupStep := env.AddStep("Create files and share on host")
waitStep := env.AddStep("Wait for share to be accessible from client")
asciiStep := env.AddStep("ASCII file access (client -> host)")
unicodeStep := env.AddStep("NFC/NFD normalization mismatch access")
env.Start()
// Access to a peer's shares additionally requires peer capabilities,
// normally granted via ACL grants. Grant every peer read/write access to
// every share, plus the sharer capability that makes share hosts show up
// in directory listings on accessing nodes.
env.ControlServer().SetGlobalAppCaps(tailcfg.PeerCapMap{
tailcfg.PeerCapabilityTaildrive: {`{"shares":["*"],"access":"rw"}`},
tailcfg.PeerCapabilityTaildriveSharer: {`true`},
})
setupStep.Begin()
setupCmd := fmt.Sprintf("mkdir -p %s && printf %%s %s > %s && printf %%s %s > %s",
shell.Quote(driveShareDir),
shell.Quote(driveASCIIContent), shell.Quote(driveShareDir+"/"+driveASCIIName),
shell.Quote(driveKanaContent), shell.Quote(driveShareDir+"/"+driveKanaNFC))
if out, err := env.SSHExec(host, setupCmd); err != nil {
setupStep.Fatalf("share dir setup: %v\n%s", err, out)
return
}
if out, err := env.Tailscale(host, "drive", "share", driveShareName, driveShareDir); err != nil {
setupStep.Fatalf("tailscale drive share: %v\n%s", err, out)
return
}
setupStep.End(nil)
// Discover the tailnet domain (the top-level directory of the WebDAV
// tree) by listing the root, then wait until the host's share is
// reachable from the client. The peer capability grants pushed above
// take a map update to arrive at both nodes.
waitStep.Begin()
var shareBase string // /<domain>/drivehost/docs, percent-encoded
if err := tstest.WaitFor(2*time.Minute, func() error {
status, body, err := webdavCurl(env, client, "PROPFIND", "/", "-H", "Depth: 1")
if err != nil {
return err
}
if status != 207 {
return fmt.Errorf("PROPFIND /: status %d: %s", status, body)
}
domain := ""
for _, m := range hrefRegex.FindAllStringSubmatch(body, -1) {
if p := strings.Trim(m[1], "/"); p != "" {
domain = p
break
}
}
if domain == "" {
return fmt.Errorf("no domain in root listing: %s", body)
}
shareBase = "/" + domain + "/" + url.PathEscape(host.Name()) + "/" + url.PathEscape(driveShareName)
status, body, err = webdavCurl(env, client, "GET", shareBase+"/"+url.PathEscape(driveASCIIName))
if err != nil {
return err
}
if status != 200 {
return fmt.Errorf("GET %s: status %d: %s", driveASCIIName, status, body)
}
return nil
}); err != nil {
waitStep.Fatalf("share never became accessible: %v", err)
return
}
waitStep.End(nil)
asciiStep.Begin()
status, body, err := webdavCurl(env, client, "GET", shareBase+"/"+url.PathEscape(driveASCIIName))
if err != nil || status != 200 || body != driveASCIIContent {
asciiStep.Fatalf("GET %s = %d, %q, %v; want 200, %q", driveASCIIName, status, body, err, driveASCIIContent)
return
}
status, body, err = webdavCurl(env, client, "PROPFIND", shareBase+"/", "-H", "Depth: 1")
if err != nil || status != 207 || !strings.Contains(body, driveASCIIName) {
asciiStep.Fatalf("PROPFIND share = %d, %v; want 207 mentioning %s:\n%s", status, err, driveASCIIName, body)
return
}
asciiStep.End(nil)
unicodeStep.Begin()
// List the directory first. The response hrefs carry the NFC bytes from
// the host's disk, and the listing primes the client's StatCache, whose
// entries live for 10 seconds. The depth 0 PROPFIND for the NFD name
// that follows is answered from that cache, so it must treat the two
// forms as the same name.
status, body, err = webdavCurl(env, client, "PROPFIND", shareBase+"/", "-H", "Depth: 1")
if err != nil || status != 207 || !strings.Contains(body, url.PathEscape(driveKanaNFC)) {
unicodeStep.Fatalf("PROPFIND share = %d, %v; want 207 mentioning NFC name:\n%s", status, err, body)
return
}
nfdPath := shareBase + "/" + url.PathEscape(driveKanaNFD)
status, body, err = webdavCurl(env, client, "PROPFIND", nfdPath, "-H", "Depth: 0")
if err != nil || status != 207 {
unicodeStep.Fatalf("PROPFIND NFD name = %d, %v; want 207:\n%s", status, err, body)
return
}
status, body, err = webdavCurl(env, client, "GET", nfdPath)
if err != nil || status != 200 || body != driveKanaContent {
unicodeStep.Fatalf("GET NFD name = %d, %q, %v; want 200, %q", status, body, err, driveKanaContent)
return
}
// Overwriting via the NFD name must update the NFC file on the host's
// disk rather than creating a second file.
const updatedContent = "updated kana content"
status, body, err = webdavCurl(env, client, "PUT", nfdPath, "--data-binary", updatedContent)
if err != nil || status/100 != 2 {
unicodeStep.Fatalf("PUT NFD name = %d, %v; want 2xx:\n%s", status, err, body)
return
}
out, err := env.SSHExec(host, "cat "+shell.Quote(driveShareDir+"/"+driveKanaNFC))
if err != nil || out != updatedContent {
unicodeStep.Fatalf("host file after PUT = %q, %v; want %q", out, err, updatedContent)
return
}
out, err = env.SSHExec(host, "ls "+shell.Quote(driveShareDir)+" | wc -l")
if err != nil || strings.TrimSpace(out) != "2" {
unicodeStep.Fatalf("host share has %s files after PUT, %v; want 2", strings.TrimSpace(out), err)
return
}
unicodeStep.End(nil)
}
var hrefRegex = regexp.MustCompile(`<D:href>([^<]*)</D:href>`)
// webdavCurl performs a WebDAV request from the given node against the
// node's local Taildrive WebDAV proxy at 100.100.100.100:8080 by running
// curl over the node's debug SSH connection. The path must already be
// percent-encoded; extraArgs are passed to curl before the URL. It returns
// the HTTP status code and the response body.
func webdavCurl(env *vmtest.Env, n *vmtest.Node, method, path string, extraArgs ...string) (status int, body string, err error) {
cmd := "curl -s --max-time 15 -X " + shell.Quote(method)
for _, arg := range extraArgs {
cmd += " " + shell.Quote(arg)
}
// curl expands the \n itself, putting the status code on its own final
// line after the unmodified response body.
cmd += ` -w '\n%{http_code}' ` + shell.Quote("http://100.100.100.100:8080"+path)
out, err := env.SSHExec(n, cmd)
if err != nil {
return 0, "", fmt.Errorf("curl: %v\n%s", err, out)
}
i := strings.LastIndexByte(out, '\n')
if i == -1 {
return 0, "", fmt.Errorf("no status line in curl output: %q", out)
}
status, err = strconv.Atoi(strings.TrimSpace(out[i+1:]))
if err != nil {
return 0, "", fmt.Errorf("bad status line in curl output: %q", out)
}
return status, out[:i], nil
}