118 Commits
Author SHA1 Message Date
codingetandCodex ebb4cec8c1 ci: run CA bundle updates daily
CI / format (pull_request) Successful in 2m52s
CI / lint (pull_request) Successful in 2m56s
CI / install (pull_request) Successful in 8m2s
CI / typetest (pull_request) Successful in 3m6s
CI / typecheck (pull_request) Successful in 3m27s
CI / node-tests (pull_request) Successful in 3m54s
CI / browser-tests (pull_request) Successful in 6m14s
Co-Authored-By: gpt-5.6-luna <noreply@openai.com>
2026-09-01 12:25:43 +00:00
webnet-actions 6aa38409c8 chore(tsconnect): update CA bundle 2026-09-01 08:17:53 +00:00
codingetandClaude 3394db7271 test(tsconnect): cover browser initialization readiness
CI / format (pull_request) Successful in 4m57s
CI / lint (pull_request) Successful in 5m3s
CI / install (pull_request) Successful in 13m1s
CI / typetest (pull_request) Successful in 2m55s
CI / typecheck (pull_request) Successful in 3m53s
CI / node-tests (pull_request) Successful in 4m36s
CI / browser-tests (pull_request) Successful in 8m56s
The node consumer tests cover the built package under node; this covers
the same readiness condition in Chromium and Firefox, which is where a
runtime that published its bridge on a global would be observed by a
real user.

The tests serve out/ and dist/ as they ship, with an import map for the
one bare specifier the built index.js carries, so they exercise the
published files rather than a bundle built specially for them.

Covered: initIPN resolving only once the bridge answers, no bridge
globals left on the page, two runtimes on one page staying independent
when one shuts down, and shutdown raising nothing afterwards. That last
one watches for error and unhandledrejection events for two seconds,
which is the shape webnet/webnet#147 took in a browser.

Verified by control: keeping the init callback global alive fails the
globals test in both browsers.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 15:31:07 +00:00
codingetandClaude 606bda58dd fix(tsconnect): reject instead of hanging if the runtime dies mid-startup
CI / lint (pull_request) Successful in 4m24s
CI / format (pull_request) Successful in 4m48s
CI / install (pull_request) Successful in 11m14s
CI / typetest (pull_request) Successful in 2m3s
CI / typecheck (pull_request) Successful in 2m39s
CI / node-tests (pull_request) Successful in 3m26s
CI / browser-tests (pull_request) Successful in 5m21s
initIPN raced the runtime only until the readiness callback fired, then
awaited the factory alone. Building the backend runs in a Go goroutine,
and if the runtime dies partway through — a panic in the bridge, say —
that goroutine dies with it and its promise never settles, so initIPN
hung forever rather than rejecting. Keep the runtime in the race until
the factory returns.

Verified by injecting a runtime exit while the factory was pending: the
built package now rejects in under a second where it previously never
returned. The error text no longer says "before signalling readiness",
since it can now surface after readiness too.

The test-app page still showed the factory-shaped example, which throws
if followed. My earlier sweep for it only covered TypeScript and
markdown.

Also advances the submodule to the matching fork fixes, which close the
same gap in startIPN and give createIPN an awaitable shutdown.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 15:30:56 +00:00
codingetandClaude 7569a2dabd docs(tsconnect): note why the startup race stays quiet on shutdown
CI / lint (pull_request) Successful in 2m44s
CI / format (pull_request) Successful in 3m7s
CI / install (pull_request) Successful in 8m39s
CI / typetest (pull_request) Successful in 2m27s
CI / typecheck (pull_request) Successful in 2m49s
CI / node-tests (pull_request) Successful in 3m23s
CI / browser-tests (pull_request) Successful in 5m19s
The promise derived from go.run() rejects on every exit, not only a
failed startup. Nothing observes that today because Promise.race
subscribes to both promises, so the late rejection still has a handler
once readiness has won. Reaching for the winner some other way would turn
every clean shutdown into an unhandled rejection, which is worth a
comment rather than a rediscovery.

Also advances the submodule to the fork's review fix for a deliberate
shutdown being reported as a panic.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 01:28:12 +00:00
codingetandClaude 1b60ae134c ci: re-run against the pushed submodule commit
CI / lint (pull_request) Successful in 2m57s
CI / format (pull_request) Successful in 2m52s
CI / install (pull_request) Successful in 9m26s
CI / typetest (pull_request) Successful in 2m27s
CI / typecheck (pull_request) Canceled after 3m9s
CI / browser-tests (pull_request) Canceled after 3m10s
CI / node-tests (pull_request) Canceled after 3m23s
The submodule pointer referenced a fork commit that had not been pushed
yet, so CI could not check it out.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 01:15:52 +00:00
codingetandClaude 855e2a8026 test(tsconnect): cover the built package from a native ESM consumer
CI / install (pull_request) Failing after 36s
CI / typecheck (pull_request) Skipped
CI / typetest (pull_request) Skipped
CI / node-tests (pull_request) Skipped
CI / browser-tests (pull_request) Skipped
CI / lint (pull_request) Canceled after 1m42s
CI / format (pull_request) Canceled after 1m44s
The suite runs under tsx, which loads modules differently enough to hide
a startup failure that only a plain node consumer would hit. That is why
webnet/webnet#188 went unnoticed: source tests never exercised the path
the bug was on.

These spawn a real node process against out/index.js and dist/main.wasm
with NODE_OPTIONS cleared, and check that one IPN initializes, that no
bridge globals are left behind, that a second runtime starts after the
first exits, and that the process ends on its own after shutdown rather
than being held open by a leftover scheduler timeout.

Verified by reverting the mechanism: reading globalThis.newIPN straight
after go.run() fails the first test with the exact error from the issue,
and keeping the callback global alive fails the second.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 01:14:01 +00:00
codingetandClaude 28d15c5f88 feat(tsconnect)!: initialize exactly one IPN per WASM runtime
initIPN returned a factory that could build any number of IPNs from one
Go runtime, but nothing about the lifecycle was per-IPN: shutdown() exits
the runtime, so the second instance was always going to die with the
first. The signature now says what was always true.

initIPN(wasm, config) returns the single IPN. The raw factory never
escapes, so a second IPN is impossible by construction, with the fork's
atomic guard behind it.

Startup no longer reads a global. The loader installs a callback under a
generated name, passes the name through go.env, and the runtime invokes
it once the bridge is built. Readiness is the call, not a guess about how
far the Go scheduler has run, and nothing is left on globalThis. Racing
that against go.run() means a runtime that dies during startup rejects
instead of hanging.

A failed build now terminates the runtime through the fork's terminate
callback and waits for it, so a rejected initIPN leaves no live runtime
and no armed scheduler timeouts.

useBuildIpn takes an init function rather than a builder plus a separate
parameter object; the two-phase shape only existed because the WASM load
and the config arrived at different times. Building an IPN is now async
and starts a runtime, so the hook gained the unmount handling
useBuildIpnWorker already had: unmounting mid-init shuts the result down
instead of stranding a runtime and a live tailnet node.

The integration tests keep four nodes, now four runtimes, and drop the
workarounds that existed because they shared one. The comment claiming a
fresh runtime costs 30-50 s was measuring the tailnet connect, not the
WASM: instantiating one takes about 85 ms.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-30 01:13:57 +00:00
codingetandCodex 80db78c795 test(tailshare): drive autostart with the browser clock
CI / lint (pull_request) Successful in 1m57s
CI / format (pull_request) Successful in 2m26s
CI / install (pull_request) Successful in 7m48s
CI / typetest (pull_request) Successful in 2m22s
CI / typecheck (pull_request) Successful in 3m10s
CI / node-tests (pull_request) Successful in 3m28s
CI / browser-tests (pull_request) Successful in 5m34s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-29 00:31:50 +00:00
codingetandCodex bad8d7c219 test(tailshare): cover credential lifecycle in browsers
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-29 00:31:50 +00:00
codingetandClaude 14bb372490 feat(taildrive): browse every peer's shares as one filesystem
CI / lint (pull_request) Successful in 2m15s
CI / format (pull_request) Successful in 2m17s
CI / install (pull_request) Successful in 6m54s
CI / typetest (pull_request) Successful in 2m30s
CI / typecheck (pull_request) Successful in 3m3s
CI / node-tests (pull_request) Successful in 3m53s
CI / browser-tests (pull_request) Successful in 7m39s
`TaildriveVFS` lists the peers found by `listDrivePeersWithShares` at its root
and delegates everything below one to that peer's `DAVClient`, so a caller sees
`/<peer>/<share>/...` instead of one filesystem per peer.

The root and peer directories are answered locally: an unreachable peer is
simply absent from the listing rather than failing it, and nothing can be
created, modified, or removed at those two levels. Peers are keyed by
`stableNodeID`, and every peer in a display-name collision is listed as
`<name> (<stableNodeID>)` so that the names do not depend on discovery order.
Discovery is cached for `refreshInterval` and can be forced with `refresh()`;
concurrent callers share one probe. One connection pool serves every peer.

Issue #170 specified a `/<magic-dns-suffix>/<peer>/<share>/...` tree. That
first segment is derived from `self.name` and so is constant for every entry
beneath it, which means it disambiguates nothing: `IPNDrivePeer.name` is
already a MagicDNS base name for native peers and an FQDN for shared-in ones,
and the two cannot collide. Dropping it, as the issue discussion asked, also
drops the optional `NetworkMap.Domain` mode and the tsconnect bridge work it
needed.

Closes #170

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 23:59:42 +00:00
codingetandClaude dcb85e284c refactor(taildrive): split the client barrel from its discovery code
Makes room for a second client module without an import cycle through the
barrel, matching the layout `packages/webdav/src/client` already uses.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 22:20:58 +00:00
codingetandClaude 06fcad59ee fix(http): never write a zero-length chunk into a chunked body
CI / lint (pull_request) Successful in 3m13s
CI / format (pull_request) Successful in 3m24s
CI / install (pull_request) Successful in 10m23s
CI / typecheck (pull_request) Successful in 3m5s
CI / typetest (pull_request) Successful in 1m44s
CI / node-tests (pull_request) Successful in 2m32s
CI / browser-tests (pull_request) Successful in 4m7s
A chunk of size zero is the last-chunk, so writing one for an empty piece of a
body ended the message early and left the real terminator in the peer's read
buffer, where it was parsed as the start of the next message. On a keep-alive
connection the following request or response then failed with a 400, which is
how a `DAVClient` upload of an empty file broke every later request on that
connection. The BYOB stream path already skipped empty chunks; the other two
encoders now do the same.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 22:20:51 +00:00
codingetandClaude a837fdfac7 ci: expire CI build artifacts after a week
CI / format (pull_request) Successful in 2m10s
CI / lint (pull_request) Successful in 2m16s
CI / install (pull_request) Successful in 7m18s
CI / typecheck (pull_request) Successful in 2m1s
CI / typetest (pull_request) Successful in 2m10s
CI / node-tests (pull_request) Successful in 2m53s
CI / browser-tests (pull_request) Successful in 4m3s
The install job uploads node_modules, the turbo cache, and the build
output on every pull request run, which adds up to tens of gigabytes of
storage for data only the downstream jobs in the same run ever read.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 18:04:57 +00:00
codingetandClaude c38c78893b refactor(tailshare): keep node types out of the browser project
CI / format (pull_request) Successful in 1m53s
CI / lint (pull_request) Successful in 2m6s
CI / install (pull_request) Successful in 6m6s
CI / typetest (pull_request) Successful in 2m6s
CI / node-tests (pull_request) Successful in 2m20s
CI / typecheck (pull_request) Successful in 2m22s
CI / browser-tests (pull_request) Successful in 3m38s
Adding node types for the new unit tests applied them to every file, so
browser code could import a node builtin and still typecheck. Split the
project: the app compiles with no ambient type packages and excludes the
tests, and a test project adds node types back for the test files alone.

Verified by importing node:fs into a browser module and confirming the
app project rejects it while the test project does not.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 17:30:23 +00:00
codingetandClaude 023814a604 fix(tailshare): bind a failed build's redaction to the key it used
build() overwrote both snapshots with no regard for an attempt already in
flight, and AutoInit re-arms its timer whenever a preference changes, so
a second build could land between attempt A starting and attempt A
rejecting. The handler then redacted with key B and rendered key A. Take
an in-flight guard, and route all three failure paths through one helper
so they cannot drift apart.

The same guard stops a re-fired build from overwriting configAtBuildRef
after a successful connection, which had been quietly clearing the
restart banner.

setAuthKeyPersisted(false) removed the stored copy without taking it into
memory first, so after a reload, where the key exists only in storage,
unchecking the box dropped the credential instead of keeping it for the
tab.

Reported by gpt-5.6-sol on PR #237.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 17:30:23 +00:00
codingetandClaude 6b9e47320e fix(tailshare): close three gaps found reviewing the credential store
The reload test never cleared the module's memory before swapping in a
fresh storage, so it passed with the localStorage fallback deleted
outright. Clear memory first, which is what a reload actually does.

Opting into persistence with no key yet typed wrote an empty string, and
since the stored value is the flag, that read back as "not persisted" and
silently dropped the opt-in for the next key. Store nothing instead.

The store holds one key for both the field and the build snapshot, so
discarding on registration could throw away a replacement the user had
typed for the next build. Only discard what this connection registered
with.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 17:30:23 +00:00
codingetandClaude 3dabcf49a5 fix(tailshare): keep Tailscale auth keys out of persisted preferences
An auth key enrolls a device on the tailnet, and serializing it into
tailshare:config alongside the hostname and exit node left it readable by
every same-origin script long after the registration flow needed it.

Move it to a credential store that lives in the tab's memory and is
discarded once the node reaches Running. Persisting it is now an explicit
opt-in with its own storage key and a warning, and the presence of that
stored copy is the flag, so there is no separate setting to drift out of
sync. The raw key no longer reaches the prepare context: consumers see
whether one is set, and the input is write-only.

parseConfig no longer reads authKey, so a key left in an older
tailshare:config is ignored and drops out on the next settings write.
There is no migration.

Splitting parseConfig and the failure redaction out of IpnContext gives
tailshare its first unit tests, covering the storage rules, the opt-in,
and that no failure path serializes the key. Reload, worker success,
initialization failure, and fallback still need a browser harness.

Closes #190

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 17:30:23 +00:00
codingetandClaude a6af0449fb feat(tsconnect-worker): let a keyless client attach to a registered IPN
An auth key only ever feeds the initial registration, so comparing it on
every later connection turns a tab that no longer holds the key into a
hard mismatch with no fallback. Compare it in one direction: a requested
config that omits the key attaches to whatever the worker registered, and
a different key is still rejected.

Settles the omitted-key question from #168 so that #190 can discard the
credential after registration. The rest of the reconfiguration design
stays with #168.

Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
2026-08-28 17:30:09 +00:00
codingetandCodex f5da644fff docs: align project terminology and policies
CI / lint (pull_request) Successful in 1m51s
CI / format (pull_request) Successful in 2m3s
CI / install (pull_request) Successful in 6m59s
CI / typetest (pull_request) Successful in 1m53s
CI / typecheck (pull_request) Successful in 2m30s
CI / node-tests (pull_request) Successful in 2m58s
CI / browser-tests (pull_request) Successful in 3m45s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-27 18:54:43 +00:00
codingetandCodex 53cbd5228e docs: define Tailshare product direction
CI / lint (pull_request) Successful in 2m2s
CI / format (pull_request) Successful in 2m15s
CI / install (pull_request) Successful in 7m2s
CI / typetest (pull_request) Successful in 1m55s
CI / typecheck (pull_request) Successful in 2m27s
CI / node-tests (pull_request) Successful in 2m52s
CI / browser-tests (pull_request) Successful in 3m52s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-27 18:23:50 +00:00
codingetandCodex 5c7d787b90 test(transport): restore stream regression coverage
CI / lint (pull_request) Successful in 1m39s
CI / format (pull_request) Successful in 1m51s
CI / install (pull_request) Successful in 6m6s
CI / typetest (pull_request) Successful in 1m52s
CI / typecheck (pull_request) Successful in 2m10s
CI / node-tests (pull_request) Successful in 2m34s
CI / browser-tests (pull_request) Successful in 3m34s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 23:24:18 +00:00
codingetandCodex c218117597 refactor(transport): group operation helpers
CI / format (pull_request) Successful in 1m26s
CI / lint (pull_request) Successful in 1m36s
CI / install (pull_request) Successful in 6m7s
CI / typetest (pull_request) Successful in 1m29s
CI / typecheck (pull_request) Successful in 1m59s
CI / node-tests (pull_request) Successful in 2m16s
CI / browser-tests (pull_request) Successful in 3m19s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex 3aad6182b2 refactor: remove transport helper aliases
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex 33d9d115cf docs(transport): clarify pump cleanup timing
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex ccc7496ca1 test(transport): cover wrapper cancellation contract
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex 304961a839 fix(transport): tighten cancellation helpers
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex 048cea10f3 feat(transport): add cancellable stream I/O
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 18:06:52 +00:00
codingetandCodex 999acf82e9 ci: serialize Turbo test tasks
CI / format (pull_request) Successful in 1m21s
CI / lint (pull_request) Successful in 1m21s
CI / install (pull_request) Successful in 6m6s
CI / typetest (pull_request) Successful in 1m37s
CI / typecheck (pull_request) Successful in 1m57s
CI / node-tests (pull_request) Successful in 2m16s
CI / browser-tests (pull_request) Successful in 3m17s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-26 16:24:16 +00:00
codingetandClaude b8a12b45b8 docs(agents): distinguish authorship from review in commit trailers
CI / install (pull_request) Successful in 6m5s
CI / lint (pull_request) Successful in 1m21s
CI / format (pull_request) Successful in 1m21s
CI / typecheck (pull_request) Successful in 2m13s
CI / typetest (pull_request) Successful in 1m26s
CI / browser-tests (pull_request) Successful in 3m18s
CI / node-tests (pull_request) Successful in 47s
Co-Authored-By trailers and Agent/* labels have so far only specified
which model to credit, not whether the agent authored the work at
all. A share of this repo is hand-written by the maintainer with an
agent reviewing after the fact, and that distinction was previously
only tracked as agent memory, not repo-visible instruction.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-26 00:00:18 +00:00
codingetandClaude 34bbe46fc9 fix(ssh): drop the per-session cumulative request counters
CI / format (pull_request) Successful in 1m17s
CI / lint (pull_request) Successful in 1m17s
CI / install (pull_request) Successful in 5m46s
CI / typetest (pull_request) Successful in 1m15s
CI / node-tests (pull_request) Successful in 1m51s
CI / typecheck (pull_request) Successful in 1m53s
CI / browser-tests (pull_request) Successful in 2m54s
Both loops counted every request a session had ever received and aborted
the channel past 4096. That was the right shape while the queue behind
them was unbounded; now that Channel bounds the backlog at its source, a
lifetime ceiling guards nothing and costs something. window-change is
counted, and a client sends one per terminal resize, so a long-lived
interactive shell could be killed for behaving normally.

Removing them leaves the peer bounded by the backlog cap only if the loop
is uniformly serial, so windowChange and signal are now awaited. They were
typed `=> void` and called bare, and void-return assignability let an
async handler through to run unawaited.

Removes SessionLimits.maxRequests, which is a breaking change.

Closes #231.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 17:38:05 +00:00
codingetandClaude 43385dc940 fix(ssh): refuse over-quota global requests instead of dropping the peer
CI / format (pull_request) Successful in 1m15s
CI / lint (pull_request) Successful in 1m17s
CI / install (pull_request) Successful in 5m52s
CI / typetest (pull_request) Successful in 1m25s
CI / node-tests (pull_request) Successful in 1m49s
CI / typecheck (pull_request) Successful in 1m54s
CI / browser-tests (pull_request) Successful in 2m50s
Dropping the connection was the wrong answer to a bound that can be kept.
SSH_MSG_REQUEST_FAILURE carries no payload, so a refusal is fully
re-derivable at the point it is written, and refusals that are still
adjacent in the queue collapse into one entry holding a count. Admitting
a real request starts a fresh entry, which preserves the reply order RFC
4254 §4 requires and keeps the queue bounded: refusal entries alternate
with slot-limited requests, so they cannot outnumber them by more than one.

A refused request that wanted no reply is dropped outright, since nothing
is owed for it. Only a peer that piles up more refusals than it could be
owed replies for still loses the connection.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 11:10:24 +00:00
codingetandClaude db2ba0f768 test(ssh): wait for the abort to be recorded, not for the closed flag
CI / format (pull_request) Successful in 1m14s
CI / lint (pull_request) Successful in 1m23s
CI / install (pull_request) Successful in 5m50s
CI / typetest (pull_request) Successful in 1m26s
CI / node-tests (pull_request) Successful in 1m49s
CI / typecheck (pull_request) Successful in 1m51s
CI / browser-tests (pull_request) Successful in 2m51s
abort() marks the channel closed before its CHANNEL_CLOSE has gone out and
the failure has been stored, so a poll on closed can win that race and read
a null failure. CI hit it; the local run had been passing by luck.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 01:22:35 +00:00
codingetandClaude b4886507bb test(sftp): show the in-flight cap stalling the peer, not buffering for it
CI / lint (pull_request) Successful in 1m9s
CI / format (pull_request) Successful in 1m9s
CI / install (pull_request) Successful in 5m40s
CI / typetest (pull_request) Successful in 59s
CI / node-tests (pull_request) Failing after 1m49s
CI / typecheck (pull_request) Successful in 1m51s
CI / browser-tests (pull_request) Successful in 3m3s
The pipelining test only proved requests still complete under the cap. It
did not show the peer being slowed, which is the whole claim: with the cap
at one and every handler parked, the client's send() blocks on the channel
window and finishes once the handlers are released. Raising the cap to 64
makes the test fail, so the stall comes from the cap rather than the gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 01:13:09 +00:00
codingetandClaude 714837ad24 fix(ssh): let close() answer queued sessions before the transport goes
CI / lint (pull_request) Successful in 59s
CI / format (pull_request) Successful in 1m3s
CI / install (pull_request) Successful in 5m4s
CI / typetest (pull_request) Successful in 1m0s
CI / typecheck (pull_request) Successful in 1m34s
CI / node-tests (pull_request) Successful in 1m33s
CI / browser-tests (pull_request) Successful in 2m31s
The queued-session drain ran after mux.close() had already torn the
transport down, so the CHANNEL_CLOSE it sends never reached the peer and
the test asserting the drain passed with the drain removed. Await the
aborts, and assert the client sees a clean protocol close rather than a
channel that merely ended up closed.

Make the channel-request backlog cap configurable alongside every other
limit. Driving 4096 encrypted requests through loopback timed out on CI;
a test that sets the bound to 8 does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-24 00:02:19 +00:00
codingetandClaude b1e746bfb7 fix(ssh,sftp): bound peer-controlled work in the server read loops
CI / format (pull_request) Successful in 56s
CI / lint (pull_request) Successful in 58s
CI / install (pull_request) Successful in 4m42s
CI / typetest (pull_request) Successful in 1m9s
CI / node-tests (pull_request) Failing after 1m21s
CI / typecheck (pull_request) Successful in 1m29s
CI / browser-tests (pull_request) Successful in 2m31s
PR #160 capped channel requests in one place. Every other queue and
unawaited dispatch loop a peer feeds was unbounded.

SSH: cap the per-channel request queue, the mux accept queue, unaccepted
sessions, and live direct-tcpip forwards; drop a connection whose peer
exceeds the global-request quota, since RFC 4254 §4 forbids answering it
ahead of the requests still running. Drain the session queue on failure
and on close, and clear the request queue when a channel fails.

SFTP: throttle the dispatch loop instead of failing it, since pipelining
is normal client behaviour; hold each slot until the reply is sent so the
send chain cannot grow in its place. Clamp SSH_FXP_READ to a reply size
the server is willing to allocate, and cap open handles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 23:52:53 +00:00
codingetandClaude 45e9e9b203 docs(http): scope the one-framing-header guarantee to bodied messages
CI / lint (pull_request) Successful in 1m0s
CI / format (pull_request) Successful in 1m0s
CI / install (pull_request) Successful in 4m43s
CI / typetest (pull_request) Successful in 1m0s
CI / typecheck (pull_request) Successful in 1m16s
CI / node-tests (pull_request) Successful in 1m21s
CI / browser-tests (pull_request) Successful in 2m17s
formatBody decides nothing for a status that forbids a body, so a framing
header a handler set on a 1xx, 204, 205 or 304 survives untouched. Say so
rather than claim more than the code does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 23:28:22 +00:00
codingetandClaude 5262b8c2da docs(http): describe the framing and injection rules
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 23:28:22 +00:00
codingetandClaude da3df17fbb fix(http): reject ambiguous framing and header injection
@webnet/http decided request framing from headers it never validated and
wrote headers it never sanitised. Both are the peer's to exploit, and the
server is the exposed side.

On the way in, bodyReader gave a peer three ways to make us frame a
message differently from a proxy in front of us: send both
Transfer-Encoding and Content-Length and have Transfer-Encoding silently
win; write `Chunked` or repeat the field, so the case-sensitive exact
match fell through to Content-Length; or write `Content-Length : 5`,
which readHeaders turned into the distinct name `"content-length "` and
ignored. A Content-Length of `0x10` or `1e3` parsed to a length no
decimal reader would agree with.

Each of those is refused now: one framing header, `chunked` matched
case-insensitively and only as the whole field, field names checked as
RFC 9110 tokens, values checked for control characters, and a bare
decimal Content-Length. The client reading a response gets the same
treatment, since a malicious response frames as badly as a request.

HttpProtocolError carries the status a server should answer with, and
ServerConnection.handle sends it and closes rather than dropping the
connection without a word. That also covers the existing limits, which
answer 413, 414 or 431 instead of going silent.

On the way out, writeHeaders interpolated names and values straight into
the wire format, so a CR or LF in either ended the line early and handed
the rest to the peer as headers of its own, or as a whole second message.
This is reachable: http-static and webdav both set Content-Type and ETag
from VFS stat metadata, and serving content someone else supplied is the
point of tailshare. @webnet/ftp already guards its equivalent with
assertNoCrlf. The check goes in writeHeaders itself, so no call site can
forget it, and it throws, since unlike FTP's close reason there is no
path here that cannot handle one.

formatBody also emitted both framing headers of its own accord: a caller
that set Content-Length and then handed over a stream got
Transfer-Encoding on top, which is the same ambiguity we now refuse to
read. Exactly one survives. A caller-declared length is kept rather than
replaced, so a peer can still show progress, and sendBody holds the
stream to exactly that many bytes instead of letting a short or long body
desync the connection. A Content-Length that cannot frame anything is
dropped in favour of chunked. Removing a header needs
MutableHeaders.delete and WritableHttp.removeHeader, which are new.

Closes #222

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 23:28:22 +00:00
codingetandClaude 002a46857d fix(http): stop a body stream detaching the read buffer
Enqueueing a chunk on a byte stream detaches its ArrayBuffer, and the
body readers hand out views onto the read buffer's own storage rather
than copies. Reading a request body through req.stream() therefore left
ReadBuffer pointing at detached memory as soon as the body spanned more
than one transport chunk: its length went negative and the next read
threw, so the server answered 500 to an ordinary upload.

Only our own client kept this quiet, by always framing bodies as chunked;
any peer that sends Content-Length hits it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 23:28:22 +00:00
codingetandCodex 58b9b6e6ab fix(ftp): pin data dials to control peer
CI / install (pull_request) Successful in 7m33s
CI / lint (pull_request) Successful in 2m48s
CI / format (pull_request) Successful in 2m57s
CI / typecheck (pull_request) Successful in 3m28s
CI / typetest (pull_request) Successful in 2m57s
CI / browser-tests (pull_request) Successful in 5m24s
CI / node-tests (pull_request) Successful in 1m9s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-23 03:07:06 +00:00
codingetandCodex b414101f4d fix(ftp): reject unsafe passive transfers
CI / format (pull_request) Successful in 3m8s
CI / lint (pull_request) Successful in 3m11s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 5m24s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-23 03:01:48 +00:00
codingetandClaude dafdbe889c test(websocket): pin the close-then-violation path
CI / format (pull_request) Successful in 2m41s
CI / lint (pull_request) Successful in 2m45s
CI / install (pull_request) Successful in 7m34s
CI / typetest (pull_request) Successful in 2m45s
CI / typecheck (pull_request) Successful in 2m53s
CI / node-tests (pull_request) Successful in 3m5s
CI / browser-tests (pull_request) Successful in 4m40s
A violation read after close() was already sent leaves the peer with the
status of the first Close frame, since a side sends only one, while the
caller still sees the error. Nothing covered that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:19:39 +00:00
codingetandClaude 214affb610 fix(websocket): fail the read on a protocol violation
CI / format (pull_request) Successful in 3m2s
CI / lint (pull_request) Successful in 3m4s
CI / install (pull_request) Successful in 7m33s
CI / typetest (pull_request) Successful in 2m51s
CI / node-tests (pull_request) Successful in 3m10s
CI / typecheck (pull_request) Successful in 3m20s
CI / browser-tests (pull_request) Successful in 5m1s
Sending the peer a Close frame and then ending the iteration made a
violation indistinguishable from an orderly close. The iterator now closes
the connection as before and then throws WebSocketProtocolError, carrying
the same status code it sent. A peer that closes normally still ends the
iteration without an error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:02:49 +00:00
codingetandClaude 6fba0cdcba fix(websocket): bound frames and messages, close two RFC 6455 gaps
CI / lint (pull_request) Successful in 2m34s
CI / format (pull_request) Successful in 3m15s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 7m19s
readFrame buffered whatever payload length a peer announced, and
continuation frames accumulated with no cap on their count or on the
assembled size, so a peer could make a connection allocate without bound.
It was the only protocol package without such a limit. maxFrameSize,
maxMessageSize and maxFragments now bound all three and are configurable
through the client and server options; exceeding one closes the connection
with 1009.

Two conformance gaps sat in the same function. The 64-bit extended length
was accumulated with arithmetic that silently loses precision above 2^53
and never checked that its most significant bit is zero (§5.2). Masking
was parsed but never enforced (§5.1): a server accepted unmasked client
frames and a client accepted masked server frames. Both now fail the
connection with 1002.

Closes #200

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 23:55:32 +00:00
codingetandClaude b534fe21ea docs(http): correct the count of changed server defaults
CI / install (pull_request) Successful in 20m13s
CI / lint (pull_request) Successful in 6m45s
CI / format (pull_request) Successful in 6m37s
CI / typecheck (pull_request) Successful in 3m46s
CI / typetest (pull_request) Successful in 3m51s
CI / node-tests (pull_request) Successful in 3m57s
CI / browser-tests (pull_request) Successful in 5m54s
Five limits changed from Infinity, not four, and the inherited option
types now say so where the fields are declared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:14:38 +00:00
codingetandClaude 6750609a56 fix(http): finite server limits and close rejected connections
CI / format (pull_request) Failing after 3m18s
CI / lint (pull_request) Failing after 3m19s
CI / install (pull_request) Failing after 3m20s
CI / typecheck (pull_request) Skipped
CI / typetest (pull_request) Skipped
CI / node-tests (pull_request) Skipped
CI / browser-tests (pull_request) Skipped
onConnect returning false is documented as closing the connection, but
Server.listen only skipped handling it and leaked the transport.

Every server limit except maxHeaderCount, maxTargetLength and
maxLineLength defaulted to Infinity, so a default-constructed server had
no header timeout, no idle timeout and no body size limit. Timeout
watchdogs are unref'd so those defaults cannot pin a Node process open.

Closes #201

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:02:36 +00:00
codingetandCodex d946eca671 feat(binary): split codecs from browser utils
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-20 19:56:53 +02:00
codingetandCodex 6499f202ad fix(tooling): typecheck support workspaces
CI / install (pull_request) Successful in 8m10s
CI / lint (pull_request) Successful in 3m7s
CI / format (pull_request) Successful in 3m8s
CI / typecheck (pull_request) Successful in 3m39s
CI / typetest (pull_request) Successful in 3m22s
CI / browser-tests (pull_request) Successful in 5m49s
CI / node-tests (pull_request) Successful in 1m22s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-20 17:12:54 +00:00
codingetandCodex e15718a012 fix(http): reject truncated response bodies
CI / format (pull_request) Successful in 2m53s
CI / lint (pull_request) Successful in 2m57s
CI / install (pull_request) Successful in 8m7s
CI / typetest (pull_request) Successful in 2m59s
CI / node-tests (pull_request) Successful in 3m17s
CI / typecheck (pull_request) Successful in 3m26s
CI / browser-tests (pull_request) Successful in 5m17s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-18 23:13:06 +00:00
codingetandCodex 2798322a0a test(transport): separate WebRTC close from cleanup
CI / lint (pull_request) Successful in 2m45s
CI / format (pull_request) Successful in 2m45s
CI / install (pull_request) Successful in 7m45s
CI / typetest (pull_request) Successful in 2m50s
CI / typecheck (pull_request) Successful in 3m3s
CI / node-tests (pull_request) Successful in 3m13s
CI / browser-tests (pull_request) Successful in 5m6s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-17 02:12:35 +00:00
codingetandCodex 6a84570c63 test(transport): update browser EOF assertion
CI / lint (pull_request) Successful in 2m31s
CI / format (pull_request) Successful in 2m35s
CI / install (pull_request) Successful in 8m55s
CI / typetest (pull_request) Successful in 2m13s
CI / node-tests (pull_request) Successful in 3m6s
CI / typecheck (pull_request) Successful in 3m8s
CI / browser-tests (pull_request) Failing after 4m25s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-17 01:56:10 +00:00
codingetandCodex ba9983de32 feat(transport): signal end of stream explicitly
CI / lint (pull_request) Successful in 2m10s
CI / format (pull_request) Successful in 2m25s
CI / install (pull_request) Successful in 7m4s
CI / typetest (pull_request) Successful in 2m23s
CI / typecheck (pull_request) Successful in 2m41s
CI / node-tests (pull_request) Successful in 2m51s
CI / browser-tests (pull_request) Failing after 4m19s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-17 01:42:34 +00:00
codingetandCodex 493eef3b7b docs: correct package guidance and label ownership
CI / lint (pull_request) Successful in 4m19s
CI / format (pull_request) Successful in 4m2s
CI / install (pull_request) Successful in 10m5s
CI / typetest (pull_request) Successful in 3m11s
CI / typecheck (pull_request) Successful in 3m18s
CI / node-tests (pull_request) Successful in 3m25s
CI / browser-tests (pull_request) Successful in 5m24s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-15 02:54:37 +00:00
codingetandClaude 790a795d5a docs(agents): allow both provenance labels on co-authored work
CI / lint (pull_request) Successful in 3m58s
CI / format (pull_request) Successful in 4m29s
CI / install (pull_request) Successful in 9m18s
CI / typetest (pull_request) Successful in 2m58s
CI / node-tests (pull_request) Successful in 3m3s
CI / typecheck (pull_request) Successful in 3m11s
CI / browser-tests (pull_request) Successful in 5m3s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 02:22:37 +00:00
codingetandClaude f659f230cd docs: add per-package READMEs and position Tailshare as the flagship app
CI / lint (pull_request) Successful in 3m39s
CI / format (pull_request) Successful in 3m24s
CI / install (pull_request) Successful in 9m4s
CI / typetest (pull_request) Successful in 3m5s
CI / typecheck (pull_request) Successful in 3m20s
CI / node-tests (pull_request) Successful in 3m31s
CI / browser-tests (pull_request) Successful in 5m16s
Give every workspace a README describing what it does, its entry points,
and a short usage example, and link them from the root package table.

Frame Tailshare in the root README as the application the libraries exist
to be composed into, with the other apps as development and testing
surfaces. Its own README records what works today and what the open
issues intend for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 23:51:59 +00:00
codingetandCodex 2cfb14eb29 refactor: rename drive package to webdav
CI / lint (pull_request) Successful in 3m59s
CI / format (pull_request) Successful in 4m18s
CI / install (pull_request) Successful in 9m3s
CI / typetest (pull_request) Successful in 3m14s
CI / typecheck (pull_request) Successful in 3m19s
CI / node-tests (pull_request) Successful in 3m35s
CI / browser-tests (pull_request) Successful in 5m35s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-14 23:08:01 +00:00
codingetandCodex 0947533a4a docs: refresh package and label guidance
CI / lint (pull_request) Successful in 4m29s
CI / format (pull_request) Successful in 4m30s
CI / install (pull_request) Successful in 9m25s
CI / node-tests (pull_request) Successful in 2m58s
CI / typecheck (pull_request) Successful in 3m17s
CI / typetest (pull_request) Successful in 3m17s
CI / browser-tests (pull_request) Successful in 5m14s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-14 22:40:06 +00:00
codingetandCodex 0a25ed2e7c docs: replace AI changes log with disclosure
CI / install (pull_request) Successful in 8m50s
CI / lint (pull_request) Successful in 3m13s
CI / format (pull_request) Successful in 3m0s
CI / typetest (pull_request) Failing after 5m49s
CI / browser-tests (pull_request) Failing after 29m30s
CI / node-tests (pull_request) Failing after 29m30s
CI / typecheck (pull_request) Failing after 29m30s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-14 20:48:23 +00:00
codingetandCodex 6d8b38eb79 fix(tsconnect-worker): bound worker lifecycle replies
CI / lint (pull_request) Successful in 2m54s
CI / format (pull_request) Successful in 3m8s
CI / install (pull_request) Successful in 7m30s
CI / typetest (pull_request) Successful in 2m34s
CI / typecheck (pull_request) Successful in 2m56s
CI / node-tests (pull_request) Successful in 3m7s
CI / browser-tests (pull_request) Successful in 4m46s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-09 00:50:07 +00:00
codingetandCodex 1c344eff74 fix(tsconnect-worker): allow slow disconnect cleanup
CI / lint (pull_request) Successful in 3m3s
CI / format (pull_request) Successful in 3m12s
CI / install (pull_request) Successful in 7m49s
CI / typetest (pull_request) Successful in 2m47s
CI / node-tests (pull_request) Successful in 3m7s
CI / typecheck (pull_request) Successful in 3m7s
CI / browser-tests (pull_request) Successful in 4m47s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-07 23:59:56 +00:00
codingetandCodex cb236406a0 fix(tsconnect-worker): harden client disconnect
CI / lint (pull_request) Successful in 2m57s
CI / format (pull_request) Successful in 3m9s
CI / install (pull_request) Successful in 7m30s
CI / typetest (pull_request) Successful in 2m36s
CI / typecheck (pull_request) Successful in 2m54s
CI / node-tests (pull_request) Successful in 3m0s
CI / browser-tests (pull_request) Successful in 4m48s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-07 23:37:10 +00:00
codingetandCodex 6dd8bf378f fix(tsconnect-worker): detect configuration mismatches
CI / lint (pull_request) Successful in 3m59s
CI / format (pull_request) Successful in 3m54s
CI / install (pull_request) Successful in 8m32s
CI / typetest (pull_request) Successful in 2m50s
CI / node-tests (pull_request) Successful in 3m17s
CI / typecheck (pull_request) Successful in 3m19s
CI / browser-tests (pull_request) Successful in 5m5s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-07 22:32:48 +00:00
codingetandClaude 136fb457a0 feat(ssh,sftp): route subsystems and share one SSH connection
CI / format (pull_request) Successful in 2m59s
CI / lint (pull_request) Successful in 3m15s
CI / install (pull_request) Successful in 7m24s
CI / typetest (pull_request) Successful in 2m43s
CI / typecheck (pull_request) Successful in 3m14s
CI / node-tests (pull_request) Successful in 3m13s
CI / browser-tests (pull_request) Successful in 4m39s
Closes #108.

serveSession() no longer rejects: a failing handler is reported through an
optional onError hook instead, so the documented fire-and-forget call
pattern cannot take down every other session on the connection.

Add a subsystem handler so one accept loop can dispatch exec, shell, and
subsystems. On that shape, sftp gains SFTPClient.connect/fromChannel over a
caller-owned connection or channel, and serveSFTPChannel() serving an
already-accepted channel with an already-authorized VFS. An SFTPClient now
closes only the channel it uses and only a connection it dialed itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 03:01:17 +00:00
codingetandClaude d66067f9fb docs: record the ssh session review round
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 03:00:13 +00:00
codingetandClaude dd4fcdd5b7 fix(ssh): contain malformed session requests from a peer
A payload too short for its fields makes the reader throw. Neither request
loop is awaited by its caller, so the throw escaped as an unhandled
rejection — by default a process exit any authenticated peer could trigger
with a single short pty-req, env, signal, exec, or exit-status.

Guard both loops, so a malformed request fails only that request. Also
register the abort listener before session negotiation, so a signal can
cancel a start request the peer never answers; skip exit-status on an
already-closed channel; stop double-counting the env budget on a repeated
name; and bound peer requests on the client as the server already does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 03:00:13 +00:00
codingetandClaude c06289a262 feat(ssh): add exec, shell, and PTY session support
Closes #103.

Deliver stderr on its own queue, generalise channel requests, and answer a
peer's CHANNEL_CLOSE as RFC 4254 §5.3 requires. On top of that, add client
exec/openShell/run and a server-side serveSession() handler model with
explicit approval hooks and bounded env, PTY, and request limits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 03:00:13 +00:00
codingetandClaude e9b90a2c64 docs: record the optional-operation call site audit
CI / lint (pull_request) Successful in 2m57s
CI / format (pull_request) Successful in 3m14s
CI / install (pull_request) Successful in 7m40s
CI / typetest (pull_request) Successful in 2m35s
CI / typecheck (pull_request) Successful in 2m52s
CI / node-tests (pull_request) Successful in 2m57s
CI / browser-tests (pull_request) Successful in 4m35s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude 90bbc2fb2f test(drive,http-static,ftp,sftp): cover a runtime unsupported optional operation
Every call site the audit touched had coverage for an optional operation being
absent and none for one that is present and rejects with unsupported for a
path, which is the case the policies decide. Each assertion is on the bytes or
the protocol status, not on the call having returned.

Refs #178, #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude d51a33cae0 docs(smb2,ftp,sftp): record why these keep their own traversal
The audit's other outcome. SMB2 copy is the implementation of the optional
operation rather than a caller, and needs a case-insensitive overlap check the
shared one does not do. The FTP and SFTP clients' recursive delete implements a
required operation out of protocol verbs chosen from a stat they already hold;
a shared deleteRecursive shim would restat every entry and still could not tell
a symlink from its target, which is what decides whether descending is right.

Refs #178

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude 6d95aab10d refactor(sftp,ftp): decide the rename policy in one place
Both servers refused RENAME and RNFR/RNTO on method presence, then called move
and let the session's error mapping handle whatever it threw. That already gave
a runtime unsupported the same SSH_FX_OP_UNSUPPORTED and 502 the presence check
gives, so this changes no behaviour; it replaces the two paths with moveFallback
under the native-only policy, which states the decision #179 asks for rather
than leaving it implied by an early return.

native-only rather than a shim: rename is expected to be cheap and roughly
atomic, and copy-then-delete would turn a rename of a large directory into a
recursive transfer the peer has no way to decline.

Refs #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude a121d03257 fix(drive): answer a runtime unsupported PROPPATCH the way an absent one is
A backend with setProps that rejects unsupported for this path got 501 while a
backend without setProps got 403, for the same inability. 403 is the answer
kept: WebDAV allows it for a property the server will not store, where 501 says
PROPPATCH itself is unimplemented, which is not what the server means.

Refs #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude 0b02dcede0 refactor(drive): copy and move through the shared fallbacks
copyRecursive predated @webnet/vfs/fallback and disagreed with it: it did not
reject an overlapping source and destination, left a partial destination behind
on failure, and tolerated an existing directory mid-tree rather than replacing
the destination.

The policy is on-unsupported, so a native copy or move that rejects unsupported
for this path falls back to the traversal. That code guarantees nothing changed,
and WebDAV has no way to tell the client which of the two it got, so refusing
would only lose it the operation.

The destination is now statted twice per request, once by the handler to answer
201 or 204 and once by the fallback to decide whether to clear it.

Refs #178, #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude 1c75e2de6f fix(sftp,ftp): window an offset read lazily instead of discarding eagerly
The SFTP server drained the prefix inside openReadStream, before the reader was
handed back, holding the SSH_FXP_READ reply open for the whole discard. FTP had
its own skipBytes for the same job. Both use readFileRangeFallback now, which
windows lazily and cancels the source once the window is filled, and skipBytes
goes with its only caller.

Refs #178, #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandClaude 12828c4f39 fix(http-static,drive): serve a Range through the shared fallback
http-static had its own boundedStream for the same job readFileRangeFallback
does. Both servers now use the shared one with the on-unsupported policy: a
Range a backend could serve is not refused, so a native readFileRange that
rejects unsupported for this path falls back like an absent one does.

Refs #178, #179

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:41:41 +00:00
codingetandCodex 802ea449dd fix(tailshare): address initialization failure review
CI / lint (pull_request) Successful in 3m7s
CI / format (pull_request) Successful in 3m20s
CI / install (pull_request) Successful in 7m30s
CI / typetest (pull_request) Successful in 2m39s
CI / node-tests (pull_request) Successful in 2m56s
CI / typecheck (pull_request) Successful in 2m58s
CI / browser-tests (pull_request) Successful in 4m37s
Co-Authored-By: gpt-5.6-terra <noreply@openai.com>
2026-08-05 23:59:16 +00:00
codingetandCodex 9c9db0c17a fix(tailshare): handle insecure and failed initialization
Closes #123

Closes #131

Co-Authored-By: gpt-5.6-terra <noreply@openai.com>
2026-08-05 23:57:30 +00:00
codinget ced0e59fba chore(vfs): statically ensure MemoryVFS implements every method
CI / lint (pull_request) Successful in 3m2s
CI / format (pull_request) Successful in 3m23s
CI / install (pull_request) Successful in 7m34s
CI / typetest (pull_request) Successful in 2m32s
CI / typecheck (pull_request) Successful in 2m50s
CI / node-tests (pull_request) Successful in 2m55s
CI / browser-tests (pull_request) Successful in 4m35s
2026-08-05 22:17:47 +00:00
codingetandClaude 529e83223e test(drive): declare PROPPATCH over a minimal filesystem unsupported
Opting out skipped the test entirely, leaving the path with no coverage. The
tri-state capability with a forbidden alias keeps the assertions that the call
rejects and the properties are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:33 +00:00
codingetandClaude 132c542b3f docs: record the minimal-filesystem conformance work
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:33 +00:00
codingetandClaude e83a37da6c test(drive,ftp,sftp): run conformance over a minimal backing filesystem
Each package ran the suite against a server backed by a full MemoryVFS, so
the servers' fallback paths were never entered. A second run reduces the
server's filesystem to the required operations while leaving the client under
test unchanged; it fails on #177 without the previous commit. Where a server
answers rather than shims — WebDAV 403 for PROPPATCH, FTP 502 and SFTP
SSH_FX_OP_UNSUPPORTED for rename — the run declares that instead of opting
out, which is the decision #179 revisits.

Fixes #181

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:33 +00:00
codingetandClaude 1458423679 fix(drive): serve the requested window on a Range without readFileRange
The fallback branch discarded range.start and streamed the file from byte
zero while the response advertised the requested window, so a client asking
for bytes 5-9 got bytes 0-4 and stored them at the wrong offset.
readFileRangeFallback windows the stream and calls the native operation when
there is one.

Fixes #177

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:33 +00:00
codingetandClaude da7c3d3e21 feat(vfs): share the optional-stripping helpers in conformance
withoutOptional and unsupportedOptional replace the local copies in the
fallback tests; the protocol packages need the same reduction to put their
servers' fallback paths under the suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:33 +00:00
codingetandCodex 6c66acb882 fix(vfs): reject overlapping copies
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-05 21:29:33 +00:00
codingetandClaude abd605a2ab docs: record the statAndReaddir policy and the issues filed
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 21:29:23 +00:00
codingetandClaude 016e506393 fix(vfs): let the statAndReaddir callsites fall back on unsupported
Every one of these callers needs the listing to answer the request, and the
stat plus readdir it falls back to are required operations, so there is no
reason to stop at an implementation that has the combined call but cannot serve
it for this path. The default policy stops there because for most operations a
fallback is materially different from the native one; for this one it is only
slower.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 01:44:37 +00:00
codingetandClaude b72ee0c9f1 test(vfs): cover the review's untested boundaries
CI / lint (pull_request) Successful in 2m38s
CI / format (pull_request) Successful in 2m43s
CI / install (pull_request) Successful in 8m55s
CI / typetest (pull_request) Successful in 2m40s
CI / typecheck (pull_request) Successful in 2m58s
CI / node-tests (pull_request) Successful in 3m9s
CI / browser-tests (pull_request) Successful in 4m52s
A stream that fails after the native call resolved is deliberately not retried,
and cancelling the fallback's output while a read is in flight must still
release the source. Both were argued from the code rather than demonstrated.
The root path overlaps every other path, so it is rejected at either end of a
copy; that followed from the guard without being stated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 00:34:14 +00:00
codingetandClaude 4387ea9837 docs: record the vfs fallback work
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / format (pull_request) Canceled after 1m58s
CI / lint (pull_request) Canceled after 2m4s
CI / install (pull_request) Canceled after 2m10s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 00:13:37 +00:00
codingetandClaude cf485af184 test(vfs): cover shimmed and unsupported conformance variants
An implementation that answers unsupported at runtime was not expressible: the
capability flags only said present or absent, so the suite could not check that
such a rejection carries the right code and leaves the filesystem alone.

The fallbacks are then run against the same suite with the optional operations
first removed and then rejecting, which is what shows they can stand in for
them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 00:12:56 +00:00
codingetandClaude a81437430c fix(vfs): honour empty ranges in the drive and ftp clients
Both clients failed the boundary behaviour the conformance suite now requires.
WebDAV answers a range at or past the end of a file with 416, which RFC 9110
defines as unsatisfiable and the client reported as an unexpected status; at
this level the window is simply empty. FTP carries no end bound on REST, so an
end below the start transferred from the start to the end of the file instead
of nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-05 00:12:43 +00:00
codingetandClaude c8352a319e test(vfs): pin readFileRange boundary behaviour
Serving a byte range needs an empty window and an unanswerable request to be
distinguishable, so a start at or past the end of a file and an end below the
start return an empty stream rather than failing. The suite did not say so, and
NodeVFS passed both bounds to createReadStream, which rejects an inverted range
with a plain RangeError.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 20:00:38 +00:00
codingetandClaude 1660b1df2e refactor(vfs): share the statAndReaddir fallback
The optional operation arrived with three byte-identical copies of the same
fallback, one per call site.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 19:59:04 +00:00
codingetandClaude 4c17366172 feat(vfs): add fallbacks for optional operations
Callers of an optional AsyncVFS operation each had to decide what to do when
an implementation lacked it, so the same fallbacks were written repeatedly and
diverged. withFallbacks presents every optional operation as available, so a
consumer can call it without branching on method presence, and the operations
that cannot be built from the required ones say so with the unsupported code
rather than appearing absent.

The policy chooses when a fallback may stand in. Only an unsupported rejection
ever triggers one: any other failure means the operation was attempted, and
retrying it another way would hide that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 19:57:24 +00:00
codingetandClaude 3270e373a0 feat(vfs): map native unsupported statuses onto the vfs code
Each client already had a native response meaning the operation does not exist
here, and each discarded it: SFTP OP_UNSUPPORTED and FTP 502 surfaced as bare
protocol errors, WebDAV 501 hit the unexpected-status branch, and SMB2
STATUS_NOT_SUPPORTED reached callers as forbidden, which claims a permission
problem the server never reported.

Without this a server emits the status and its own client throws the meaning
away, so no caller can act on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 19:53:37 +00:00
codingetandClaude 9ae97a37ad feat(vfs): add an unsupported error code
Optional AsyncVFS operations could only be declared by method presence, which
is global to an instance and cannot express a backend that performs an
operation for some shares, paths or negotiated protocol versions and not
others. Those cases were forced onto forbidden, which claims the caller lacks
permission when the backend said the operation does not exist here.

The outbound wire mappings are exhaustive over VFSErrorCode, so each one names
its protocol's own unsupported response.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 19:50:48 +00:00
codingetandCodex 394d720aeb docs(vfs): qualify overwrite replacement
CI / lint (pull_request) Successful in 2m0s
CI / format (pull_request) Successful in 2m6s
CI / install (pull_request) Successful in 7m14s
CI / typetest (pull_request) Successful in 2m5s
CI / node-tests (pull_request) Successful in 2m33s
CI / typecheck (pull_request) Successful in 2m35s
CI / browser-tests (pull_request) Successful in 6m52s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 14:39:04 +00:00
codingetandCodex c9ff59bd9a docs(vfs): define overwrite replacement
CI / format (pull_request) Successful in 2m0s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / lint (pull_request) Canceled after 2m12s
CI / install (pull_request) Canceled after 2m28s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 14:36:59 +00:00
codingetandCodex e00d55d2af fix(vfs): preserve overwrite destinations
CI / lint (pull_request) Successful in 2m8s
CI / format (pull_request) Successful in 2m8s
CI / install (pull_request) Successful in 6m52s
CI / typetest (pull_request) Successful in 2m12s
CI / typecheck (pull_request) Successful in 2m43s
CI / node-tests (pull_request) Successful in 2m43s
CI / browser-tests (pull_request) Successful in 4m34s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 13:50:22 +00:00
codingetandCodex a82ddab3a2 fix(drive): preserve overwrite destination
CI / lint (pull_request) Successful in 2m8s
CI / format (pull_request) Successful in 2m11s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 6m25s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 13:44:07 +00:00
codingetandCodex e1dc165e03 fix(vfs): address combined metadata review
CI / lint (pull_request) Successful in 2m34s
CI / format (pull_request) Successful in 2m37s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 5m5s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 13:39:10 +00:00
codingetandCodex 93b2dff943 refactor(vfs): reuse combined directory metadata
CI / format (pull_request) Successful in 2m21s
CI / lint (pull_request) Successful in 2m35s
CI / install (pull_request) Successful in 9m0s
CI / typetest (pull_request) Successful in 2m35s
CI / node-tests (pull_request) Successful in 3m4s
CI / typecheck (pull_request) Successful in 3m13s
CI / browser-tests (pull_request) Successful in 4m46s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 00:51:42 +00:00
codingetandCodex e21ad5f24f feat(vfs): combine stat and directory listing
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-04 00:49:30 +00:00
codingetandCodex dc613b5dfe refactor(taildrive): clarify discovery exports
CI / lint (pull_request) Successful in 2m47s
CI / format (pull_request) Successful in 3m0s
CI / install (pull_request) Successful in 9m49s
CI / typetest (pull_request) Successful in 2m41s
CI / typecheck (pull_request) Successful in 2m59s
CI / node-tests (pull_request) Successful in 3m13s
CI / browser-tests (pull_request) Successful in 5m0s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-03 23:08:54 +00:00
codingetandCodex a60f1b626e fix(taildrive): harden peer probes
CI / format (pull_request) Successful in 1m55s
CI / lint (pull_request) Successful in 2m4s
CI / install (pull_request) Successful in 5m52s
CI / typetest (pull_request) Successful in 2m7s
CI / node-tests (pull_request) Successful in 2m18s
CI / typecheck (pull_request) Successful in 2m20s
CI / browser-tests (pull_request) Successful in 3m47s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-03 01:27:54 +00:00
codingetandCodex a0ad4bf17b feat(taildrive): discover peers with shares
CI / lint (pull_request) Successful in 1m49s
CI / format (pull_request) Successful in 2m0s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 5m12s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-08-03 01:23:10 +00:00
codingetandClaude ab987f50d0 test(tsconnect): assert drive peers are reported online
CI / format (pull_request) Successful in 2m25s
CI / lint (pull_request) Successful in 2m28s
CI / install (pull_request) Successful in 7m35s
CI / typetest (pull_request) Successful in 2m15s
CI / typecheck (pull_request) Successful in 2m31s
CI / node-tests (pull_request) Successful in 2m32s
CI / browser-tests (pull_request) Successful in 3m56s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 01:39:46 +00:00
codingetandClaude a6b71597ca fix(tsconnect): filter offline and unreachable taildrive peers
CI / format (pull_request) Successful in 2m35s
CI / lint (pull_request) Successful in 2m37s
CI / typecheck (pull_request) Canceled after 0s
CI / typetest (pull_request) Canceled after 0s
CI / node-tests (pull_request) Canceled after 0s
CI / browser-tests (pull_request) Canceled after 0s
CI / install (pull_request) Canceled after 5m55s
listDrivePeers returned every peer holding PeerCapabilityTaildriveSharer,
which is an ACL grant usually given to a whole group or tag, so offline
peers and peers with no reachable peerAPI were included. Bump the
tailscale submodule to the fix and correct the TS doc comments, which
claimed the result was peers exposing a share.

The submodule bump also picks up the removal of the obsolete ssh, fetch
and setExitNodeEnabled wasm bridge APIs, already on the fork's webnet
branch and unused here.

Closes #143

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 01:34:07 +00:00
codinget f4131462af build(turbo): fix turbo cache with submodules
CI / format (pull_request) Successful in 2m39s
CI / lint (pull_request) Successful in 2m55s
CI / install (pull_request) Successful in 7m57s
CI / typetest (pull_request) Successful in 2m4s
CI / node-tests (pull_request) Successful in 2m40s
CI / typecheck (pull_request) Successful in 2m44s
CI / browser-tests (pull_request) Successful in 4m13s
2026-08-01 00:19:50 +00:00
webnet-actions f766c853d9 chore(tsconnect): update CA bundle 2026-07-31 21:15:18 +00:00
codingetandCodex 1287baf203 ci: paginate CA updater PR discovery
CI / format (pull_request) Successful in 2m16s
CI / lint (pull_request) Successful in 2m36s
CI / install (pull_request) Successful in 9m51s
CI / typetest (pull_request) Successful in 2m40s
CI / typecheck (pull_request) Successful in 3m5s
CI / node-tests (pull_request) Successful in 3m17s
CI / browser-tests (pull_request) Successful in 4m59s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-07-31 20:52:07 +00:00
codingetandCodex 5ac5f08378 ci: schedule tsconnect CA bundle refresh
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-07-31 20:52:07 +00:00
codingetandCodex d28697eeee fix(tsconnect): publish only used build outputs
CI / lint (pull_request) Successful in 2m49s
CI / format (pull_request) Successful in 2m54s
CI / install (pull_request) Successful in 7m37s
CI / typetest (pull_request) Successful in 2m2s
CI / node-tests (pull_request) Successful in 2m25s
CI / typecheck (pull_request) Successful in 2m35s
CI / browser-tests (pull_request) Successful in 3m55s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-07-31 22:35:18 +02:00
codingetandClaude 962ed20219 docs: drop the duplicated #147 disclosure entry
CI / format (pull_request) Successful in 2m43s
CI / lint (pull_request) Successful in 2m47s
CI / install (pull_request) Successful in 9m49s
CI / typecheck (pull_request) Successful in 2m46s
CI / typetest (pull_request) Successful in 2m43s
CI / node-tests (pull_request) Successful in 2m21s
CI / browser-tests (pull_request) Successful in 4m23s
The union merge driver kept both the pre- and post-rebase versions of the
tsconnect shutdown entry; the older one predates the dead-handle change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 16:49:20 +00:00
codingetandClaude 279733fc01 test: run Go runtime shutdowns in-process again
CI / lint (pull_request) Successful in 3m58s
CI / format (pull_request) Successful in 2m55s
CI / install (pull_request) Successful in 13m11s
CI / typecheck (pull_request) Successful in 3m35s
CI / typetest (pull_request) Successful in 1m30s
CI / node-tests (pull_request) Successful in 1m34s
CI / browser-tests (pull_request) Successful in 3m59s
The child-process probes, the missing shutdown() in the tsconnect
integration teardown, and the *.probe.ts tsconfig excludes all existed
only to keep the late "Go program has already exited" errors away from
node:test. That bug is fixed, so drop the workarounds; the restored
in-process shutdowns are the regression test for it.

Refs #147, #152

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 00:39:07 +00:00
codingetandClaude 342b996828 fix(tsconnect): treat handles from an exited runtime as closed
CI / lint (pull_request) Successful in 2m34s
CI / format (pull_request) Successful in 2m34s
CI / install (pull_request) Successful in 7m2s
CI / typetest (pull_request) Successful in 1m59s
CI / node-tests (pull_request) Successful in 2m24s
CI / typecheck (pull_request) Successful in 2m31s
CI / browser-tests (pull_request) Successful in 3m53s
With _resume no longer throwing after exit, a call into a dead Go handle
resolves with undefined instead of failing — a read loop would spin on it
rather than break out. Conn, PacketConn and TCPListener now report closed
once their runtime has exited, so those calls raise ClosedError and
close() is a no-op.

Refs #147

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 00:38:58 +00:00
codingetandClaude 90f262e301 fix(tsconnect): drop the Go runtime's pending work on exit
CI / format (pull_request) Successful in 3m27s
CI / lint (pull_request) Successful in 3m27s
CI / install (pull_request) Successful in 11m17s
CI / typetest (pull_request) Successful in 2m11s
CI / node-tests (pull_request) Successful in 3m8s
CI / typecheck (pull_request) Successful in 3m24s
CI / browser-tests (pull_request) Successful in 5m56s
wasm_exec.js keeps the Go scheduler's setTimeout armed and every js.FuncOf
callback registered across program exit, and _resume throws "Go program has
already exited" when they fire. After a clean shutdown() that surfaces as
uncaught errors and unhandled rejections seconds later: a scheduler timeout,
DERP WebSocket close listeners, and a fetch continuation.

initIPN now clears the scheduled timeouts from runtime.wasmExit and makes
_resume a no-op once the runtime has exited.

Fixes #147

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 23:51:40 +00:00
codingetandCodex ad0d5259ab test(tsconnect-worker): isolate Go runtime shutdowns
CI / lint (pull_request) Successful in 2m24s
CI / format (pull_request) Successful in 2m32s
CI / install (pull_request) Successful in 7m2s
CI / typetest (pull_request) Successful in 1m54s
CI / node-tests (pull_request) Successful in 2m8s
CI / typecheck (pull_request) Successful in 2m19s
CI / browser-tests (pull_request) Successful in 3m37s
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-07-30 00:38:22 +02:00
codinget afbc809240 chore: remove deprecated APIs: setExitNodeEnabled, ping(disco), fetch, ssh
CI / format (pull_request) Successful in 2m41s
CI / lint (pull_request) Successful in 2m46s
CI / install (pull_request) Successful in 10m45s
CI / typetest (pull_request) Successful in 2m49s
CI / typecheck (pull_request) Successful in 3m2s
CI / node-tests (pull_request) Successful in 3m10s
CI / browser-tests (pull_request) Successful in 6m13s
2026-07-29 21:38:42 +00:00
264 changed files with 15013 additions and 2323 deletions
-1
View File
@@ -1 +0,0 @@
AI_CHANGES.md merge=union
+5 -2
View File
@@ -33,6 +33,7 @@ jobs:
with:
name: node_modules-${{ github.run_id }}
path: node_modules.tar.gz
retention-days: 7
- name: Archive turbo cache
run: tar -czf turbo-cache.tar.gz .turbo
@@ -41,6 +42,7 @@ jobs:
with:
name: turbo-cache-${{ github.run_id }}
path: turbo-cache.tar.gz
retention-days: 7
- name: Archive build artifacts
run: |
@@ -51,6 +53,7 @@ jobs:
with:
name: build-artifacts-${{ github.run_id }}
path: build-artifacts.tar.gz
retention-days: 7
lint:
runs-on: ubuntu-latest
@@ -167,7 +170,7 @@ jobs:
tar -xzf build-artifacts.tar.gz
- name: Run node tests
run: npm run test
run: npm run test -- --only --concurrency=1
- name: Test package boundaries
if: always()
@@ -203,4 +206,4 @@ jobs:
run: npx playwright install --with-deps chromium firefox
- name: Run browser tests
run: npm run test:browser
run: npm run test:browser -- --only --concurrency=1
+83
View File
@@ -0,0 +1,83 @@
name: Update CA bundle
on:
schedule:
- cron: "17 8 * * *"
workflow_dispatch:
permissions:
contents: write
pull-requests: write
concurrency:
group: update-ca-bundle
cancel-in-progress: false
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Refresh CA bundle
id: update
run: |
cp packages/tsconnect/ca/cacert.pem "$RUNNER_TEMP/cacert-before.pem"
npm run update-ca-bundle --workspace=packages/tsconnect
if git diff --quiet -- packages/tsconnect/ca/cacert.pem; then
echo "changed=false" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "changed=true" >> "$GITHUB_OUTPUT"
packages/tsconnect/scripts/describe-ca-bundle-update.sh \
"$RUNNER_TEMP/cacert-before.pem" \
packages/tsconnect/ca/cacert.pem > "$RUNNER_TEMP/pr-body.md"
- name: Push update branch
if: steps.update.outputs.changed == 'true'
env:
UPDATE_BRANCH: automation/update-ca-bundle
run: |
git config user.name "webnet-actions"
git config user.email "actions@noreply.gitea.codinget.me"
git add packages/tsconnect/ca/cacert.pem
git commit -m "chore(tsconnect): update CA bundle"
git push --force origin "HEAD:refs/heads/$UPDATE_BRANCH"
- name: Create or update pull request
if: steps.update.outputs.changed == 'true'
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
UPDATE_BRANCH: automation/update-ca-bundle
PR_TITLE: "chore(tsconnect): refresh CA bundle"
run: |
api="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
auth_header="Authorization: token $GITEA_TOKEN"
body="$(jq -Rs . < "$RUNNER_TEMP/pr-body.md")"
page=1
number=""
while [[ -z "$number" ]]; do
pulls="$(curl -fsSL -H "$auth_header" \
"$api/pulls?state=open&limit=50&page=$page")"
number="$(jq -r --arg branch "$UPDATE_BRANCH" \
'.[] | select(.head.ref == $branch) | .number' <<< "$pulls" | head -n 1)"
[[ "$(jq length <<< "$pulls")" -lt 50 ]] && break
page=$((page + 1))
done
if [[ -n "$number" ]]; then
payload="$(jq -n --arg title "$PR_TITLE" --argjson body "$body" \
'{title: $title, body: $body}')"
curl -fsSL -X PATCH -H "$auth_header" -H "Content-Type: application/json" \
-d "$payload" "$api/pulls/$number"
else
payload="$(jq -n --arg title "$PR_TITLE" --arg head "$UPDATE_BRANCH" \
--arg base "main" --argjson body "$body" \
'{title: $title, head: $head, base: $base, body: $body}')"
curl -fsSL -X POST -H "$auth_header" -H "Content-Type: application/json" \
-d "$payload" "$api/pulls"
fi
+36 -7
View File
@@ -4,6 +4,11 @@ Guidelines for AI agents (Claude Code, etc.) working in this repository.
`CLAUDE.md` is a symlink to `AGENTS.md`: they are the same file, so edit either one and never both — and don't be surprised if only one of them shows up in a listing or diff.
## Project documentation
- [`docs/CONTEXT.md`](docs/CONTEXT.md) defines the canonical product, file-access, security, planning, and API-stability terms.
- [`docs/ROADMAP.md`](docs/ROADMAP.md) records product direction, theme sequencing, quality gates, and release criteria. Gitea milestones and issues own live work status.
## Project structure
- This is an npm workspace repository. The root `package.json` is the workspace manifest, `packages/*` contains the workspace packages, and `package-lock.json` is the lockfile.
@@ -11,6 +16,21 @@ Guidelines for AI agents (Claude Code, etc.) working in this repository.
- Turbo is used as the task runner behind several root scripts (`build`, `test`, `typecheck`, and related commands). Its presence does not change the package manager or workspace layout.
- `tailscale/` is a git submodule containing the forked Tailscale source used by `packages/tsconnect`; it is not another npm workspace.
### Package map
- Foundations: `binary`, `transport`, `state-transfer`, and `vfs`.
- Tailscale integration: `tsconnect`, `tsconnect-worker`, `tsconnect-redux`, `tsconnect-react`, and `taildrive`.
- Protocols: `http`, `websocket`, `webdav`, `ftp`, `ssh`, `sftp`, and `smb2`.
- File serving: `http-static`.
- Shared UI and utilities: `react`, `utils`, `xml`, and `browser-test-utils`.
- Applications: `example-app`, `test-app`, and `tailshare`. These are private workspaces; do not treat them as published libraries.
Use the package's manifest and public exports as the source of truth. The package table in `README.md` is the concise repository overview and should be updated whenever a workspace is added, removed, renamed, or substantially repurposed.
Every workspace has its own `README.md` describing what it does and, where applicable, its entry points and a short usage example. Keep it accurate when a package's public API changes, and add one for any new workspace along with its row in the root table.
`packages/tailshare` is the flagship application and sets first-party product priorities. The reusable libraries can land before Tailshare integrates them. Do not describe `example-app` or `test-app` as the main application.
## Model attribution
Before creating commits, PRs, reviews, or applying `Agent/*` labels, use the exact model currently running — not merely its generic model family.
@@ -26,16 +46,29 @@ Use the model's established display name for commit attribution. For PRs and iss
- Always base PRs against `origin/main`: fetch and pull `origin/main` before branching unless a different base is specified.
- Keep the PR description up to date: update it after each push to reflect the current state of the branch, not just the initial intent.
- Before committing, run linting, formatting, and typechecking scoped to the affected packages or files where practical.
- Apply the org-level `Agentic` label and the established org-level `Agent/<model-slug>` label for the exact coordinating model to every PR or issue you open. These labels already exist at the organisation level — do not create repo-level duplicates. The `Agent/*` labels are exclusive (only one may be set at a time). The org-level `Human` label identifies work opened by a human; do not apply it to work opened by an agent. Use `tea api` (the `tea` label flags are unreliable): look up the label IDs with `tea api orgs/webnet/labels`, then apply them with `tea api repos/{owner}/{repo}/issues/<number>/labels -d '{"labels":[<id1>,<id2>]}'`.
- Apply labels according to the label taxonomy below.
- After pushing to the PR branch or opening the PR, watch the CI results (e.g. via the `tea` CLI or available harness tools) and address any errors before proceeding.
- When the PR work appears complete (typically after push and CI passes): if the harness provides a tool for spawning a new thread or agent, spawn an autonomous code review using a medium-size model (e.g. Sonnet for the Claude lineup). Watch the review output and address findings as appropriate.
- When everything has settled, or if you need the user to unblock you: if the harness or other instructions provide a push notification tool, send the user a push notification with a short summary; include the full details in the main conversation thread.
- When asked to finalise a PR: run all relevant tests and scoped checks as appropriate; rely on CI for global linting, formatting, and typechecking; ensure the AI disclosure in `AI_CHANGES.md` and the PR description are up to date; then remove the `WIP:` prefix from the PR title to mark it ready for merging.
- When asked to finalise a PR: run all relevant tests and scoped checks as appropriate; rely on CI for global linting, formatting, and typechecking; ensure the PR description is up to date; then remove the `WIP:` prefix from the PR title to mark it ready for merging.
- Reading `tea` mergeability output:
- `tea` reports `mergeable: false` not only for git-level conflicts but also whenever the `WIP:` prefix is present, since that blocks merging in the web UI. Only treat `mergeable` as meaningful during finalisation, after the prefix has been removed.
- `tea` output includes a section header listing conflicting files; the header itself is always printed. Its mere presence does not mean there are conflicts — actual conflicting files are listed under it, if any.
- Auto-finalise the PR (without waiting to be asked) if you are confident the user will not request changes — for example, if the work is trivial, mechanical, or was fully specified upfront with no ambiguity. In that case, run the finalise steps above immediately after CI passes and the review is clean.
## Issue and pull request labels
Labels are defined at the `webnet` organization level. Do not create repository-level duplicates, and query `tea api orgs/webnet/labels?limit=100` before applying labels rather than hard-coding numeric IDs.
- Provenance: every issue and PR gets `Human`, `Agentic`, or — when a human and an agent genuinely wrote it together, rather than one drafting and the other editing at the margins — both. Work an agent contributed to also gets exactly one `Agent/<model-slug>` label for the coordinating model; never combine multiple `Agent/*` labels. Do not remove one of a co-authored pair to satisfy the usual one-of rule.
- Kind: classify issues with exactly one of `Kind/Bug`, `Kind/Feature`, `Kind/Enhancement`, or `Kind/Maintenance`.
- Priority: classify issues with exactly one `Priority/P0` through `Priority/P4`, where P0 is critical and P4 is wishlist-tier.
- Area: apply the `Component/*` and `Protocol/*` labels for areas that directly own part of the implementation or are a primary target of the work. These families are not exclusive, but mentions, known callers, and downstream effects alone do not require a label.
- Security: add `Security` to security-sensitive work in addition to its other labels.
- Pull requests must always carry provenance labels. Also add the relevant classification labels when the PR needs to be discoverable independently of a linked issue; do not add unrelated labels mechanically.
Use `tea api` because the `tea` label flags are unreliable. Resolve the current organization label IDs, then apply the complete desired set with `tea api -X PUT repos/{owner}/{repo}/issues/<number>/labels -d '{"labels":[<id1>,<id2>]}'`. `PUT` replaces the label set; the default `POST` only adds to it, so removing a label needs `PUT`. Label endpoints treat pull requests as issues.
## Reviewing pull requests
- Always use the `tea` CLI to read PR details and post comments — use `tea pr` subcommands, not `gh`.
@@ -49,6 +82,7 @@ Use the model's established display name for commit attribution. For PRs and iss
Co-Authored-By: <Model Name> <noreply@provider.example>
```
For example: `Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>` or `Co-Authored-By: GPT-5.5 <noreply@openai.com>`. Do not hardcode a model name — use whichever model is actually running.
- Add the trailer only for work an agent actually wrote or materially drafted — not for a commit or PR the maintainer hand-wrote and an agent merely reviewed. A significant share of this repository is hand-written on branches an agent later reviewed; the trailer records who was involved, not who wrote the code, so it must not imply otherwise. In commit messages, PR descriptions, issue write-ups, and review comments, say "reviewed by" rather than "authored by" when that's the actual role. Check `git log` or ask if authorship is unclear.
## Worktrees
@@ -62,8 +96,3 @@ Follow the existing style of the codebase. Use `packages/http` as the primary re
## Dependencies
Do not add new dependencies without first asking the user, unless the intent to do so is unambiguously implied by the task (e.g. initialising a React app, adding a Postgres client when the task is explicitly about connecting to Postgres). When in doubt, ask.
## AI disclosure
`AI_CHANGES.md` contains the full AI disclosure log and must be kept up to date. `README.md` links to it but does not contain the disclosure itself.
At the end of every branch, review the commits (e.g. `git log main..HEAD --oneline`) and append entries to `AI_CHANGES.md` for any new work that was assisted or authored by an AI agent.
-129
View File
@@ -1,129 +0,0 @@
# AI disclosure
This file records all work in this repository that was assisted or authored by an AI agent. It is append-only; merge conflicts are resolved by keeping all lines (see `.gitattributes`).
- **`@webnet/tsconnect` — integration coverage for the untested bridge surface (PR #146)**: Claude Code (Claude Opus 5) extended the headscale integration suite from a TCP listen/dial pair to real two-node coverage of UDP datagrams, `listenICMP` (including a hand-built ICMP echo request and its reply), `listenTLS`/`dialTLS` against a run-time-generated self-signed CA plus the wrong-CA rejection case, `upgradeTls` on a live connection, Taildrop end-to-end (`listFileTargets`, `sendFile`, `waitingFiles`, `openWaitingFile`, `deleteWaitingFile`), `serveDrive`/`listDrivePeers` over a real netmap, `suggestExitNode`, and `setExitNode`. `serveDrive`, `listDrivePeers` and `upgradeTls` previously had tests only against mock raw objects that never loaded the wasm. Taildrop registers its own node pair from a new optional `TSCONNECT_TEST_USER_AUTH_KEY`, because `canPutFile` requires an untagged peer and the shared key registers `tag:test`; `setExitNode` is gated on `TSCONNECT_TEST_EXIT_NODE_ID`, and both are documented in `.env.example`. `shutdown()` runs in a child process (`src/shutdown.probe.ts`, excluded from the browser build via a new `*.probe.ts` tsconfig exclude) and the shared suite no longer tears the runtime down, because an exiting Go runtime leaves scheduler timeouts that throw afterwards and `node:test` claims the exception first (issue #147). Writing the tests surfaced two pre-existing gaps, worked around rather than fixed: `":port"` is normalised only for TCP `listen`, so `listen("udp4", ":0")` and `listenTLS(":0", …)` fail with `ParseAddrPort(":0"): no IP`; and a UDP socket bound to `0.0.0.0` cannot reach the tailnet. Verified against a tailscale build from the rebased fork branch across three consecutive green runs.
- **Tailshare and example-app frontend typechecking (issue #122)**: `gpt-5.6-terra` added Turbo-discoverable TypeScript checks to both frontend packages, updated their library targets and declarations for current dependency typings, removed stale tsconnect source includes, and corrected the example HTTP demo for the current listener and server APIs so both applications typecheck cleanly.
- **Tailshare and example-app frontend typechecking — review follow-up**: `gpt-5.6-terra` preserved the example HTTP demo's original `GET /` behavior while migrating it to the current server API, following an independent Claude review.
- **Repository package-boundary enforcement (issue #107)**: `gpt-5.6-sol` removed all cross-package `_internals` imports and cross-package re-exports, replaced wildcard barrels with explicit reviewed exports, moved shared length-prefixed wire encoding into the utilities package, relocated shared path-helper coverage to VFS, preserved SSH malformed-auth and SFTP cleanup regressions, and added tested ESLint enforcement, package-boundary documentation, and SSH public-surface typetests.
- **`@webnet/tsconnect-worker` — deterministic async test cleanup**: `gpt-5.6-sol` fixed a flaky Go/WASM test teardown by awaiting the runtime shutdown started by `disconnect()`, replaced nearby fixed-delay MessagePort waits with direct call assertions and an ordered delivery barrier, and converted TTL tests to mocked timers so they wait for every relevant expiration rather than whichever equal-duration timer happens to fire first.
- **`@webnet/sftp` — stable keyless server identity**: GPT-5.6 Luna fixed `SFTPServer` to lazily generate and cache one ephemeral host key per server instance, and added a regression test covering sequential connections and host-key fingerprints.
- **`@webnet/vfs` — canonical path helpers**: Codex (GPT-5.6 Luna) added public POSIX VFS path utilities for normalization, resolution, parents, basenames, and joining. The helpers clamp traversal at the VFS root, preserve backslashes and Unicode names, and are covered by focused edge-case tests. MemoryVFS, FsaVFS, FTP, and SFTP now reuse the shared implementation; NodeVFS retains its separate traversal-rejection guard.
This project was set up with the assistance of [Claude Code](https://claude.ai/code) (Anthropic). The following were written by Claude Code:
- **`@webnet/http-static` — VFS-backed static HTTP handler**: GPT-5.6 Luna added a new workspace package exposing `createStaticHandler`, with request/context path resolution, prefix and suffix lookup, index files, HTML/JSON directory listings, fallback paths, metadata and conditional responses, HEAD support, and single-byte range streaming. Added focused unit coverage and package build/test/typecheck scaffolding.
- **`@webnet/http-static` — review fixes**: GPT-5.6 Luna addressed review findings by rejecting requests outside a configured prefix, returning 416 for ranges on empty files, normalizing method matching, and honoring `If-None-Match: *`.
- **`@webnet/http-static` — precedence test vectors**: GPT-5.6 Luna added a seeded `MemoryVFS` matrix covering every requested path across default/custom/disabled indexes, suffix probing, and fallback configurations.
- **`@webnet/http-static` — HTTP-relative directory listings**: GPT-5.6 Luna updated generated JSON paths and HTML links to use the request URL pathname, including configured HTTP prefixes, and added prefixed listing coverage.
- The `tailscale` submodule fork and its Go-side `tsconnect` patches (the `webnet` branch)
- The `packages/tsconnect` TypeScript SDK
- The `packages/test-app` Vite test application
- The repo tooling setup (ESLint, Prettier, lint-staged, commitlint, dpdm, TypeScript 6)
- `AGENTS.md` — the agent guidelines file (worktree/submodule rules, commit-trailer convention, PR finalisation checklist)
The following were hand-written:
- The `packages/http` HTTP/1.1 server library (HTTP parsing, server/client connection management, chunked transfer encoding, router/middleware system, response serialisation)
The following were hand-written but with substantial Claude Code contributions:
- **`packages/http` — low-level transport layer**: reviewed and significantly reworked by Claude Code, fixing multiple bugs (errcallback not cleared after a successful read, `ServerConnection` draining the body after closing the connection, `PooledDialer` throwing instead of queuing waiters, `shouldClose()` doing a case-sensitive header comparison) and adding transport primitives (`halfClose`, `readEnded`, `whenClosed`, `remoteAddr`, `localAddr`)
- **`packages/http` — timeout support**: the `headersTimeout`, `keepAliveTimeout`, and `bodyTimeout` options across both client and server were implemented by Claude Code
- **`packages/http` — parse error messages**: improved by Claude Code to include the offending values
- **`packages/http` — package and build scaffolding**: initial `package.json`, `tsconfig.json`, and build configuration were set up by Claude Code
The test suite for `packages/http` was mostly generated by Claude Code, which also identified and fixed several further bugs: a body-size-limit error being swallowed on the final body chunk, a stale idle-connection entry left behind by `rejectConnection`, and a `read()` call hanging indefinitely after a socket close.
- **`packages/http``hijack()` on server and client responses**: the `hijack()` method on `ServerResponse` and `ClientResponse`, the `ReadBuffer.drain()` helper, and the `prependTransport()` utility were implemented by Claude Code
- **`packages/http` — 1xx informational response support**: implemented by Claude Code. Server side: automatic `100 Continue` (sent lazily when the handler reads the body) and `res.sendInformational()` for 103 Early Hints etc. Client side: default skip mode, `interim: "collect"` to capture 1xx into `res.informational[]`, `conn.requestStream()` async generator that yields each interim response and the final one as they arrive, and `fetchStream()` / `f.stream()` to expose the same streaming behaviour through the fetch API with proper connection pool management.
- **`packages/http` — WebSocket support**: implemented by Claude Code. `upgradeWebSocket(req, res)` for server-side handshake; `connectWebSocket(dialer, url, options?)` to open a new WebSocket connection, or `connectWebSocket(res, key)` to promote an existing `fetch()`/`fetchStream()` 101 response — both return a `WebSocketConnection` async iterable. Frame codec (read/write), masking, fragmented-message reassembly, ping/pong, and the close handshake are all implemented from scratch using the Web Crypto API (`crypto.subtle.digest` for SHA-1, `crypto.getRandomValues` for mask keys) with no external dependencies. Two fixed bugs in `fetch.ts` were required for pool safety: a case-insensitive `Connection: upgrade` check and immediate pool ejection on 101 to prevent a microtask race before hijack. Exported as three tree-shakeable entry points: `@webnet/http/websocket` (combined), `@webnet/http/websocket/client`, and `@webnet/http/websocket/server`.
- **`packages/http` — 3xx redirect support**: implemented by Claude Code. `redirect.mode` (`"manual"` / `"same-connection"` / `"same-origin"` / `"follow"`), `redirect.max`, `redirect.filter` (string array / Set / callback), `redirect.credentials` (`"keep"` / `"strip-cross-origin"` / `"strip"`), `redirect.body` (`"resubmit"` / `"strip-non-resubmit"` / `"strip"`), and `redirect.collect` to gather followed 3xx into `res.redirects[]`. In streaming mode all redirects and interims are yielded as they arrive; a `drain()` method was added to `ClientConnection` to support clean connection hand-off between redirect hops.
- **`packages/http` — lint fix and WebSocket test coverage**: Claude Code fixed a `prefer-const` lint error in `ServerConnection` by refactoring `HijackFn` to accept `res` as a parameter (eliminating a forward-reference `let res!` pattern), updated the ESLint config to recognise `_`-prefixed variables as intentionally unused (`varsIgnorePattern`), and added test coverage for extended-length WebSocket frames (2-byte and 8-byte), multi-chunk `ReadBuffer` slicing, empty close frames, unsolicited PONG frames, invalid port URLs, and the `fetchStream()` 101 early-break path.
- **`packages/http` — typed context extension via `next(extra)`**: the `Router<T>` generic, the `Next<TAdd>` conditional type, and the context-passing chain were designed and implemented by Claude Code
- **Monorepo restructuring**: the following packages were created or substantially reworked by Claude Code — `@webnet/transport` (transport abstractions, buffer utilities, node and loopback implementations extracted from `@webnet/http`); `@webnet/websocket` (WebSocket support extracted from `@webnet/http`); `@webnet/tsconnect-redux` (Redux state bindings extracted from `@webnet/tsconnect`); `@webnet/tsconnect-react` (React hooks and context extracted from `@webnet/tsconnect`). `@webnet/http`'s `Server` class was refactored to take a `Handler` argument instead of extending `Router`; `Router` was moved to a `./router` sub-export. `@webnet/tsconnect`'s `Conn` and `TCPListener` were updated to formally implement `RawTransport` and `RawListener`; `IPNDialer` implementing `RawDialer` was added.
- **`@webnet/tsconnect` — drop pre-compression of assets**: Claude Code removed the gzip/brotli pre-compression of `main.wasm` (from the `tsconnect` fork's build-pkg) and `cacert.pem` (from `packages/tsconnect/build.sh`), dropping the corresponding `.br`/`.gz` package exports and adding `scripts/asset-sizes.sh` (exposed as `npm run asset-sizes`) to let consumers measure raw, gzip, and brotli sizes of the built assets.
- **`packages/xml`**: a thin XML parse/serialize package with conditional exports — browser builds use the native `DOMParser`, Node.js builds use `@xmldom/xmldom`. Authored by Claude Code.
- **`packages/drive`**: WebDAV client and server with an async VFS abstraction. Includes `MemoryVFS`, `NodeVFS` (node:fs), `FsaVFS` (browser File System Access API, with an OPFS factory method), `createDAVHandler()` (server-side HTTP handler compatible with `@webnet/http`), and `DAVClient` (which implements `AsyncVFS` so it can be used as a backing store for another server instance). Full WebDAV Level 2 locking support (`LockStore` interface, `InMemoryLockStore`, LOCK/UNLOCK methods, `If:` header enforcement, `lockdiscovery`/`supportedlock` properties, and client-side `lock()`/`unlock()`/`refreshLock()` with optional `lockToken` on all mutating methods). Authored by Claude Code.
- **`@webnet/tsconnect` — streaming Taildrop**: Claude Code removed all full-file buffering from the Taildrop send and receive paths. `IPN.sendFile` now accepts a `ReadableStream<Uint8Array>` + `declaredSize`; `IPN.openWaitingFile` returns `Promise<ReadableStream<Uint8Array>>`. On the Go/WASM side, a new `jsStreamReader` (`io.ReadCloser`) pulls chunks from a JS `ReadableStreamDefaultReader` via awaited `.read()` Promises (channel+`js.FuncOf` pattern), and `jsReadableStream` wraps a Go `io.ReadCloser` in a pull-based JS `ReadableStream`. `UserIPNFileOps.openReader` now returns a `ReadableStream` instead of a `Uint8Array`. A new `FsaFileOps` class (with `FsaFileOps.createFromOpfs()`) provides an OPFS-backed `UserIPNFileOps` where received chunks land directly on disk and downloads stream back through Go without buffering. `InMemoryFileOps.openReader` was updated to emit stored chunks one-by-one via a `ReadableStream`.
- **`@webnet/tsconnect``IPN.shutdown()`**: Claude Code added a `shutdown()` method to `IPN` (TypeScript) and `jsIPN` (Go/WASM). Calling it stops the `LocalBackend`, closes the safesocket listener to unblock `srv.Run`, and signals `main()` to return so the Go runtime exits. The TypeScript side awaits the `go.run()` Promise (captured in `initIPN` and threaded into each `IPN`) as the authoritative "Go runtime has exited" signal, avoiding a race where Go deletes `_inst` before a callback-based resolve could fire. Go-side nil guards were added for `lb` and `ln` so `shutdown()` is safe to call even when `run()` was never invoked.
- **`@webnet/tsconnect` — multi-environment WASM loading and Node.js fixes**: Claude Code investigated Worker/SharedWorker and Node.js/Bun compatibility. The Go WASM and `wasm_exec.js` are already Worker-compatible (`js.Global()` maps to the Worker's `globalThis`; `wasm_exec.js` stubs `fs`/`process`/`path` when absent). Three issues were found and fixed for Node.js:
1. `initIPN` accepted only a URL string and called `fetch()`, which does not support `file://` URLs in Node.js. Claude Code added a `WasmSource` union type (`string | URL | ArrayBuffer | ArrayBufferView | Response | ReadableStream<Uint8Array>`) dispatching to `WebAssembly.instantiate` for binary sources and `WebAssembly.instantiateStreaming` for URL/Response/ReadableStream inputs.
2. `wasm_exec.js` installs ENOSYS stubs for `globalThis.fs` when it is falsy. In Node.js `globalThis.fs` is undefined, so the stubs are installed — and this is the correct behaviour: tsconnect's WASM routes all network calls through JavaScript's `fetch()` and WebSocket APIs (which use Node.js's own DNS resolver), so the Go net package's `/etc/resolv.conf` read should remain a no-op. An earlier iteration set `globalThis.fs` to Node.js's real `fs`, which caused Go to read the host's `/etc/resolv.conf` directly and attempt to use those nameservers, breaking in environments where they are unreachable (e.g. Tailscale-managed entries without Tailscale running). `wasm_exec.js` is now imported directly in `index.ts` and `Go` is extracted from `globalThis` inline — a separate `env-node.ts`/`env-web.ts` split was explored but both files were identical, so the indirection was removed.
3. In the `tailscale` submodule, two Go-side bugs were fixed: `safesocket_js.go` used a hardcoded memconn address `"Tailscale-IPN"`, so a second `newIPN()` call in the same WASM process would `log.Fatal`; an atomic counter now gives each instance a unique address. And `wasm_js.go`'s `listen()` rejected the standard `":0"` (any-interface) address form that netstack does not accept; it now normalises `:port` to `0.0.0.0:port`.
Claude Code also wrote the full automated test suite for `@webnet/tsconnect`: unit tests for `InMemoryFileOps` and `InMemoryState` (no WASM required), and integration tests that spin up real Tailscale nodes against a headscale control server, verifying `initIPN` WASM loading, `/localapi/v0/status`, and two-node TCP dial/listen.
- **`@webnet/tsconnect``getIceServers()`**: Claude Code implemented `getIceServers(ipn)`, which fetches the tailnet's DERPMap via LocalAPI and converts it to an `RTCIceServer[]` list for use with `new RTCPeerConnection({ iceServers })`. DERP servers run an integrated RFC 5389-compliant STUN server on UDP 3478 by default; since tsconnect in the browser cannot use raw UDP (all traffic goes through DERP over WebSocket), WebRTC is the only way to establish a direct peer-to-peer connection. Also added `DERPMap`, `DERPRegion`, and `DERPNode` types. Unit and integration tests included.
- **`@webnet/tsconnect` — service advertisement**: Claude Code added `SetExplicitServices` to `LocalBackend` in the tailscale fork (bypassing the OS portlist gate), wired a `setServices(services)` WASM binding, and added a `services` field to the netmap JSON for both self and peers (stripping internal peerapi entries). On the TypeScript side: `IPNService` type, `services: IPNService[]` on `IPNNetMapNode`, and `IPN.setServices()`. In `@webnet/tsconnect-redux`: `getPeersByService`, `getPeersByServiceDescription` (both memoized with `createSelector`), and `getSelfServices` (referentially stable empty-array fallback). Claude Code also wrote integration tests for `setServices()` and fixed two bugs found while making them reliable: `SetExplicitServices` previously only sent a "lite" map update that discarded the control server's response, so `notifyNetMap` never fired with the new services — fixed by adding `Auto.RestartMap()` to force a fresh streaming netmap; and `userServicesFromView` returned a nil slice when a node advertised no services, which serialized as `null` instead of `[]` in the netmap JSON.
- **`@webnet/tsconnect` — taildrive WebDAV bridge and `listDrivePeers()`**: Claude Code implemented `jsFileSystemForRemote` in the tailscale fork (`cmd/tsconnect/wasm/drive.go`), a `drive.FileSystemForRemote` backed by a JS callback: request bodies stream to JS chunk-by-chunk via `readBodyChunk()`, response bodies stream back via `write()`/`end()` with `http.Flusher.Flush()` after each chunk, so large transfers never buffer in memory. `sys.DriveForRemote` is set at `newIPN` time; `setDriveHandler` (wired to `IPN.serveDrive(fn)`) registers the actual handler later, since the Tailscale auth/permission-parsing (`DriveSharingEnabled`, `ParsePermissions`) happens entirely Go-side before the JS handler is invoked. Also added `listDrivePeers` (wired to `IPN.listDrivePeers()`), mirroring native `driveRemotesFromPeers`: returns peers carrying `PeerCapabilityTaildriveSharer`, gated on `DriveAccessEnabled()`. On the TypeScript side: `DriveSharePermission`, `DrivePermissions`, `JsDriveRequest`, `JsDriveResponse`, `RawDriveHandler`, `IPNDrivePeer` types. Claude Code also created the new `@webnet/taildrive` package (`./server` sub-export) with `bridgeDriveHandler(handler)`, which adapts an `@webnet/http` `Handler` (e.g. from `@webnet/drive`'s `createDAVHandler`) to the Go bridge's raw request/response callback shape — kept in a separate package from `@webnet/tsconnect` and `@webnet/drive` to avoid a circular dependency between the two. Unit tests cover `bridgeDriveHandler`'s request/response translation (headers, all `Body` union variants, `hasBody` detection) and `IPN.serveDrive`/`listDrivePeers` argument validation and JSON handling against a fake `RawIPN`, so they run without a WASM build. A true end-to-end test against a live control plane is blocked on Headscale ACL support for `nodeAttrs`/`grants`/Taildrive, which only lands in Headscale v0.29.0 (not yet released stably at the time of writing).
- **`packages/http``ReadableStream` body support**: implemented by Claude Code. Renamed the `AsyncIterable`-returning method to `iter()` and added a new `stream()` returning a `ReadableStream<Uint8Array>` with BYOB (byte stream) support where available. `WritableHttp.body` now accepts `ReadableStream<Uint8Array>`, written as chunked transfer encoding with BYOB reads and periodic flushes every 2^16 bytes. The `iterableToStream()`/`streamToIterable()` helpers in `packages/drive` were removed as they are no longer needed. `@webnet/taildrive`'s `bridgeDriveHandler` (predating this rename) was updated to match: `ctx.req.stream()` now returns a real BYOB-capable `ReadableStream<Uint8Array>` instead of an `AsyncIterable`, a `ctx.req.iter()` method was added for the old semantics, and response bodies that are a `ReadableStream` are now forwarded correctly instead of falling through to JSON serialization.
- **`@webnet/transport``NodeListener` bug fixes**: two bugs fixed by Claude Code (Claude Sonnet 4.6): double invocation of the error callback (fired both directly from `close()` and from the server's `"close"` event), and a connection leak when `accept()` was rejected while a late-arriving connection triggered the still-registered `once("connection")` handler.
- **`@webnet/transport` — WebRTC DataChannel transport**: Claude Code (Claude Sonnet 4.6) implemented `@webnet/transport/webrtc` — a `RawTransport` over an `RTCDataChannel`. Designed to coexist on a shared `RTCPeerConnection` alongside other channels (multiplexed file transfers, audio/video). Write-side backpressure defers `write()` promises via `bufferedamountlow` when `bufferedAmount` exceeds the high-watermark. Receive-side backpressure for slow local readers (e.g. OPFS writes) removes the `message` listener so the browser's internal SCTP buffer fills and signals flow control back to the sender; the listener is re-added once the read queue drains. All WebRTC types are defined as local structural interfaces so the package needs no DOM lib. 114 tests at 100% coverage.
- **`@webnet/tsconnect-worker`**: new package authored by Claude Code (Claude Sonnet 4.6). Wraps the `IPN` and `tsconnect-redux` Redux store in a `SharedWorker`, exposing every IPN method to clients via message-port RPC with per-object `MessageChannel`s for `Conn`, `TCPListener`, `PacketConn`, drive handler, and SSH sessions. Key features: `IndexedDBState` for sync IPN state storage (unavailable `localStorage` workaround — loads entire store into a `Map` at startup, serves reads/writes synchronously, fires async IDB writes on mutation); Web Locks API for client liveness detection (worker cleans up all resources opened by a disconnected client: connections, listeners, packet connections, drive handler); Redux action broadcast to all clients on every dispatch; full state snapshot sent to each new client on connect; graceful IPN shutdown when the last client leaves. Two tsconfigs: DOM lib for `IpnWorkerClient` and proxy classes (`WorkerConn`, `WorkerTCPListener`, `WorkerPacketConn`, `WorkerSSHSession`); WebWorker lib for the worker entry point. Round-2 improvements (Claude Sonnet 4.6): `IndexedDBState` moved to `@webnet/tsconnect/helpers` and exported from the package; `buildIpnStore` now accepts an optional `preloadedState`, and the worker sends the full Redux state snapshot (`preloadState` message) instead of synthetic actions on client connect; `WorkerConn` formally implements `RawTransport`, `WorkerTCPListener` implements `RawListener`, `WorkerPacketConn` implements `IpnPacketConn`; a new `IpnClient` interface added to `@webnet/tsconnect` (with `IpnSSHTermConfig` and `IpnPacketConn`) implemented by both `IPN` and `IpnWorkerClient` to prevent method drift; `IPN.ssh()` made async; `WorkerSSHSession.resize()`/`close()` made fire-and-forget (sync) to implement `IPNSSHSession`; `ReadableStream` transfer fallback via `MessageChannel` sub-protocol for Safari compatibility on both `sendFile` and `openWaitingFile`; `stateStorage: "memory"` option added to `WorkerConfig` for ephemeral (non-persisted) IPN state. `@webnet/tsconnect-react` updated by Claude Code (Claude Sonnet 4.6): `IpnContext` widened from `IPN` to `IpnClient`; `useBuildIpnWorker(worker, config, runParams?, workerOptions?)` hook added to connect to a SharedWorker and return an `IpnWorkerClient`, with StrictMode-safe cleanup via `disconnect()`; `disconnect()` method added to `IpnWorkerClient` to release the Web Lock without stopping the worker; `workerOptions` parameter added to support both classic (webpack-bundled) and module workers. `example-app` updated by Claude Code (Claude Sonnet 4.6): SharedWorker demo added — `src/worker.ts` is the webpack worker entry, `IpnProvider` runs `useBuildIpnWorker` when SharedWorker is available (falling back to main-thread `useBuildIpn`), switches the Redux `Provider` to the worker client's embedded store on connect; the Debug UI shows a "use SharedWorker" checkbox (checked by default when available, with appropriate state/taildrop labels for each mode). Round-3 bug fixes (Claude Sonnet 4.6): liveness detection now fires for all clients including those that connect during init (unified `onHello` handler in `onconnect`; `registerClient` always receives `lockName` and acquires the lock immediately); dead `bodyCallbacks` field removed from drive pending map; drive cleanup only installs the no-op handler when no other client still has drive registered; `wrapDispatch` `as any` replaced with targeted two-step cast; `IndexedDBState.setState` logs write failures via `tx.onerror`; `pumpStreamToPort` fire-and-forget made explicit with `.catch(() => {})`.
- **`AGENTS.md` / `CLAUDE.md` and `AI_CHANGES.md`**: PR review instructions (use `tea`, interactive vs autonomous modes) and AI disclosure consolidation into `AI_CHANGES.md` authored by Claude Code (Claude Sonnet 4.6).
- **`@webnet/vfs` — VFS package extraction**: Claude Code (Claude Sonnet 4.6) split the VFS abstraction (`AsyncVFS`, `Stat`, `VFSError`, `VFSErrorCode`) and its three implementations (`MemoryVFS`, `NodeVFS`, `FsaVFS`) out of `@webnet/drive` into a new standalone `@webnet/vfs` package, following the same pattern as the earlier `@webnet/transport` split from `@webnet/http`. `@webnet/drive` and `@webnet/test-app` now import directly from `@webnet/vfs`. The drive package's `./vfs/*` sub-exports were removed. The `NodeVFS` test suite moved with the implementation.
- **`@webnet/tsconnect``FsaFileOps` limits and change notification**: Claude Code (Claude Sonnet 4.6) added `maxFiles`/`maxTotalSize`/`maxFileSize` limit getters/setters, `fileCount`/`totalSize`/`openFiles` getters, and `onChange`/`offChange` change-notification to `FsaFileOps`, mirroring the existing `InMemoryFileOps` API. File sizes are tracked in memory via a `#fileSizes` map that is updated on every `openWriter`/`write`/`remove`/`rename`; `createFromOpfs` now scans the directory at startup so pre-existing files count toward limits. The constructor was updated to accept an optional `{ maxFiles, maxTotalSize, maxFileSize, initialSizes }` options bag. Follow-up fixes (also Claude Sonnet 4.6): moved `#fileSizes.set` in `openWriter` to after all async FSA ops so a failure doesn't leave a phantom entry; fixed `rename` to stat and track previously-untracked files under their new name rather than letting them become invisible; documented the `stat`/`totalSize` lag during active writes.
- **`@webnet/tsconnect`** - `InMemoryFileOps` limits and change handlers reviewed by Claude Code (Sonnet 4.6)
- **Browser testing infrastructure**: Claude Code (Claude Sonnet 4.6) added browser integration testing using Playwright as a library within the existing node:test runner. Key design decisions: `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1` in `.npmrc` prevents binary downloads on `npm ci` (explicit `npx playwright install` in CI only); browser test files use the `*.browser.ts` extension to avoid matching the existing `src/**/*.test.ts` glob so the regular test suite is unaffected; `@webnet/browser-test-utils` is a new private package that exports `forBrowsers()` (registers test suites for Chromium and Firefox, handles browser lifecycle) and `serveDirectory()` (minimal HTTP server using `path.resolve`/`path.relative` to guard against path traversal). `test:browser` per-package scripts added; `test:browser:coverage` intentionally omitted since c8 only sees Node orchestration code, not browser-side code inside `page.evaluate()`. Browser tests use `page.evaluate()` with dynamic `import()` to load built package modules into real browser contexts. Integration tests written for: `DataChannelTransport` (real `RTCPeerConnection` loopback, send/receive and close-propagation); `FsaVFS` (OPFS round-trip, stat, readdir, delete); `IndexedDBState` (multi-instance persistence, empty-DB initialisation); `FsaFileOps` (write/read/stat/remove/rename/listFiles cycle). CI pipeline added at `.gitea/workflows/test-browser.yml`.
- **`@webnet/browser-test-utils``__name` polyfill**: Claude Code (Claude Sonnet 4.6) fixed a `ReferenceError: __name is not defined` crash in all `*.browser.ts` tests. tsx hardcodes `keepNames: true` in its internal esbuild options, which injects a `__name` helper at module scope and wraps named function/const assignments with `__name(fn, "name")`. Playwright's `page.evaluate()` serializes callbacks via `.toString()`, capturing only the function body — so the module-level helper is absent in the browser context. Fixed by calling `page.addInitScript()` in `forBrowsers`'s `newPage` to inject a matching polyfill (`Object.defineProperty(target, "name", ...)`) into every page before any `evaluate()` runs.
- **`ControlledFileOps` — observable file-ops binding**: Claude Code (Claude Sonnet 4.6) added a `ControlledFileOps` interface to `@webnet/tsconnect` that extends `UserIPNFileOps` with readable metrics (`fileCount`, `openFiles`, `totalSize`), configurable limits (`maxFiles`, `maxTotalSize`, `maxFileSize`), and an `onChange(handler) => unsubscribe` subscription — already satisfied structurally by `InMemoryFileOps` and `FsaFileOps`. Added a `fileOps` Redux slice to `@webnet/tsconnect-redux` (stores `FileOpsState | null`; `null` when file ops are not configured) with `setFileOpsState` action, `selectFileOps` selector, and a `getFileOpsState` convenience selector. Added `bindFileOpsToStore(fileOps, store)` to `@webnet/tsconnect-redux`'s binding module: takes an initial snapshot and re-dispatches on every `onChange` call; the returned unsubscribe is stored by the worker for lifecycle safety. In `@webnet/tsconnect-worker`: the `FsaFileOps` instance was hoisted from `init()` to module scope; `bindFileOpsToStore` is called after the store is created; a new `setFileOpsConfig({ maxFiles?, maxTotalSize?, maxFileSize? })` method on `IpnWorkerClient` proxies to a new worker-side `handleCall` case that mutates the live instance and throws back any limit-violation errors. All state updates propagate via the existing `wrapDispatch` broadcast, so every connected client's store reflects the current file-ops state and limits in real time. Unit tests for `bindFileOpsToStore` added to `@webnet/tsconnect-redux`.
- **`ControlledFileOps` — observable file-ops binding**: Claude Code (Claude Sonnet 4.6) added a `ControlledFileOps` interface to `@webnet/tsconnect` that extends `UserIPNFileOps` with readable metrics (`fileCount`, `openFiles`, `totalSize`), configurable limits (`maxFiles`, `maxTotalSize`, `maxFileSize`), and an `onChange(handler) => unsubscribe` subscription — already satisfied structurally by `InMemoryFileOps` and `FsaFileOps`. Added a `fileOps` Redux slice to `@webnet/tsconnect-redux` (stores `FileOpsState | null`; `null` when file ops are not configured) with `setFileOpsState` action, `selectFileOps` selector, and a `getFileOpsState` convenience selector. Added `bindFileOpsToStore(fileOps, store)` to `@webnet/tsconnect-redux`'s binding module: takes an initial snapshot and re-dispatches on every `onChange` call. In `@webnet/tsconnect-worker`: the `FsaFileOps` instance was hoisted from `init()` to module scope; `bindFileOpsToStore` is called after the store is created; a new `setFileOpsConfig({ maxFiles?, maxTotalSize?, maxFileSize? })` method on `IpnWorkerClient` proxies to a new worker-side `handleCall` case that mutates the live instance and throws back any limit-violation errors. All state updates propagate via the existing `wrapDispatch` broadcast, so every connected client's store reflects the current file-ops state and limits in real time.
- **`@webnet/tsconnect-redux` — typecheck fix**: Claude Code (Claude Sonnet 4.6) added `@types/node` to `devDependencies` and `"types": ["node"]` to `tsconfig.json` so that the `node:test`/`node:assert/strict` imports in `binding.test.ts` resolve during `tsc --noEmit`.
- **`@webnet/tsconnect-worker` — main-thread fallback and `connectWithFallback`**: Claude Code (Claude Sonnet 4.6) added a full main-thread fallback path for environments that do not support `SharedWorker` (e.g. Chrome on Android before 2025). Key additions:
- **`IpnClientHandle` interface**: extends `IpnClient`, adding `connectionMode`, `store`, `state`, `running`, `fileOps`, `run()`, `disconnect()`. Both `IpnWorkerClient` and the new `IpnMainThreadHandle` implement it.
- **`IpnMainThreadHandle`**: wraps an `IPN` instance with a Redux store (wired via `runWithStore` at construction so future state changes fire user callbacks set later in `run()`). Proxies all `IpnClient` methods directly. `disconnect()` shuts down IPN and releases the lock.
- **`connectMainThread(config)`**: initialises IPN in the main thread. For IndexedDB-backed state, acquires a web lock keyed to the DB name (`"tsconnect-idb:<dbName>"`); rejects immediately if held. In-memory state skips the lock entirely.
- **`connectWithFallback(workerUrlOrFactory, config, opts?)`**: tries the SharedWorker path first; falls back to `connectMainThread` if `SharedWorker` is undefined, `opts.disableSharedWorker` is true, or the worker path throws.
- **`useBuildIpnWorker` updated** in `@webnet/tsconnect-react`: now uses `connectWithFallback` and returns `IpnClientHandle | null` instead of `IpnWorkerClient | null`.
- **Tests**: 6 Node tests (all pass — WASM loads, `IpnMainThreadHandle` is returned, `run()` fires callbacks, `disconnect()` is idempotent, `connectWithFallback` falls back in Node); 6 browser tests in Chromium and Firefox (SharedWorker path → `connectionMode=worker`; `disableSharedWorker: true``connectionMode=main-thread`; IDB lock contention → `connectMainThread` rejects immediately). Browser tests use esbuild (hoisted transitive dep) to bundle the package inline in the test's `before()` hook. `WorkerConfig.wasmUrl` widened to `string | ArrayBuffer | ArrayBufferView` so tests can pass raw WASM bytes (Node.js `fetch()` does not support `file://` URLs). Node tests restructured to share a single WASM+IPN instance per suite via `before()`/`after()` hooks, preventing uncaught Go goroutine errors from sequentially shutting down multiple WASM runtimes. Code review addressed: `IpnClientHandle.disconnect()` JSDoc clarifies the main-thread-vs-worker semantics difference; `connectWithFallback` logs a `console.warn` for diagnosed fallbacks; lock release uses a closure rather than an unbound method reference.
- **CI — lint, format, typecheck, typetest, and build workflows**: Claude Code (Claude Sonnet 4.6) added `.gitea/workflows/checks.yml` with five jobs (`lint`, `format`, `typecheck`, `typetest`, `build`) following the same structure as the existing `test-node.yml` and `test-browser.yml` workflows. Also added a `@webnet/tsconnect#typecheck` package-level override in `turbo.json` so that package's typecheck task depends on its own build (required because `src/index.ts` imports `../dist/wasm_exec.js`, a WASM artifact absent in a fresh environment).
- **`@webnet/tsconnect-worker` — FileOps limits at init**: Claude Code (Claude Sonnet 4.6) added `fileOpsMaxFiles`, `fileOpsMaxTotalSize`, and `fileOpsMaxFileSize` fields to `WorkerConfig` in `protocol.ts`, and wired them through to `FsaFileOps.createFromOpfs()` in `worker.ts`. Limits can also be changed after startup via the existing `setFileOpsConfig()` method on `IpnWorkerClient`.
- **`@webnet/tsconnect-worker` — FileOps limits at init**: Claude Code (Claude Sonnet 4.6) added `fileOpsMaxFiles`, `fileOpsMaxTotalSize`, and `fileOpsMaxFileSize` fields to `WorkerConfig` in `protocol.ts`, and wired them through to `FsaFileOps.createFromOpfs()` in `worker.ts`. Limits can also be changed after startup via the existing `setFileOpsConfig()` method on `IpnWorkerClient`. A `FileOpsLimits` named type was extracted to `protocol.ts` and shared between the worker-side handler and the client-side `setFileOpsConfig` signature to prevent future drift.
- **`AGENTS.md` / `CLAUDE.md` — PR workflow improvements**: Claude Code (Claude Sonnet 4.6) added guidance for always basing PRs on `origin/main` (fetch/pull before branching), watching CI results after push, spawning an autonomous code-review agent (Sonnet-class) once CI passes, and sending a push notification to the user when work settles or needs unblocking.
- **Test coverage gap-fill**: Claude Code (Claude Sonnet 4.6) added unit tests to four previously uncovered packages. `packages/xml`: 17 tests for `el()`, `text()`, and `stringify()` (namespace prefixing, escaping, self-closing vs. open/close tags, all namespaces hoisted to root), plus `test`/`test:coverage` scripts. `packages/tsconnect-redux`: 60 tests covering all ~25 exported selectors (state, login, exitNode, self, lockedOut, suggestedExitNode, fileTargets, peer lookups by id/name/service/description, self-services, outgoing/incoming/waiting file selectors, fileOps state). `packages/vfs`: 44 direct `MemoryVFS` unit tests (stat, readdir, writeFile, readFileRange, mkdir, delete recursive, copy, move, setProps; all VFSError codes). `packages/tsconnect-worker`: 38 tests for `WorkerConn`, `WorkerTCPListener`, `WorkerPacketConn`, `WorkerSSHSession`, `pumpStreamToPort`, and `portToReadableStream` using `MessageChannel` pairs (no browser APIs required); added `@types/node` devDep and `"types":["node"]` to tsconfig. HTTP redirect tests were found to already exist in `packages/http/src/client/fetch.test.ts` (lines 5851203).
- **`AGENTS.md` / `CLAUDE.md` — PR labeling and auto-finalise**: Claude Code (Claude Sonnet 4.6) added guidance to apply the org-level `Agentic` and `Agent/<model-line>` labels (which already exist at org level; `Agent/*` are exclusive) to every agent-opened PR, and to auto-finalise PRs without waiting for the user when the work is trivial or fully specified upfront.
- **`@webnet/react` and `@webnet/utils` — new packages**: Claude Code (Claude Sonnet 4.6) extracted the `useClient` and `useLocalStorage` hooks from `packages/tailshare` into a new standalone `@webnet/react` package, and extracted the `download`, `upload`, `readBlob`, and `fmtSize` utilities into a new `@webnet/utils` package. Both packages are plain TypeScript with no dependencies beyond React (peer dep for `@webnet/react`) and are built with `tsc`.
- **`@webnet/react` and `@webnet/utils` — integration into `example-app` and `test-app`**: Claude Code (Claude Sonnet 4.6) wired `@webnet/react` and `@webnet/utils` into the consumer apps. In `example-app`: replaced the local `useClient` duplicate with the package version; added `useLocalStorage` to persist the SharedWorker toggle (`ipn:useWorker`) across page reloads; replaced the local `fmtSize` with `@webnet/utils`; simplified the `WaitingFileDebug` download handler to use `download()` from `@webnet/utils`. Two bugs were fixed in `@webnet/utils/download` during integration: a typo (`suggegestedName``suggestedName`) in the `showSaveFilePicker` call, and synchronous `URL.revokeObjectURL` reverted to a deferred `setTimeout` for Safari compatibility. In `test-app`: added `@webnet/utils` as a dependency and exposed it on `window.utils` for console testing.
- **`@webnet/tsconnect-worker` — flaky closed-state test fix**: Claude Code (Claude Sonnet 4.6) replaced racy `setTimeout(r, 10)` waits in four "rejects immediately when closed" tests (`read`, `write`, `accept`, `readFrom`) with a deterministic `while (!x.closed) await setImmediate()` poll. The 10 ms sleep was not always sufficient under CI load, causing the "closed" message to arrive _after_ the method under test created a pending promise, yielding `"packet conn closed"` instead of the expected `"already closed"` error.
- **CI — consolidate workflows and eliminate redundant install/build**: Claude Code (Claude Sonnet 5) merged `checks.yml`, `test-node.yml`, and `test-browser.yml` into a single `.gitea/workflows/ci.yml`. Previously all 7 jobs independently repeated checkout + `tailscale` submodule clone + `npm ci`, and 5 of them independently rebuilt the whole workspace via Turbo. Live smoke tests against this Gitea instance found `actions/cache` (and `setup-node`'s built-in `cache: npm`, same API) times out against the built-in cache proxy, and `actions/upload-artifact`/`download-artifact` v4 refuse to run at all (GHES detection), but v3 of the artifact actions work correctly. Given that, a new `install` job now does the real work once (submodule clone, `npm ci`, build) and hands `node_modules` plus Turbo's local cache off to `typecheck`/`typetest`/`node-tests`/`browser-tests` (all `needs: install`) via `actions/upload-artifact@v3`/`download-artifact@v3`, instead of each job reinstalling and rebuilding from scratch. `lint`/`format` never touched the submodule or build output, so they dropped that step and stay independent for fast feedback. The shared checkout+submodule+setup-node preamble was factored into a composite action, `.gitea/actions/setup/action.yml` (composite actions require the repo to already be checked out, so `actions/checkout` stays a separate first step in every job rather than being absorbed into the composite action). A `concurrency` group cancels superseded runs on the same ref. One real bug was found and fixed during this work: `packages/xml` and `packages/vfs` pin a newer local TypeScript than the workspace root, installed by npm as nested `packages/{xml,vfs}/node_modules/typescript`; the first version of the `install` job's `node_modules` archive only captured the root `node_modules`, so downstream jobs silently typechecked those two packages against the wrong TypeScript version and produced spurious errors — fixed by archiving `packages/*/node_modules` alongside the root. Cross-run caching (reusing a previous run's install/build) is not available until the Gitea instance's cache backend is fixed server-side; that's a separate, out-of-scope follow-up.
- **`@webnet/tsconnect-worker` — remaining flaky message-wait fixes**: Claude Code (Claude Sonnet 5) found the same fixed-delay race pattern still present in the rest of `worker.test.ts` (the `WorkerSSHSession.resize()`/`close()` tests called out as flaky in CI, plus the other `close()`-message, callback-firing, and `pumpStreamToPort` tests), where a flat `setTimeout(r, 10)` was used to wait for a `postMessage` to be delivered before asserting on it. Added a generic `waitFor(predicate, timeoutMs?)` helper that polls via `setImmediate` and replaced every such fixed sleep with a poll on the actual condition (message present in the captured array, or callback fired). Following an autonomous review, the four remaining ad hoc `while (!x.closed) await setImmediate()` spin-loops from the earlier b917cd8 fix were also consolidated onto the same `waitFor` helper for consistency.
- **`AGENTS.md` / `CLAUDE.md``tea` mergeability and symlink clarifications**: Claude Code (Claude Fable 5) documented that `CLAUDE.md` is a symlink to `AGENTS.md`, that `tea` reports `mergeable: false` while the `WIP:` title prefix is present (so mergeability is only meaningful at finalisation), and that the conflicting-files section header in `tea` output is always printed — only files listed under it indicate actual conflicts.
- **`AGENTS.md` / `CLAUDE.md` — exact model attribution**: Codex (GPT-5.6-Sol) documented that agents must use the exact running model for commits, PRs, reviews, and `Agent/*` labels; prefer harness-provided identity, fall back to Codex CLI session metadata when needed, and never guess or reuse another session's model identity.
- **`AGENTS.md` / `CLAUDE.md` — project structure**: Codex (GPT-5.6 Luna) documented that this repository uses npm workspaces and `package-lock.json`, with Turbo as the task runner and `tailscale/` as a git submodule rather than an npm workspace; agents should not treat it as a pnpm repository.
- **CI — require built WASM test artifacts**: Codex (GPT-5.6 Terra) made `test` and `test:coverage` depend on each package's own build, archives/restores workspace build outputs for every downstream CI job, and turns missing tsconnect WASM artifacts into CI-only test failures while preserving local skips.
- **`@webnet/ftp` — new package (FTP/FTPS client and server)**: Claude Code (Claude Fable 5, with Claude Sonnet 5 subagents on the common protocol layer) authored a new `@webnet/ftp` package: an FTP + implicit-FTPS server exposing any `AsyncVFS` over the `RawListener` transport interface, and an `FTPClient implements AsyncVFS` (mirroring `DAVClient`) over a `RawDialer`. Server command set targets FileZilla/KDE kio/classic `ftp` compatibility: USER/PASS (pluggable `authenticate` callback returning a per-user VFS), FEAT/OPTS UTF8/SYST/TYPE/MODE/STRU/PBSZ/PROT, PWD/CWD/CDUP, PASV/EPSV passive data channels via a caller-injected `dataListen` factory (TLS listeners = FTPS data channels; PORT/EPRT and AUTH TLS are 502 stubs pending active-mode and transport TLS-upgrade support), LIST/NLST (unix `ls` format), MLSD/MLST, STAT, SIZE/MDTM, REST/RETR/STOR streaming directly between the data connection and VFS web streams, DELE/RMD/MKD, RNFR/RNTO, ABOR. Client prefers EPSV/MLSD and falls back to PASV and unix-`ls` LIST parsing (for vsftpd, which lacks MLSD); operations are serialized on a single control connection with a mutex held across each verb+data-stream sequence; `readFileRange` maps to REST plus client-side truncation; `copy` is omitted (no server-side copy in FTP). Error translation both ways via `vfsErrorToReply`/`replyToVFSError` tables. 125 tests: unit tests for the codec/listing/address/time/path helpers and loopback integration suites (full AsyncVFS conformance, LIST/PASV fallback paths, raw protocol-level server assertions).
- **Development apps — explicit remote-host mode**: Codex (GPT-5.6 Terra) added a safe local `dev` mode and an explicit `dev:host` mode to the Vite test app and webpack example app. The hosted mode binds to all interfaces and permits forwarded hosts only when `WEBNET_DEV_HOST=1`; workspace documentation covers trusted-network exposure and WebSocket proxying for HMR.
- **`@webnet/ftp` — real-server integration coverage**: Codex (GPT-5.5) added an opt-in integration suite for a real FTP server. It is skipped unless `FTP_TEST_HOST`, `FTP_TEST_USER`, and `FTP_TEST_PASS` are configured, with optional `FTP_TEST_PORT` and `FTP_TEST_DIR`; it verifies upload, listing, download, rename, and recursive cleanup.
- **`AGENTS.md` — scoped pre-commit checks and CI-based PR finalisation**: Codex (GPT-5.6 Luna) documented running linting, formatting, and typechecking scoped to affected packages or files before commits, relying on CI for global checks during PR finalisation, and monitoring CI after pushes.
- **`AGENTS.md` — issue labeling**: Codex (GPT-5.6 Luna) extended the required `Agentic` and model-specific `Agent/*` labels from pull requests to issues, and documented the org-level `Human` label for human-opened work.
- **Explicit TLS upgrade (STARTTLS / AUTH TLS) on `RawTransport`**: Claude Code (Claude Fable 5) added an optional in-place `upgradeTls(options?: TlsUpgradeOptions)` method and an optional `isTls` getter to `RawTransport` in `@webnet/transport`, for protocols that negotiate in plaintext and then upgrade the existing connection (SMTP STARTTLS, FTPS AUTH TLS). `TlsUpgradeOptions` is a union of client mode (`serverName`/`insecureSkipVerify`/`caCerts`; `serverName` required unless skipping verification, since an upgraded connection only knows its peer IP) and server mode (`isServer: true` with `certPem`/`keyPem`). All implementations require a quiescent transport (no pending read, no buffered data) and close the connection on handshake failure. `NodeTransport` swaps its socket for a `node:tls` `TLSSocket` wrapping the same socket (listener re-wiring extracted into `#attach`/`#detach`; a peer aborting the handshake surfaces as a plain close on the server side, handled explicitly); a checked-in self-signed localhost cert fixture backs a real bidirectional STARTTLS test suite. `@webnet/tsconnect`'s `Conn` swaps its raw Go handle via a new `upgradeTLS` wasm bridge method (added to `wrapConn` in the `tailscale` submodule, with the TLS client-config construction factored out of `dialTLS` into `tlsClientConfigFromJS`; server mode uses `tls.X509KeyPair` + `tls.Server` like `listenTLS`) and guards against in-flight reads/writes with an operation counter; `dialTLS`-created conns now report `isTls: true`. `@webnet/tsconnect-worker` gained `upgradeTls`/`upgraded`/`upgradeError` messages in the per-conn protocol, an `isTls` field on the `conn`/`accepted` messages, and matching `WorkerConn` support. Verified end-to-end with a live STARTTLS handshake against smtp.gmail.com:587. Three AI review rounds hardened the feature: an autonomous Sonnet review found that reads/writes weren't blocked while an upgrade handshake was in flight (fixed with an `upgrading` guard in all three implementations); a neutral Fable 5 review found the server-side pipelined-ClientHello race (a fast peer's handshake bytes landing in the internal buffer before `upgradeTls` is called are now unshifted back into the stream) and the worker `upgradeError` conflating validation failures with handshake failures (the reply now carries a `closed` flag); a GPT-5.6 review found that pre-handshake configuration failures leaked the Go conn (the wrapper marked it closed while Go left it open — Go now closes on every error path) and that a synchronous invalid-PEM throw could permanently strand the node transport in upgrading state (TLS socket construction moved inside the failure path). Final semantics, documented on `RawTransport`: synchronous option-validation rejections leave the transport usable; any failure after the upgrade starts closes it.
- **Cross-tab state and ownership transfer (`@webnet/transport`, `@webnet/tsconnect-worker`, `@webnet/http`, `@webnet/drive`)**: designed and orchestrated by Claude Code (Claude Fable 5), implemented by Claude Opus 4.8 and Claude Sonnet 4.5 subagents. Adds a mechanism to move live resources (tcp/tls conns, listeners, udp packet conns, plus arbitrary JSON/byte/transferable payloads) from one SharedWorker client tab to another without reopening connections, e.g. to pop a protocol panel out into its own window that survives the originator closing. `@webnet/transport` gains the `StateTransferable` interface (`transferState()` detaches and returns a structured-cloneable state) and `isStateTransferable`. `@webnet/tsconnect-worker`: worker-side resources are tracked per client under stable resource ids and can be detached into a TTL'd pending-transfer registry (`transfers.ts`) that survives the owning client's death (IPN shutdown is deferred while transfers are pending) and re-bridged to the claiming client; clients register an app-supplied `clientKey` in `hello`, and `IpnWorkerClient` gains a broker API (`sendTransfer`/`onTransfer`, convenience `transfer`/`onAdopt`, `claim`, `transferSupported`, `workerDialer`); envelopes addressed to a not-yet-connected key are queued in the worker and flushed on registration (Fable 5 fixed a handshake bug found by the browser tests where flushed envelopes arriving before `ready` were dropped). `@webnet/http`: `ClientConnection.canExport()/exportTransport()` and `ConnectionPool.exportIdle()/seed()` move idle keep-alive connections (with any buffered prefix bytes) between pools. `@webnet/drive`: `DAVClient` implements `StateTransferable<DavTransferState>` and `DAVClient.adopt()` rebuilds a client from claimed transports, degrading gracefully to re-dialing when claims fail. Covered by Node unit tests (registry, proxies, broker, pool, DAV) and a two-tab Playwright suite proving a transferred listener survives originator-tab death.
- **Cross-tab state and ownership transfer (`@webnet/transport`, `@webnet/tsconnect-worker`, `@webnet/http`, `@webnet/drive`)**: designed and orchestrated by Claude Code (Claude Fable 5), implemented by Claude Opus 4.8 and Claude Sonnet 4.5 subagents. Adds a mechanism to move live resources (tcp/tls conns, listeners, udp packet conns, plus arbitrary JSON/byte/transferable payloads) from one SharedWorker client tab to another without reopening connections, e.g. to pop a protocol panel out into its own window that survives the originator closing. `@webnet/transport` gains the `StateTransferable` interface (`transferState()` detaches and returns a structured-cloneable state) and `isStateTransferable`. `@webnet/tsconnect-worker`: worker-side resources are tracked per client under stable resource ids and can be detached into a TTL'd pending-transfer registry (`transfers.ts`) that survives the owning client's death (IPN shutdown is deferred while transfers are pending) and re-bridged to the claiming client; clients register an app-supplied `clientKey` in `hello`, and `IpnWorkerClient` gains a broker API (`sendTransfer`/`onTransfer`, convenience `transfer`/`onAdopt`, `claim`, `transferSupported`, `workerDialer`); envelopes addressed to a not-yet-connected key are queued in the worker and flushed on registration (Fable 5 fixed a handshake bug found by the browser tests where flushed envelopes arriving before `ready` were dropped). `@webnet/http`: `ClientConnection.canExport()/exportTransport()` and `ConnectionPool.exportIdle()/seed()` move idle keep-alive connections (with any buffered prefix bytes) between pools. `@webnet/drive`: `DAVClient` implements `StateTransferable<DavTransferState>` and `DAVClient.adopt()` rebuilds a client from claimed transports, degrading gracefully to re-dialing when claims fail. Covered by Node unit tests (registry, proxies, broker, pool, DAV) and a two-tab Playwright suite proving a transferred listener survives originator-tab death. Following review (GPT 5.6-Terra and an autonomous Claude Sonnet 4.5 pass), Fable 5 fixed an ownership-crossing bug (results of in-flight reads/accepts at detach time are now parked in a per-resource generation-tagged backlog and delivered to the claiming owner in order, instead of leaking on the old client entry) and five smaller findings (nested-token collection in `transfer()`, handshake replay ordering, DAV export error path, pool slot pruning, main-thread stub rejection semantics). After the branch was rebased onto the TLS-upgrade work, Fable 5 integrated the two features: transferred conns keep their TLS state across ownership transfer (isTls on claim replies and conn ResourceMeta), upgradeTls is serialized through the per-resource channel lock, and transferState refuses while an upgrade is in flight.
- **`@webnet/ftp` — explicit FTPS (`AUTH TLS`)**: Codex (GPT-5.6-Sol) added an `"explicit"` client security mode with verified control/data upgrades, configurable CA roots and an explicit insecure override that still sends the configured hostname for SNI. The server now handles `AUTH TLS`, advertises it through `FEAT`, supports optional or required TLS-before-login policy, and applies `PROT P` to passive data transports. Passive listeners arm acceptance when PASV/EPSV starts so Node transports cannot lose early connections, while TLS handshakes remain deferred until the transfer command. Real TLS tests cover protected round trips, SNI in insecure mode, custom and untrusted roots, hostname mismatch, required-login policy, invalid server credentials, missing upgrade support, and data-handshake recovery.
- **`@webnet/ftp` — explicit FTPS (`AUTH TLS`)**: Codex (GPT-5.6-Sol) added an `"explicit"` client security mode with verified control/data upgrades, configurable CA roots and an explicit insecure override that still sends the configured hostname for SNI. The server now handles `AUTH TLS`, advertises it through `FEAT`, supports optional or required TLS-before-login policy, and applies `PROT P` to passive data transports after the RFC-required protected-control and `PBSZ 0` sequence. Passive listeners arm acceptance when PASV/EPSV starts so Node transports cannot lose early connections, while TLS handshakes remain deferred until the transfer command; abandoned early accepts are closed when a passive channel is replaced. Real TLS tests cover protected round trips, SNI in insecure mode, custom and untrusted roots, hostname mismatch, required-login policy, invalid server credentials, missing upgrade support, data-handshake recovery, and passive transport cleanup.
- **`@webnet/smb2` — new SMB2/3 client package**: Claude Code (Fable 5) designed and implemented a new `@webnet/smb2` package: an SMB2/3 client that runs over the existing `RawTransport`/`RawDialer` abstraction (TCP/445 in Node, relayed through tsconnect's `IPNDialer` in the browser) and exposes a remote Windows/Samba share as an `@webnet/vfs` `AsyncVFS` (`SMB2Client`), mirroring how `@webnet/drive` exposes WebDAV. It targets default Windows 10/11 (including 24H2's mandatory SMB signing) and supported Samba 4.x, negotiating dialects {2.0.2, 2.1, 3.1.1}. Highlights:
- **Isomorphic crypto, no Node APIs, no new dependencies.** The primitives Web Crypto lacks are hand-written in TypeScript and only run over small NTLM blobs: MD4, MD5, HMAC-MD5. SMB3 AES-CMAC signing is built on Web Crypto `AES-CBC` (RFC 4493 subkey construction); the SP800-108 key-derivation function on Web Crypto `HMAC-SHA256`; SHA-512 preauth-integrity and HMAC-SHA256 signing use Web Crypto natively. All primitives are covered by published test vectors (RFC 1320/1321/2202/4231/4493, NIST SP800-108, and the MS-NLMP §4.2.4 NTLMv2 sample).
- **Authentication:** NTLMv2 inside NTLMSSP inside a minimal hand-rolled SPNEGO (DER) wrapper, with the message-integrity code (MIC) and channel-binding AV pair.
- **Signing** (HMAC-SHA256 for 2.x, AES-CMAC for 3.x) and 3.1.1 SHA-512 preauth-integrity hashing are implemented; the client both signs its requests and **verifies inbound response signatures** (rejecting unsigned or invalid responses on a signed session, per [MS-SMB2] 3.2.5.1.3). SMB3 encryption is deferred (the transform layer is structured so it can be added without restructuring), and no cipher is advertised so an encryption-mandating share fails cleanly at TREE_CONNECT.
- **VFS surface:** `stat`, `readdir`, `readFile`/`readFileRange` (streaming, credit-aware, chunked to the negotiated max read size), `writeFile`, `mkdir`, `delete` (recursive), `move` (rename), `copy`, and `setProps` (timestamps), plus `connect`/`disconnect`. NTSTATUS codes are translated to `VFSError` codes at the boundary.
- **Tests:** unit tests with the crypto/auth vectors above, protocol codec round-trip tests, and an in-repo mock SMB2 server that performs a real SMB 3.1.1 signed round-trip end-to-end over the loopback transport (including tests that tampered and unsigned responses are rejected). An opt-in `smb2.integration.test.ts` (gated behind `SMB2_TEST_*` env vars) runs against a real Samba/Windows share.
- Wired into `@webnet/test-app` (exposed as `window.smb2`) for a browser smoke check.
- The protocol command codecs and the mock server were drafted by subordinate Sonnet agents from exact wire-layout specifications; the crypto, authentication, connection/session state machine, and VFS mapping were written and reviewed by Fable 5.
- **`@webnet/sftp` — new package (SFTP v3 client and server)**: Claude Code (Claude Fable 5, coordinating; Claude Opus 4.8 subagents implemented the SSH transport, auth/channels, and SFTP client/server layers, and a Claude Sonnet 5 subagent scaffolded the package and pure codecs) authored a new `@webnet/sftp` package: an SFTP v3 client (`SFTPClient implements AsyncVFS`, mirroring `DAVClient`/`FTPClient`) over a `RawDialer`, and an `SFTPServer` serving any `AsyncVFS` over a `RawListener` via an http-style `listen(listener, opts)` accept loop. Both run in the browser (via tsconnect's `IPNDialer`) and Node with no Node APIs in package source and no new dependencies. The package implements a full SSH-2 transport from scratch on Web Crypto (`crypto.subtle`/`crypto.getRandomValues`) with zero hand-written primitives: version-banner exchange, binary packet framing with per-direction sequence numbers, `curve25519-sha256` key exchange with `ssh-ed25519`/`rsa-sha2-256`/`rsa-sha2-512` host-key verification, RFC 4253 key derivation, `aes128/256-gcm@openssh.com` and `aes128/256-ctr` with `hmac-sha2-256`/`hmac-sha2-256-etm@openssh.com` ciphers (continuous CTR counter and GCM invocation nonce tracked across packets), and transparent peer- or self-initiated rekey. On top of that: ssh-userauth (client offers a direct signed Ed25519 publickey request then falls back to password; server drives none/publickey(PK_OK)/password through a pluggable `authenticate` callback returning a per-user `AsyncVFS`), a connection-protocol mux with a session channel and bidirectional window flow control giving end-to-end backpressure, and the SFTP v3 layer. The client pipelines requests over one channel (request-id dispatch map, serialized wire writes so a split WRITE stays contiguous), maps AsyncVFS verbs onto FXP operations with pull-driven read streams (≤8 outstanding 32 KiB READs) and bounded-inflight writes, client-side recursive delete, and `posix-rename@openssh.com` for overwriting `move`. The server maps FXP back onto the VFS with a handle table, 100-entry READDIR batches with unix `ls -l` longnames, sequential streaming reads/writes, `SETSTAT`/`FSETSTAT` no-ops (so OpenSSH `put` succeeds), REALPATH, and v3 rename semantics; responses are serialized per session and handler errors reply a status without tearing down the connection. Ed25519 auth keys are parsed from the unencrypted `openssh-key-v1` format (encrypted keys are out of scope); host keys are optionally verified via a `verifyHostKey({ type, key, fingerprint })` callback and can be generated with the exported `generateHostKey()`. 203 tests: crypto/kex/cipher/codec/key/path units, loopback transport suites (incl. a 1000-packet encrypted echo and mid-stream rekey), per-layer auth/channel/client/server suites, and a real-client-against-real-server end-to-end suite (4 MiB windowed transfer, ranged reads, pipelined concurrency, cancel-then-continue, password/publickey/per-user auth, host-key verification), plus an env-gated suite against a real OpenSSH sshd. Verified interoperable against the OpenSSH `sftp` CLI (which surfaced and fixed a pre-subsystem `env` channel-request handling gap). An autonomous code review (Claude Sonnet 5) followed by fixes (Claude Fable 5) hardened the server against a malformed post-handshake packet leaking the transport, a write-queue deadlock when a backing `vfs.writeFile` fails mid-stream, an unenforced channel receive window, and a zero-length SFTP packet. A second independent review (Claude Fable 5) found and fixed a client-side data-corruption bug: the pipelined reader trusted requested offsets, so a spec-legal short mid-file READ (pipes/special files, some non-OpenSSH servers) left an un-requested gap; the reader now reconciles against the bytes actually returned and re-issues from the true offset. It also now cancels the source stream on a `writeFile` error. Known intentional limitations (streaming model over `AsyncVFS`, which has no chmod/utimes/positioned-write primitives): SETSTAT/FSETSTAT are accepted as no-ops, and writes must be sequential from offset 0.
- **`@webnet/sftp` — new package (SFTP v3 client and server)**: Claude Code (Claude Fable 5, coordinating; Claude Opus 4.8 subagents implemented the SSH transport, auth/channels, and SFTP client/server layers, and a Claude Sonnet 5 subagent scaffolded the package and pure codecs) authored a new `@webnet/sftp` package: an SFTP v3 client (`SFTPClient implements AsyncVFS`, mirroring `DAVClient`/`FTPClient`) over a `RawDialer`, and an `SFTPServer` serving any `AsyncVFS` over a `RawListener` via an http-style `listen(listener, opts)` accept loop. Both run in the browser (via tsconnect's `IPNDialer`) and Node with no Node APIs in package source and no new dependencies. The package implements a full SSH-2 transport from scratch on Web Crypto (`crypto.subtle`/`crypto.getRandomValues`) with zero hand-written primitives: version-banner exchange, binary packet framing with per-direction sequence numbers, `curve25519-sha256` key exchange with `ssh-ed25519`/`rsa-sha2-256`/`rsa-sha2-512` host-key verification, RFC 4253 key derivation, `aes128/256-gcm@openssh.com` and `aes128/256-ctr` with `hmac-sha2-256`/`hmac-sha2-256-etm@openssh.com` ciphers (continuous CTR counter and GCM invocation nonce tracked across packets), and transparent peer- or self-initiated rekey. On top of that: ssh-userauth (client offers a direct signed Ed25519 publickey request then falls back to password; server drives none/publickey(PK_OK)/password through a pluggable `authenticate` callback returning a per-user `AsyncVFS`), a connection-protocol mux with a session channel and bidirectional window flow control giving end-to-end backpressure, and the SFTP v3 layer. The client pipelines requests over one channel (request-id dispatch map, serialized wire writes so a split WRITE stays contiguous), maps AsyncVFS verbs onto FXP operations with pull-driven read streams (≤8 outstanding 32 KiB READs) and bounded-inflight writes, client-side recursive delete, and `posix-rename@openssh.com` for overwriting `move`. The server maps FXP back onto the VFS with a handle table, 100-entry READDIR batches with unix `ls -l` longnames, sequential streaming reads/writes, `SETSTAT`/`FSETSTAT` no-ops (so OpenSSH `put` succeeds), REALPATH, and v3 rename semantics; responses are serialized per session and handler errors reply a status without tearing down the connection. Ed25519 auth keys are parsed from the unencrypted `openssh-key-v1` format (encrypted keys are out of scope); host keys are optionally verified via a `verifyHostKey({ type, key, fingerprint })` callback and can be generated with the exported `generateHostKey()`. 202 tests: crypto/kex/cipher/codec/key/path units, loopback transport suites (incl. a 1000-packet encrypted echo and mid-stream rekey), per-layer auth/channel/client/server suites, and a real-client-against-real-server end-to-end suite (4 MiB windowed transfer, ranged reads, pipelined concurrency, cancel-then-continue, password/publickey/per-user auth, host-key verification), plus an env-gated suite against a real OpenSSH sshd. Verified interoperable against the OpenSSH `sftp` CLI (which surfaced and fixed a pre-subsystem `env` channel-request handling gap). An autonomous code review (Claude Sonnet 5) followed by fixes (Claude Fable 5) hardened the server against a malformed post-handshake packet leaking the transport, a write-queue deadlock when a backing `vfs.writeFile` fails mid-stream, an unenforced channel receive window, and a zero-length SFTP packet. A second independent review (Claude Fable 5) found and fixed a client-side data-corruption bug: the pipelined reader trusted requested offsets, so a spec-legal short mid-file READ (pipes/special files, some non-OpenSSH servers) left an un-requested gap; the reader now reconciles against the bytes actually returned and re-issues from the true offset. It also now cancels the source stream on a `writeFile` error. Known intentional limitations (streaming model over `AsyncVFS`, which has no chmod/utimes/positioned-write primitives): SETSTAT/FSETSTAT are accepted as no-ops, and writes must be sequential from offset 0.
- **`@webnet/ssh` — SSH-2 package split out of `@webnet/sftp`, plus TCP port forwarding**: Claude Code (Claude Fable 5) extracted the hand-written SSH-2 stack (transport, curve25519 kex, ciphers, host keys, userauth, connection-protocol channels) out of `@webnet/sftp` into a new standalone `@webnet/ssh` package via pure `git mv`s, so it can be reused independently; `@webnet/sftp` now depends on `@webnet/ssh` and consumes its low-level pieces through `@webnet/ssh/_internals`. The server auth path was decoupled from `@webnet/vfs`: `authenticateServer<T>` is now generic over an authentication context and rejects with a new `SSHAuthError` rather than `VFSError`, which `@webnet/sftp` maps back to `VFSError("forbidden")` at its own boundary to preserve behaviour. On top of the split, TCP forwarding primitives were added. The channel mux (`ConnectionMux`) gained configurable accepted channel types and a global-request handler, generic `CHANNEL_OPEN` handling that surfaces incoming opens (session / direct-tcpip / forwarded-tcpip with parsed endpoints) as an `IncomingOpen` the consumer can `accept()`/`reject()`, outbound `openDirectTcpip`/`openForwardedTcpip`, and an in-order `globalRequest`/`onGlobalRequest` path for `tcpip-forward`/`cancel-tcpip-forward` (RFC 4254 §7). A `channelTransport()` adapter wraps a `Channel` as a `@webnet/transport` `RawTransport` (EOF surfaced as a throw with `readEnded`, `halfClose` as `CHANNEL_EOF`, non-blocking `close()` so a forwarded stream can't deadlock on the peer close handshake), and an internal `pipe()` bridges two transports. The public API exposes `SSHClientConnection` (connect over a `RawTransport`, `openSubsystem`, `openSession`, `openDirectTcpip`, `dialer()` returning a `RawDialer` for local forwarding, and `requestRemoteForward()` returning a `RemoteForward` that implements `RawListener` for remote forwarding) and `SSHServerConnection<T>` (auth hook returning the context, `acceptSession()`, and optional `directTcpip`/`tcpipForward` hooks that plug arbitrary `RawTransport`/`RawListener` implementations into the forwarding paths). The design deliberately keeps `openSession()`/session-channel handling generic so later PRs can add exec/shell/pty request plumbing (for scp/rsync clients and a shell-delegating server) without reshaping the connection API. `@webnet/sftp`'s client and server were refactored onto the new connection classes with no behaviour change (its full suite passes unmodified). New tests cover direct-tcpip open round-trips and accept/reject, global-request ordering/failure, the channel/RawTransport adapter (EOF, half-close, windowed backpressure), and loopback end-to-end forwarding in both directions; an env-gated (`SSH_TEST_*`) interop suite exercises `ssh -L`/`ssh -R` equivalents against a real OpenSSH sshd (direct-tcpip to the sshd banner, and a remote-forward loop dialed back through direct-tcpip). Two autonomous code-review rounds followed. A Sonnet review found port-only remote-forward keying (two forwards on the same port collided), unguarded `CHANNEL_OPEN_CONFIRMATION` sends in the accept loops, and unrejected pending `RemoteForward.accept()` waiters on close — all fixed. A second Fable review then found three more serious issues, all fixed by Claude Fable 5 with regression tests: (1) `Channel.send()` deadlocked forever if the peer closed the channel while the sender was blocked on window exhaustion (`_deliverClose` now wakes window waiters and `send()` aborts on a remotely-closed channel); (2) `pipe()`'s copy loop only guarded reads, so a write failure on a forwarded socket became an `unhandledRejection` that crashes the Node process by default — a remotely-triggerable DoS — now the write is guarded and both `void pipe()` call sites swallow; (3) connect-phase failures leaked the underlying socket (`SFTPClient.#doConnect` and `SSHClientConnection.connect` now close on `openSubsystem`/auth failure; verified against an sshd with no sftp subsystem, which previously hung the process to the runner timeout). Four smaller fixes: `authenticateServer` accepts a falsy auth context (uid `0` etc.) instead of rejecting it, `#matchForward`'s port fallback fails closed on an ambiguous port rather than misrouting, `acceptSession()` rejects instead of hanging once the accept loop has died, and a duplicate `tcpip-forward` closes the superseded listener (and `RemoteForward` teardown closes queued-but-unaccepted transports).
- **`@webnet/ftp` — FTP client state transfer**: GPT-5.6 Terra implemented transferable FTP control connections: `FTPClient.transferState()` exports worker-backed connections plus buffered reply bytes and negotiated session state; `FTPClient.adopt()` claims them or reconnects and authenticates on expiry. Active data transfers are refused, and focused tests cover claim, fallback, and the safety guard.
- **`@webnet/ftp` — state-transfer ownership fix**: GPT-5.6 Terra addressed a GPT-5.6 Sol review finding by making detachment atomic under the control lock; queued commands now reject after handoff and a regression test covers that race.
- **`@webnet/smb2` — cross-tab state transfer (`StateTransferable`)**: Claude Code (Fable 5) implemented worker-side ownership transfer for `SMB2Client` (issue #99), consistent with `DAVClient`. `SMB2Client.transferState()` detaches an idle client and exports a structured-cloneable `SMB2TransferState`: the parked-transport token plus everything needed to keep using the authenticated session on the adopting side — server/share/port config, negotiated dialect and max transact/read/write sizes, message-id and credit counters, session and tree ids, the 3.1.1 preauth hash, and the _derived_ signing key with the signing-required flag. Credentials are never serialized; `SMB2Client.adopt(state, { claim, dialer, username, password, reconnect? })` takes them from the adopter's configuration, validates state version/shape before claiming, resumes message-id and signing state exactly once on the claimed transport, closes the claimed transport if resumption fails, and (only with `reconnect: true`) falls back to a fresh dial-on-demand session when the claim is expired/duplicate — transferred signing/session state is never reused on a new transport. Transfer is rejected unless the client is quiescent (no request in flight, no connect pending, no read/write stream still owning an SMB file handle, no buffered incoming bytes), and the source client becomes permanently detached after a successful transfer instead of silently reconnecting. Tests extend the mock server with per-connection signing state, an SMB 2.1 mode, request-signature verification, and message-id recording, covering: signed 2.1/3.1.1 sessions continuing with the next message id after a `structuredClone`d transfer (with response-signature verification still enforced), busy/streaming/connect-pending rejection, config-only transfer for never-connected clients or non-transferable transports, malformed/version-mismatched/duplicate/expired state handling, claimed-resource cleanup on failed adoption, and reconnect fallback; the opt-in Samba integration suite gained a live transfer round-trip. A browser two-tab transfer test is deferred: it needs a reachable SMB server inside the worker's tailnet, which the browser CI environment does not provide (the same limitation applies to the existing DAV/FTP transfer work).
- **Agent label documentation**: `gpt-5.6-sol` updated `AGENTS.md` to require exact `Agent/<model-slug>` labels and coordinating-model attribution for orchestrated work.
- **`@webnet/utils` — shared binary reader/writer (issue #79)**: GPT-5.6 Luna implemented a dependency-free, endian-aware binary codec primitive with bounds-checked reads, growable writes, offset-correct `DataView` handling, alignment, skipping, random access, patching, and explicit finish-copy semantics. SSH and SMB2 now delegate their cursor mechanics to it while retaining protocol-specific encodings, with coverage for both endiannesses, growth, subarray offsets, overruns, alignment, patching, and zero-length operations.
- **`@webnet/state-transfer` — generic state-transfer contract extraction**: Codex (GPT-5.6 Luna) extracted `StateTransferable` and `isStateTransferable` from `@webnet/transport` into a dependency-free workspace package, migrated current DAV/FTP/SMB2/worker consumers, removed the unused transport re-exports, and added guard tests and package metadata.
- **`@webnet/state-transfer` — review follow-up**: Codex (GPT-5.6 Luna) restored the original object-only type-guard semantics, excluded typetests from the production build, added package coverage metadata, and validated the initialized worker and full workspace with the root build, tests, typechecks, and typetests.
- **Root Node/npm engine declaration**: `gpt-5.6-sol` declared Node 24 and npm 11 as the supported root workspace toolchain and synchronized the lockfile metadata.
- **`@webnet/tailshare` - original implementation review**: Initial implementation (PR #28) hand-written, but reviewed by multiple agents: Claude Sonnet 4.6 during writing and Claude Opus 5 and GPT 5.6 Sol before merge.
- **`@webnet/tailshare` — SharedWorker support, type fixes, config consolidation, and fileOps UI**: Claude Code (Claude Sonnet 4.6) rewired the IPN connection in `tailshare` to use `@webnet/tsconnect-worker` via `useBuildIpnWorker`, enabling multiple browser tabs to share a single IPN instance through a SharedWorker (with transparent fallback to main-thread mode when SharedWorker is unavailable). The `IPN` concrete class was replaced with the `IpnClient` interface throughout, and `IpnClientHandle` is used as the return type from the worker hook. The `ipnPrepare` Redux slice was removed entirely and replaced with an `IpnPrepareContext` (React context) that reads/writes a single `tailshare:config` localStorage key as a JSON object — consolidating `ipn#hostname`, `ipn#controlURL`, `ipn#authKey`, `ipn#exitNode`, `ipn#auto` and the new `useWorker` and `fileOps` settings into one key, with explicit defaults (`useWorker: true`, `fileOps: "memory"`) set in `parseConfig` so call sites need no inline fallbacks. The config modal (`TailscaleConfig`) gained a file-storage select (`memory` / `opfs`, with OPFS disabled when SharedWorker is unavailable) and a SharedWorker toggle; the mode indicator shows whether the active connection is `worker` or `main-thread`. In worker mode, selecting OPFS enables `fileOps: true` in the WorkerConfig so received files survive page reloads; in main-thread fallback the storage is always in-memory and a warning is shown if OPFS is selected without a worker. `autostart` remains a tri-state: `undefined` latches to `true` on the first `Running` event (so subsequent page loads auto-connect), `false` permanently disables autostart, and `true` triggers it immediately on load. `TailscaleRoute` shows a spinner rather than a disabled "Enable Tailscale" button during the worker connection window. `@webnet/react` gained a `useSharedWorkerAvailable()` hook (SSR-safe via `useSyncExternalStore`) replacing the inline pattern in `IpnContext.tsx`. `useBuildIpnWorker` in `@webnet/tsconnect-react` was extended to accept `(() => SharedWorker) | null` in its first overload (implementation already handled it). `@webnet/react/tsconfig.json` and `@webnet/tsconnect-react/tsconfig.json` both received `skipLibCheck: true` to resolve a type conflict between `@types/eslint-scope` and `eslint`'s built-in types.
- **`@webnet/tsconnect-react` — initialization error callbacks**: `gpt-5.6-sol` added optional error callbacks to `useBuildIpn` and `useBuildIpnWorker`, covering synchronous builder/run failures and asynchronous worker/fallback connection failures while suppressing callbacks after effect cleanup.
- **`@webnet/vfs` — reusable AsyncVFS conformance suite (issue #80)**: Claude Code (Claude Opus 5, coordinating; Claude Sonnet 5 subagents adopted the suite in the FTP, SFTP and SMB2 packages) added a reusable conformance harness owned by `@webnet/vfs` and exposed from a test-only `@webnet/vfs/conformance` entry point, so production bundles do not acquire test code. `testAsyncVFSConformance({ name, create, capabilities, errorCodes })` takes a factory producing a fresh filesystem per test, a capability descriptor for optional operations (optional methods absent from the instance are skipped rather than failed), and an error-code alias map for protocols that cannot distinguish two `VFSError` codes; the entry point also exports the stream helpers each package had been redefining. The baseline suite covers write/read round trips and multi-chunk streaming, stat and directory listing, mkdir/delete/recursive delete/move/copy/setProps, `readFileRange` inclusive-end semantics, empty files, unicode names, nested paths, the full `VFSError` contract, stream cancellation and write-source error propagation, and sequential plus concurrent operations. MemoryVFS and NodeVFS were migrated as the reference implementations and Drive, FTP, SFTP and SMB2 adopted it, each keeping its protocol-specific tests. The suite surfaced pre-existing bugs in every package it touched, all fixed here: `MemoryVFS.readFile` handed out the entry's own array, which a byte-stream consumer transfers and detaches, so reading a file over WebDAV and cancelling the stream truncated the stored file to zero bytes; `NodeVFS.readFile` on a directory deferred `is-a-directory` to the first stream read and `delete("/", true)` removed the entire served root; the DAV client defaulted `copy`/`move` to overwriting, only honoured a non-recursive `delete` when `recursive` was explicitly `false`, returned an empty listing for `readdir` of a file, and merged rather than replaced in `setProps`, while the DAV server reported `is-a-directory` as 409 instead of RFC 4918's 405; the FTP client silently clobbered on a default `move` and discarded the server's explicit "not a directory"/"is a directory" replies; the SFTP client guessed error codes from the request verb instead of reading the code its own server embeds, and opening a directory for reading succeeded; the SMB2 client accepted deleting the share root and ignored `opts.overwrite` in `copy` (which also could not copy directories), and its mock server ignored `CreateOptions` directory flags and `ReplaceIfExists`. Three of the six packages defaulted a destructive operation to overwriting, which is the contract drift the issue describes. The harness compares large payloads byte-wise rather than with `assert.deepEqual`, because building a diff between a 128 KiB array and an empty one exhausts memory and kills the process before the failure can be reported. The SFTP client was additionally verified against a real OpenSSH `sshd` to confirm the status-parsing change still falls back correctly for third-party servers. Two independent autonomous reviews followed — Claude Sonnet 5 and Codex (GPT-5.6) — and both were addressed by Claude Opus 5: the suite had lost `readFileRange`'s error contract when the per-package tests were removed and never covered `copy` replacing an existing directory (which `NodeVFS` merged into instead); SMB2 `copy` could recurse without bound when the destination lay inside the source and could not overwrite a destination of the other type; the DAV client read every 405 as `is-a-directory` although RFC 4918 only gives it that meaning for GET and PUT; the FTP server relayed the backing filesystem's prose as 550 text and now sends conventional wording per code, which removed the need for the `forbidden`/`not-found` alias the reviewers correctly identified as covering a fixable bug rather than a protocol limit; and SFTP now marks the VFS code it embeds in `SSH_FX_FAILURE` messages so a third-party server's prose cannot be decoded as one of ours.
- **CI — bounded Node heap (issue #136)**: Claude Code (Claude Opus 5) added a workflow-level `NODE_OPTIONS: --max-old-space-size=4096` to `.gitea/workflows/ci.yml`, so every CI job that runs Node (including the per-file workers `node --test` spawns and Turbo-invoked package scripts, which inherit the variable) fails with a legible V8 heap-limit error instead of growing until the runner's own memory limit kills the task and any jobs sharing the machine.
+58 -28
View File
@@ -1,10 +1,20 @@
# webnet
A TypeScript monorepo for transport-based networking, anchored by a WebAssembly [Tailscale](https://tailscale.com) SDK. It provides a layered stack of packages — from raw transport abstractions up through HTTP, WebSocket, and WebDAV — that work in browsers, Node.js, and any environment that can supply a transport.
A TypeScript monorepo for transport-based networking, anchored by a WebAssembly [Tailscale](https://tailscale.com) SDK. It provides a layered stack of packages — from raw transport and virtual filesystem abstractions through HTTP, WebSocket, WebDAV, FTP, SSH, SFTP, SMB, and Taildrive — that work in browsers, Node.js, and any environment that can supply a transport.
Those packages can be composed into applications. [**Tailshare**](packages/tailshare) is the repository's flagship application.
## Tailshare
[`packages/tailshare`](packages/tailshare) is the flagship application of this repository and its primary first-party consumer. It is a browser app that joins a tailnet directly from the page and moves files across it. Tailshare combines Tailscale connectivity over WebAssembly, filesystems exposed through protocol packages, and peer-to-peer transfers without a server in the middle.
The other packages are reusable libraries. Tailshare sets first-party product priorities, but library work can land before Tailshare integrates it. A protocol counts as product breadth only after Tailshare exposes it through the common file experience and browser tests cover the integration. `example-app` and `test-app` support development and protocol testing; they are not the product.
Tailshare is under active development. Its README describes the current application, while the [roadmap](docs/ROADMAP.md) defines product direction and sequencing.
## Inspiration
This is heavily inspired by the [WebVM](https://webvm.io/) networking stack that levrages Tailscale in the browser.
This is heavily inspired by the [WebVM](https://webvm.io/) networking stack that leverages Tailscale in the browser.
Note that this doesn't lift any code from that project, only ideas.
Another inspiration is the [ElysiaJS](https://elysiajs.com/) documentation that seemingly allows running a webserver from the docs directly in the browser (even if it is just a feature of the framework itself and not real networking).
@@ -14,29 +24,46 @@ Another inspiration is the [ElysiaJS](https://elysiajs.com/) documentation that
Tailscale already ships a `tsconnect` package that compiles the IPN (in-process networking) stack to WASM via `GOARCH=wasm`. This repo builds on top of that with a layered set of packages:
1. **Patching tsconnect**: the `tailscale` submodule tracks a fork on the `webnet` branch that extends the Go-to-JS bridge (`wasm_js.go`) to expose lower-level networking primitives: raw TCP/UDP connections, ICMP, TLS dialing, and TCP listening.
2. **`@webnet/transport`**: declares the `RawTransport`, `RawListener`, and `RawDialer` interfaces, plus buffer utilities and transport implementations that have no external dependencies — a loopback transport and a Node.js streams adapter.
3. **`@webnet/state-transfer`**: provides the generic `StateTransferable` ownership-transfer contract and runtime type guard without transport or protocol dependencies.
4. **`@webnet/tsconnect`**: builds the WASM artifact, ships it alongside a Mozilla CA bundle and `wasm_exec.js`, and wraps the raw JS bridge in typed TypeScript classes. Its `Conn`, `TCPListener`, and `IPNDialer` implement the `@webnet/transport` interfaces, making it a drop-in transport source for the rest of the stack.
5. **`@webnet/tsconnect-redux`** / **`@webnet/tsconnect-react`**: Redux Toolkit (RTK) slice and React hooks/context for IPN state management and control, extracted from the core SDK so consumers can bring their own UI framework.
6. **`@webnet/http`**: a full HTTP/1.1 client and server over any `@webnet/transport` implementation. Features include request/response streaming, chunked transfer encoding, keep-alive, a client connection pool, automatic redirect following, and a Koa-inspired middleware router.
7. **`@webnet/websocket`**: WebSocket client and server built on `@webnet/http`. Handles the upgrade handshake, frame codec, masking, fragmented-message reassembly, ping/pong, and the close handshake — no external dependencies.
8. **`@webnet/drive`**: WebDAV Level 1 (and optionally Level 2) client and server built on `@webnet/http`. Includes an async VFS abstraction with `MemoryVFS`, `NodeVFS`, and `FsaVFS` (with an OPFS factory method) implementations, a `createDAVHandler()` server handler, and a `DAVClient` that itself implements `AsyncVFS`.
2. **`@webnet/binary`**: provides dependency-free binary readers and writers shared by the SSH, SFTP, and SMB protocol implementations.
3. **`@webnet/transport`**: declares the `RawTransport`, `RawListener`, and `RawDialer` interfaces, plus buffer utilities and transport implementations for loopback, Node.js streams, and WebRTC.
4. **`@webnet/state-transfer`**: provides the generic `StateTransferable` ownership-transfer contract and runtime type guard without transport or protocol dependencies.
5. **`@webnet/tsconnect`**: builds the WASM artifact, ships it alongside a Mozilla CA bundle and `wasm_exec.js`, and wraps the raw JS bridge in typed TypeScript classes. Its `Conn`, `TCPListener`, and `IPNDialer` implement the `@webnet/transport` interfaces, making it a drop-in transport source for the rest of the stack.
6. **`@webnet/tsconnect-redux`** / **`@webnet/tsconnect-react`**: Redux Toolkit (RTK) slice and React hooks/context for IPN state management and control, extracted from the core SDK so consumers can bring their own UI framework.
7. **`@webnet/tsconnect-worker`**: runs the IPN in a `SharedWorker`, synchronizes state through Redux, and can fall back to the main thread when workers are unavailable.
8. **`@webnet/vfs`**: defines the shared async virtual filesystem used by the file protocols, with in-memory, Node.js, and File System Access API implementations.
9. **Protocol packages**: client and server implementations for HTTP, WebSocket, WebDAV, FTP/FTPS, SSH, and SFTP are built on the transport and VFS layers, alongside an SMB client. `@webnet/http-static` serves a VFS over HTTP, while `@webnet/taildrive` connects WebDAV shares to Tailscale peers.
10. **Apps and UI helpers**: reusable browser and React utilities support the example and test applications, and above all **Tailshare**, the flagship app that composes the whole stack into a usable product.
### Packages
| Package | Description |
| -------------------------- | ------------------------------------------------------------------------------------------------------- |
| `packages/transport` | Transport interfaces (`RawTransport`, `RawListener`, `RawDialer`), loopback and Node.js implementations |
| `packages/state-transfer` | Generic ownership-transfer contract and runtime type guard |
| `packages/tsconnect` | Tailscale WASM SDK — IPN lifecycle, typed TS wrappers, transport implementation |
| `packages/tsconnect-redux` | RTK slice and thunks for IPN state management and control |
| `packages/tsconnect-react` | React hooks and context for IPN state management and control |
| `packages/http` | HTTP/1.1 client and server, connection pool, redirect following, Koa-inspired router |
| `packages/websocket` | WebSocket client and server based on `@webnet/http` |
| `packages/drive` | WebDAV (Level 1 + optional Level 2) client and server based on `@webnet/http` |
| `packages/xml` | Thin XML parse/serialize with conditional exports (native DOM / `@xmldom/xmldom`) |
| `packages/test-app` | Vite dev app for manual browser testing |
| `packages/example-app` | Example app demonstrating the full stack |
Each package has its own README with entry points and usage; the table links to them.
| Package | Description |
| ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- |
| [`packages/tailshare`](packages/tailshare) | **Flagship application**: tailnet file sharing in the browser |
| [`packages/binary`](packages/binary) | DOM-independent binary readers and writers for protocol serialization |
| [`packages/transport`](packages/transport) | Transport interfaces with loopback, Node.js streams, and WebRTC implementations |
| [`packages/state-transfer`](packages/state-transfer) | Generic ownership-transfer contract and runtime type guard |
| [`packages/tsconnect`](packages/tsconnect) | Tailscale WASM SDK, IPN lifecycle, typed wrappers, and transport implementation |
| [`packages/tsconnect-worker`](packages/tsconnect-worker) | SharedWorker and main-thread bindings for `@webnet/tsconnect` with Redux state synchronization |
| [`packages/tsconnect-redux`](packages/tsconnect-redux) | Redux Toolkit slice, bindings, actions, and selectors for IPN state management |
| [`packages/tsconnect-react`](packages/tsconnect-react) | React hooks and context for IPN state management and control |
| [`packages/vfs`](packages/vfs) | Async virtual filesystem interface, implementations, fallbacks, and conformance tests |
| [`packages/http`](packages/http) | HTTP/1.1 client and server, connection pool, redirect following, and middleware router |
| [`packages/http-static`](packages/http-static) | Static file HTTP handler backed by an async VFS |
| [`packages/websocket`](packages/websocket) | WebSocket client and server based on `@webnet/http` |
| [`packages/webdav`](packages/webdav) | WebDAV Level 1 and optional Level 2 client and server based on `@webnet/http` |
| [`packages/ftp`](packages/ftp) | FTP/FTPS client and server backed by an async VFS |
| [`packages/ssh`](packages/ssh) | SSH-2 client and server connections, channels, sessions, and TCP forwarding |
| [`packages/sftp`](packages/sftp) | SFTP client and server backed by an async VFS |
| [`packages/smb2`](packages/smb2) | SMB2/3 client implementing the async VFS interface |
| [`packages/taildrive`](packages/taildrive) | Taildrive peer discovery, a composite filesystem over peer shares, and server bridge |
| [`packages/xml`](packages/xml) | XML parsing and serialization with browser-native and Node.js backends |
| [`packages/react`](packages/react) | Shared React hooks, error display, and browser capability helpers |
| [`packages/utils`](packages/utils) | Browser upload, download, blob-reading, and formatting utilities |
| [`packages/browser-test-utils`](packages/browser-test-utils) | Shared Playwright utilities for browser integration tests |
| [`packages/example-app`](packages/example-app) | Example app demonstrating the browser stack |
| [`packages/test-app`](packages/test-app) | Vite app for manual and end-to-end protocol testing |
### Submodules
@@ -49,7 +76,10 @@ git submodule update --init tailscale
## Development
```bash
# Build the WASM and TypeScript declarations
# Build the WASM assets
npm run build-go --workspace=packages/tsconnect
# Build the TypeScript package
npm run build --workspace=packages/tsconnect
# Start the test app
@@ -84,11 +114,11 @@ symbols from the package that defines them. Wildcard exports are also forbidden
so API changes remain explicit and reviewable.
`_internals` entry points are intentionally unstable and carry no semantic
versioning compatibility guarantee. They are available only to their owning
package's tests and tooling. Cross-package `_internals` imports are forbidden in
both production and test code; tests should use public APIs or package-local
fixtures instead.
versioning compatibility guarantee. Cross-package `_internals` imports are
discouraged. If one is unavoidable, the importing package must pin the dependency
to an exact version because the entry point can change in any release. Prefer a
stable or experimental export, or use a package-local fixture in tests.
## AI disclosure
See [AI_CHANGES.md](AI_CHANGES.md) for the full log of AI-assisted and AI-authored work in this repository.
Most of the code in this repository was AI-generated. The entire patch set in the Tailscale fork under `tailscale/` was also AI-generated.
+101
View File
@@ -0,0 +1,101 @@
# Webnet language
This glossary defines the terms Webnet uses for its product, file access, security, and work planning. Use these terms in code, documentation, and issues when the distinction matters.
## Product and file access
**Tailshare**:
The flagship Webnet application. Tailshare turns the repository's networking and filesystem packages into browser-based file access for Tailscale users.
**Protocol**:
A network protocol implementation that does not depend on Tailshare's interface. HTTP, WebDAV, FTP, SFTP, and SMB are protocols.
**Source type**:
A Tailshare integration for one kind of filesystem access. A source type defines how Tailshare configures and connects its sources.
_Avoid_: protocol, plugin
**Source**:
A temporary or saved instance of a source type. A source keeps its identity when its name or settings change.
_Avoid_: drive, connection, filesystem
**Filesystem**:
A connected hierarchy of files and directories exposed by a source. Webnet filesystem packages implement the `AsyncVFS` contract.
_Avoid_: source, protocol
**Location**:
A path within a filesystem.
_Avoid_: source, endpoint
**Taildrive peer**:
A Tailscale peer that advertises at least one Taildrive share.
**Taildrive share**:
A filesystem hierarchy that a Taildrive peer exposes through its peer API.
**Tailshare storage**:
App-owned storage in the browser's origin-private filesystem. It is not visible in the device's native file manager, and clearing site data deletes it.
_Avoid_: local folder, local storage, OPFS source
**Local folder**:
A device directory that the user grants Tailshare permission to access. Browser support and permission persistence determine whether the folder remains available.
_Avoid_: device folder, Tailshare storage, local source
**Selected file**:
A file that the user gives Tailshare for one operation. A selected file is not a source unless Tailshare imports it into a filesystem.
_Avoid_: local source
## Source state and security
**Lifecycle state**:
The current connection state of a source, such as disconnected, connecting, ready, unavailable, or error.
_Avoid_: blocker
**Blocker**:
The reason a source cannot become ready, such as a locked vault, missing browser permission, failed authentication, or required security confirmation.
_Avoid_: lifecycle state, error
**Vault**:
The encrypted store for saved source secrets. One Tailshare origin and browser profile has one vault.
**App password**:
The password that unlocks the vault key. The app password is not a source credential.
_Avoid_: source password, encryption key
**Security evidence**:
An observed fact that transport or protocol code reports to Tailshare, such as validated TLS, an authenticated Tailscale peer, a verified SSH host key, or trusted loopback provenance. Tailshare evaluates the evidence before the vault releases a secret.
**Security expectation**:
The saved endpoint and identity requirements that a live source connection must satisfy.
_Avoid_: last successful connection
**Insecure consent**:
Explicit user approval for a source connection that does not satisfy its normal security expectation. Consent lasts only for the current session, is not persisted, and is bound to the source and the relevant endpoint or identity.
## Planning
**Product milestone**:
A product outcome that contains one or more themes. Before Webnet 1.0, a product milestone can span several Gitea milestones.
_Avoid_: issue, release
**Theme**:
The coherent user path or technical increment that receives current project focus. A Gitea milestone represents the active theme before Webnet 1.0.
_Avoid_: epic, umbrella issue
**Issue**:
An independently implementable and testable unit of work.
_Avoid_: theme, checklist
**Pull request**:
A reviewable delivery step for an issue. An issue can require more than one pull request.
_Avoid_: issue, theme
## API stability
**Stable export**:
A documented public entry point covered by the package's semantic-version compatibility promise.
**Experimental export**:
A separate public entry point that can make incompatible changes in a semantic-version minor release. Experimental changes still appear in the changelog.
**Internal export**:
An `_internals` entry point that can change in any release. Internal changes still appear in the changelog.
+188
View File
@@ -0,0 +1,188 @@
# Webnet roadmap
Webnet exists to make Tailshare useful. The packages remain reusable libraries, but Tailshare decides which vertical work receives first-party focus.
This document records direction and sequencing. It does not track live issue status. The active Gitea milestone owns the current theme, and Gitea issues own implementation details and acceptance criteria.
## Product direction
Tailshare serves Tailscale power users who need to find and open files across services from a web-first or mobile device. The first complete workflow is browse, preview, and download. Search, mutation, editing, and broad protocol coverage follow that workflow.
Libraries can land without Tailshare integration. A protocol does not count as product breadth until Tailshare exposes it through the common file experience and browser tests cover the integration.
## Planning model
Before Webnet 1.0, one Gitea milestone represents the active theme. A product milestone can contain several themes. After 1.0, a theme and a milestone can use the same boundary when that makes planning clearer.
One theme receives active first-party focus. P0 issues, P1 security defects, and active regressions can interrupt it. Other P1 and P2 issues remain important queue items and enter work through normal triage.
Broad checklist issues are temporary planning artifacts. Replace them with narrow issues when implementation begins. Use projects, milestones, or external planning tools to track the larger outcome.
## Files product milestone
The files milestone makes Tailshare useful for regular file access. It is complete when Tailshare provides all of these outcomes:
- built-in Taildrive and configurable WebDAV sources;
- a shared source registry and file browser;
- list view with name and size, sorting, breadcrumbs, and routed locations;
- safe previews for images, text, PDF, and browser-supported audio and video;
- download, platform sharing, and copied Tailshare links;
- responsive desktop and mobile workflows;
- temporary and saved WebDAV configurations;
- vault-backed secret storage and live connection security checks;
- clear lifecycle states, blockers, recovery actions, and technical diagnostics;
- cancellation for connection, listing, preview, download, and sharing;
- browser CI for Taildrive and WebDAV in supported execution modes;
- one week of regular maintainer use; and
- successful task-based testing by three Tailscale power users.
The first milestone does not include search, favorites, recents, multi-select, mutation, editing, visible Tailshare storage, Local folder access, offline file pinning, another protocol source, or cross-device source links.
### First theme: download one Taildrive file
The first theme ends when a user can complete this path:
1. Open Tailshare.
2. Log in to Tailscale.
3. Open Files.
4. Select a Taildrive peer and share.
5. Browse to a file.
6. Download the file.
The technical path includes the Tailshare shell, Tailscale login, `tsconnect`, transport, HTTP, WebDAV, Taildrive, the VFS read contract, the file browser, and download handling. Work outside that path belongs to a later theme unless it fixes an urgent security defect or regression.
Implement the theme in this order:
1. Define source records, source types, lifecycle states, blockers, and capability handling.
2. Add the built-in Taildrive source with a small manual inspection surface.
3. Build the routed file browser against the real Taildrive source.
4. Add download behavior, cancellation, loading states, failure recovery, and browser coverage.
5. Use the completed path regularly before opening the next theme.
Tailscale enrollment keys must not persist by default before Tailshare reaches test users. That work remains separate from the source-secret vault because the credentials have different lifecycles.
### Later files themes
The remaining files work stays in the same product milestone but moves through separate themes:
- add the generic WebDAV source type and the credential broker;
- add preview, download, and platform sharing behavior for the agreed file types;
- add saved sources, strict hosted Content Security Policy, and the encrypted vault;
- add browser CI and responsive workflow coverage;
- complete maintainer dogfooding and power-user testing.
The generic WebDAV source type uses runtime configuration schemas and migrations. It releases secrets only after the live connection satisfies the source's saved security expectation or the user grants the required insecure consent.
## Work after the files milestone
Choose the next theme after reviewing the working product. The likely choices are:
- add SMB2 to prove that a source integration, including tests, fits within one working day; or
- add Tailshare storage, import, and cross-source copy when missing local persistence blocks regular use.
The decision remains intentionally open. The state of the product matters more than a fixed ordering made before dogfooding.
Later themes can include:
- favorites, recents, and search;
- mutation and cross-source operations;
- text and source editing;
- Local folder access where the browser supports it;
- offline file pinning;
- FTP, SFTP, and other protocol sources;
- serving files over WebDAV, Tailnet, Funnel, or Taildrive;
- WebRTC transfer acceleration;
- PWA and static-generation work; and
- Node hosting and server rendering.
## Source integration rules
Tailshare registers source types statically from its `sources/` directory. Lightweight metadata loads with the application. Registration components, protocol packages, and connectors can load on demand.
Saved sources connect when the user opens them. Tailshare links contain a stable source ID and a location. They work within one browser profile and Tailshare installation. Hosted builds keep this navigation state in the URL fragment so file paths do not enter server logs. Device-hosted builds can use normal URL paths. The build selects the routing mode.
Each source type owns:
- a versioned settings type;
- a runtime JSON Schema or equivalent validator;
- settings migrations;
- secret declarations;
- a registration component; and
- a connect function.
Lossless migrations can run automatically. Destructive, security-sensitive, or irreversible migrations require confirmation. A failed migration keeps the prior record intact and disables the source with an explanation.
Adding a source type for an implemented protocol should take a few hours of production work. The component, settings, secrets, connection logic, registration, capability behavior, focused tests, and one browser integration test should fit within one working day. The common file browser must not need protocol-specific changes.
## Security direction
Tailshare separates source settings from source secrets. A permanent source asks how to store each secret. If an app password exists, encrypted storage is the default. Without an app password, the user must choose memory-only storage, unencrypted storage, or app-password setup.
One vault belongs to one Tailshare origin and browser profile. A random vault key encrypts saved secrets. A password-derived key wraps the vault key. The first format uses versioned PBKDF2 parameters. Before public 1.0, benchmark Argon2id in the production worker and on mobile hardware. Review its implementation and confirm that it works under the strict Content Security Policy. Use Argon2id by default only if it passes those checks.
The vault worker owns the unwrapped vault key and the idle timer. Manual lock is global. Locking erases the key, disconnects sources that used encrypted secrets, clears their directory data and previews, and updates every tab.
Protocol packages expose deferred credential hooks at their actual trust boundary. Tailshare owns policy evaluation and secret release. The policy uses observed security evidence and the source's saved security expectation. Automatic reconnect never becomes an automatic downgrade.
Hosted Tailshare requires a strict Content Security Policy before it stores permanent secrets. Early private builds can test the complete workflow. Selected public testers can use session-only secrets until the persistent-secret gate passes.
## Quality gates
Every network operation in the first files workflow has a loading state and supports cancellation. Progress is required before beta when Tailshare owns or buffers a transfer. Browser-owned streaming can rely on native progress until 1.0. Before 1.0, every transfer has either Tailshare-owned or browser-owned progress.
Source failures preserve the current route, clear partial resources, and provide retry or reconfiguration without affecting other sources. Diagnostics use a bounded in-memory log. Credentials never enter diagnostic output. Paths, endpoints, and peer identity require separate export consent.
Tailshare sends no automatic telemetry before 1.0. Tester validation uses a common task script, structured notes, unstructured feedback, and unprompted use after the scripted task.
Browser support follows a tested feature matrix and aims at evergreen browsers. Beta requires the core workflow on real Android hardware. iOS remains expected but unverified until real-device evidence exists. Automated WebKit and manually verified Safari are separate matrix entries.
Before public 1.0, define and apply an accessibility checklist for keyboard use, labels, focus, contrast, touch targets, and practical screen-reader checks. Do not claim formal standards conformance without evidence.
## Release stages
Webnet uses these release stages:
- **Internal alpha**: private hosted Tailshare builds and `-pre` packages in the Gitea registry.
- **Tester preview**: a public static build shared with selected users. Persistent secrets remain disabled until the strict Content Security Policy and vault are ready.
- **Beta**: the complete files milestone, saved encrypted sources, browser CI, and documented support evidence.
- **Public 1.0**: the required stable packages reach 1.0, SMB2 proves the integration contract, three power users complete the core workflow, and Webnet is announced publicly.
Public 1.0 is criteria-driven and has no announced date.
## Package stability and publication
A package reaches 1.0 when Webnet documents it for direct use or its exported types form part of another 1.0 package's public contract. Audit the candidate package set before beta. Hide accidental exports and record why other public packages remain at 0.x.
Stable exports follow semantic-version compatibility. Experimental entry points can break in a minor release. `_internals` entry points can break in any release. Changelogs include changes to all three categories.
Internal Webnet dependencies use ranges based on the imported entry point:
- allowed dependencies on `_internals` use exact versions;
- dependencies on experimental entry points allow patch releases only; and
- dependencies on stable entry points use the broadest range justified by semantic compatibility.
Each protocol package documents client and server support separately. The support matrix records runtime support, protocol versions and features, external implementations and versions, and known limits. Matrix entries distinguish automatically verified, manually verified with a date and version, expected but unverified, known nonworking, and unsupported. An entry can also be partial.
Webnet normally deprecates stable APIs and supplies migration guidance before removal. Removal occurs only in a semantic-version major release, but Webnet does not guarantee a minimum deprecation period. Documentation-only releases can give advance warning before an implementation change.
Before 1.0, publish a `SECURITY.md` with a private reporting method and the package versions that receive security fixes. Coordinate disclosure where possible. Do not promise a response time that the project cannot staff.
The initial Gitea registry can carry clearly marked `-pre` development packages while the verification pipeline is under construction. The project README and the Gitea release page must identify those packages as unchecked development work.
After the verification pipeline exists, a protected tag authorizes release of an exact commit whose required CI has passed. The first release job builds the packages once and uploads immutable tarballs with their checksums and toolchain versions. A separate job checks the tarball contents, installs them, and runs import smoke tests. It also rejects a package that imports another workspace's `_internals` entry point without an exact dependency version. Only then can the publication job send those same bytes to Gitea and npm. npm publication remains disabled until this pipeline exists.
Choose release tooling after testing independent versions, dependency-range updates, per-package changelogs, prereleases, Gitea Actions, and identical-artifact promotion.
Audit code provenance before assigning package licenses. The expected result is BSD-3-Clause for packages derived from Tailscale code, MIT for other Webnet packages, and the existing MPL-2.0 terms for the CA bundle. Preserve third-party license notices in published artifacts.
## Open decision gates
These decisions wait for evidence:
- select a release tool after a focused comparison;
- choose Argon2id parameters only after worker and mobile benchmarks;
- decide whether CI services run per job or on shared test infrastructure after measuring cost;
- define the accessibility checklist before public 1.0;
- choose the post-files theme after dogfooding;
- decide the cancellation issue split after reviewing the common contract; and
- revisit the Gitea milestone hierarchy when Redmine integration or post-1.0 planning makes another model more useful.
+88 -16
View File
@@ -1,15 +1,53 @@
import js from "@eslint/js"
import fs from "node:fs"
import path from "node:path"
import { fileURLToPath } from "node:url"
import tseslint from "typescript-eslint"
const repositoryRoot = path.dirname(fileURLToPath(import.meta.url))
const packageManifestCache = new Map()
function packageOwner(filePath) {
const parts = path.relative(repositoryRoot, filePath).split(path.sep)
return parts[0] === "packages" ? parts[1] : undefined
}
function internalPackage(specifier) {
return /^(@webnet\/[^/]+)\/(?:.*\/)?_internals(?:\.js)?$/.exec(specifier)?.[1]
}
function unwrapTypeExpression(node) {
while (
node?.type === "TSAsExpression" ||
node?.type === "TSTypeAssertion" ||
node?.type === "TSNonNullExpression" ||
node?.type === "TSSatisfiesExpression"
)
node = node.expression
return node
}
function staticSpecifier(node) {
node = unwrapTypeExpression(node)
if (node?.type === "Literal" && typeof node.value === "string") return node.value
if (node?.type === "TemplateLiteral" && node.expressions.length === 0)
return node.quasis[0]?.value.cooked
}
function exactVersion(range) {
return /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$/.test(range)
}
function packageManifest(owner) {
let manifest = packageManifestCache.get(owner)
if (manifest !== undefined) return manifest
manifest = JSON.parse(
fs.readFileSync(path.join(repositoryRoot, "packages", owner, "package.json"), "utf8"),
)
packageManifestCache.set(owner, manifest)
return manifest
}
const packageBoundary = {
rules: {
"no-relative-source-escapes": {
@@ -46,6 +84,55 @@ const packageBoundary = {
}
},
},
"exact-internal-dependencies": {
meta: {
type: "problem",
schema: [],
messages: {
exact:
"Cross-package _internals imports require {{target}} to use an exact dependency version in {{source}}; found {{range}}.",
},
},
create(context) {
const sourceOwner = packageOwner(context.filename)
if (sourceOwner === undefined) return {}
const manifest = packageManifest(sourceOwner)
function check(node, source = node.source) {
const specifier = staticSpecifier(source)
if (specifier === undefined) return
const target = internalPackage(specifier)
if (target === undefined || target === manifest.name) return
const ranges = [
manifest.dependencies?.[target],
manifest.devDependencies?.[target],
manifest.optionalDependencies?.[target],
manifest.peerDependencies?.[target],
].filter((range) => range !== undefined)
if (ranges.length > 0 && ranges.every(exactVersion)) return
context.report({
node,
messageId: "exact",
data: {
target,
source: manifest.name,
range: ranges.length > 0 ? ranges.join(", ") : "no dependency declaration",
},
})
}
return {
ImportDeclaration: check,
ImportExpression: check,
TSImportType: check,
CallExpression(node) {
const callee = unwrapTypeExpression(node.callee)
if (callee.type === "Identifier" && callee.name === "require")
check(node, node.arguments[0])
},
}
},
},
},
}
@@ -66,18 +153,7 @@ export default tseslint.config(
],
"@typescript-eslint/no-explicit-any": "warn",
"package-boundary/no-relative-source-escapes": "error",
"no-restricted-imports": [
"error",
{
patterns: [
{
group: ["@webnet/**/_internals", "@webnet/**/_internals.js"],
message:
"Cross-package _internals imports are forbidden, including in tests. Use the package's public API.",
},
],
},
],
"package-boundary/exact-internal-dependencies": "error",
"no-restricted-syntax": [
"error",
{
@@ -89,10 +165,6 @@ export default tseslint.config(
message:
"Cross-package re-exports are forbidden. Consumers should import from the owning package.",
},
{
selector: "ImportExpression[source.value=/^@webnet\\/.*\\/_internals(?:\\.js)?$/]",
message: "Cross-package _internals imports are forbidden. Use the package's public API.",
},
],
},
},
+49 -5
View File
@@ -1,4 +1,5 @@
import assert from "node:assert/strict"
import fs from "node:fs/promises"
import path from "node:path"
import { test } from "node:test"
import { ESLint } from "eslint"
@@ -14,7 +15,7 @@ async function messages(
return result.messages.map(({ message, ruleId }) => ({ message, ruleId }))
}
test("rejects cross-package internal imports", async () => {
test("rejects non-exact cross-package internal dependencies", async () => {
for (const specifier of [
"@webnet/ssh/_internals",
"@webnet/ssh/_internals.js",
@@ -22,13 +23,56 @@ test("rejects cross-package internal imports", async () => {
"@webnet/transport/loopback/_internals.js",
]) {
const result = await messages(`import { Reader } from "${specifier}"; void Reader`)
assert.ok(result.some(({ ruleId }) => ruleId === "no-restricted-imports"))
assert.ok(
result.some(
({ message, ruleId }) =>
ruleId === "package-boundary/exact-internal-dependencies" && message.includes("found *"),
),
)
}
})
test("rejects dynamic cross-package internal imports", async () => {
const result = await messages('await import("@webnet/ssh/_internals.js")')
assert.ok(result.some(({ ruleId }) => ruleId === "no-restricted-syntax"))
test("rejects other internal imports from non-exact dependencies", async () => {
for (const source of [
'await import("@webnet/ssh/_internals.js")',
"await import(`@webnet/ssh/_internals.js`)",
'await import("@webnet/ssh/_internals.js" as string)',
'await import("@webnet/ssh/_internals.js" satisfies string)',
'type Reader = import("@webnet/ssh/_internals").Reader; let reader: Reader; void reader',
'import { createRequire } from "node:module"; const require = createRequire(import.meta.url); require("@webnet/ssh/_internals")',
'import { createRequire } from "node:module"; const require = createRequire(import.meta.url); (require as typeof require)("@webnet/ssh/_internals")',
'import { createRequire } from "node:module"; const require = createRequire(import.meta.url); (<typeof require>require)("@webnet/ssh/_internals")',
'import { createRequire } from "node:module"; const require = createRequire(import.meta.url); require!("@webnet/ssh/_internals")',
'import { createRequire } from "node:module"; const require = createRequire(import.meta.url); (require satisfies typeof require)("@webnet/ssh/_internals")',
]) {
const result = await messages(source)
assert.ok(
result.some(
({ message, ruleId }) =>
ruleId === "package-boundary/exact-internal-dependencies" && message.includes("found *"),
),
)
}
})
test("allows internal imports from exact dependencies", async () => {
const fixture = await fs.mkdtemp(path.resolve("packages/package-boundary-"))
try {
await fs.writeFile(
path.join(fixture, "package.json"),
JSON.stringify({
name: "@webnet/package-boundary-fixture",
dependencies: { "@webnet/ssh": "0.1.0" },
}),
)
const result = await messages(
'import { createRequire } from "node:module"; import { Reader } from "@webnet/ssh/_internals"; type InternalReader = import("@webnet/ssh/_internals").Reader; const require = createRequire(import.meta.url); void Reader; void (null as unknown as InternalReader); await import(`@webnet/ssh/_internals.js`); require("@webnet/ssh/_internals")',
path.join(fixture, "src/check.ts"),
)
assert.deepEqual(result, [])
} finally {
await fs.rm(fixture, { recursive: true })
}
})
test("rejects relative cross-package source escapes", async () => {
+45 -11
View File
@@ -3540,12 +3540,16 @@
"@xtuc/long": "4.2.2"
}
},
"node_modules/@webnet/binary": {
"resolved": "packages/binary",
"link": true
},
"node_modules/@webnet/browser-test-utils": {
"resolved": "packages/browser-test-utils",
"link": true
},
"node_modules/@webnet/drive": {
"resolved": "packages/drive",
"node_modules/@webnet/webdav": {
"resolved": "packages/webdav",
"link": true
},
"node_modules/@webnet/example-app": {
@@ -10167,6 +10171,30 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"packages/binary": {
"name": "@webnet/binary",
"version": "0.1.0",
"devDependencies": {
"@types/node": "^25.6.0",
"c8": "^11.0.0",
"tsx": "^4.21.0",
"typescript": "^6.0.2"
}
},
"packages/binary/node_modules/typescript": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz",
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"packages/browser-test-utils": {
"name": "@webnet/browser-test-utils",
"version": "0.1.0",
@@ -10192,8 +10220,8 @@
"node": ">=14.17"
}
},
"packages/drive": {
"name": "@webnet/drive",
"packages/webdav": {
"name": "@webnet/webdav",
"version": "0.1.0",
"dependencies": {
"@webnet/http": "*",
@@ -10209,7 +10237,7 @@
"typescript": "^6.0.2"
}
},
"packages/drive/node_modules/typescript": {
"packages/webdav/node_modules/typescript": {
"version": "6.0.3",
"dev": true,
"license": "Apache-2.0",
@@ -10383,9 +10411,9 @@
"name": "@webnet/sftp",
"version": "0.1.0",
"dependencies": {
"@webnet/binary": "*",
"@webnet/ssh": "*",
"@webnet/transport": "*",
"@webnet/utils": "*",
"@webnet/vfs": "*"
},
"devDependencies": {
@@ -10413,9 +10441,9 @@
"name": "@webnet/smb2",
"version": "0.1.0",
"dependencies": {
"@webnet/binary": "*",
"@webnet/state-transfer": "*",
"@webnet/transport": "*",
"@webnet/utils": "*",
"@webnet/vfs": "*"
},
"devDependencies": {
@@ -10443,8 +10471,8 @@
"name": "@webnet/ssh",
"version": "0.1.0",
"dependencies": {
"@webnet/transport": "*",
"@webnet/utils": "*"
"@webnet/binary": "*",
"@webnet/transport": "*"
},
"devDependencies": {
"@types/node": "^25.6.0",
@@ -10495,11 +10523,14 @@
"name": "@webnet/taildrive",
"version": "0.1.0",
"dependencies": {
"@webnet/webdav": "*",
"@webnet/http": "*",
"@webnet/transport": "*",
"@webnet/tsconnect": "*"
},
"devDependencies": {
"@types/node": "^25.6.0",
"@webnet/vfs": "*",
"c8": "^11.0.0",
"tsx": "^4.21.0",
"typescript": "^6.0.2"
@@ -10528,7 +10559,7 @@
"@mantine/notifications": "^9.3.1",
"@phosphor-icons/react": "^2.1.10",
"@reduxjs/toolkit": "^2.11.2",
"@webnet/drive": "*",
"@webnet/webdav": "*",
"@webnet/http": "*",
"@webnet/react": "*",
"@webnet/tsconnect": "*",
@@ -10549,8 +10580,10 @@
"@babel/preset-react": "^7.27.1",
"@babel/preset-typescript": "^7.27.1",
"@pmmmwh/react-refresh-webpack-plugin": "^0.5.15",
"@types/node": "^25.6.0",
"@types/react": "^19.1.4",
"@types/react-dom": "^19.1.4",
"@webnet/browser-test-utils": "*",
"babel-loader": "^9.2.1",
"css-loader": "^7.1.2",
"html-webpack-plugin": "^5.6.3",
@@ -10562,6 +10595,7 @@
"sass": "^1.89.0",
"sass-loader": "^16.0.5",
"style-loader": "^4.0.0",
"tsx": "^4.21.0",
"typescript": "^6.0.2",
"webpack": "^5.99.9",
"webpack-cli": "^5.1.4",
@@ -10584,7 +10618,7 @@
"name": "@webnet/test-app",
"version": "0.1.0",
"dependencies": {
"@webnet/drive": "*",
"@webnet/webdav": "*",
"@webnet/ftp": "*",
"@webnet/http": "*",
"@webnet/sftp": "*",
+12 -10
View File
@@ -11,18 +11,20 @@
"packages/*"
],
"scripts": {
"build": "turbo run build",
"test": "turbo run test",
"check-submodule": "TURBO_NO_UPDATE_NOTIFIER=true turbo run check-submodule --output-logs errors-only",
"build": "npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run build",
"test": "npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run test",
"test:package-boundaries": "node --test eslint.config.test.js",
"test:coverage": "turbo run test:coverage",
"test:browser": "turbo run test:browser",
"test:typecheck-coverage": "node --test typecheck.config.test.js",
"test:coverage": "npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run test:coverage",
"test:browser": "npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run test:browser",
"setup:browsers": "playwright install --with-deps chromium firefox",
"typecheck": "turbo run typecheck",
"typetest": "turbo run typetest",
"lint": "eslint eslint.config.js eslint.config.test.js packages/*/src",
"lint:fix": "eslint eslint.config.js eslint.config.test.js packages/*/src --fix",
"format": "prettier --write \"eslint.config*.js\" \"packages/*/src/**/*.{ts,tsx,js,json}\"",
"format:check": "prettier --check \"eslint.config*.js\" \"packages/*/src/**/*.{ts,tsx,js,json}\"",
"typecheck": "npm run test:typecheck-coverage && npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run typecheck",
"typetest": "npm run check-submodule && RUNNING_FROM_NPM=correctly turbo run typetest",
"lint": "eslint eslint.config.js eslint.config.test.js typecheck.config.test.js packages/*/src",
"lint:fix": "eslint eslint.config.js eslint.config.test.js typecheck.config.test.js packages/*/src --fix",
"format": "prettier --write \"eslint.config*.js\" \"typecheck.config.test.js\" \"packages/*/src/**/*.{ts,tsx,js,json}\"",
"format:check": "prettier --check \"eslint.config*.js\" \"typecheck.config.test.js\" \"packages/*/src/**/*.{ts,tsx,js,json}\"",
"dpdm": "dpdm packages/*/src/index.ts",
"prepare": "husky"
},
+31
View File
@@ -0,0 +1,31 @@
# @webnet/binary
Binary readers and writers for webnet protocols.
`BinaryWriter` and `BinaryReader` are growable, bounds-checked big/little-endian cursors over `Uint8Array`. `LengthPrefixedBinaryWriter` and `LengthPrefixedBinaryReader` extend them with `u32`-length-prefixed byte strings and UTF-8 text.
The package has no transport or DOM dependency, so protocol packages can share binary serialization without depending on browser utilities.
## Entry points
| Entry point | Description |
| ---------------- | ----------------------------------------------------------------------------------------------- |
| `@webnet/binary` | Binary readers and writers, length-prefixed variants, and their cursor option and endian types. |
## Usage
```ts
import { BinaryWriter, LengthPrefixedBinaryReader } from "@webnet/binary"
const bytes = new BinaryWriter({ endian: "little" })
.u32(3)
.bytes(new Uint8Array([1, 2, 3]))
.finish()
const reader = new LengthPrefixedBinaryReader(bytes, 0, "little")
reader.string() // Uint8Array [1, 2, 3]
```
## See also
- [`@webnet/transport`](../transport) — byte-stream abstractions used by protocol clients and servers
+26
View File
@@ -0,0 +1,26 @@
{
"name": "@webnet/binary",
"version": "0.1.0",
"description": "Binary readers and writers for webnet protocols",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"exports": {
".": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"scripts": {
"build": "rm -rf dist && tsc --project tsconfig.json",
"test": "tsx --test --test-timeout=10000 'src/**/*.test.ts'",
"test:coverage": "c8 --src src --exclude 'src/**/*.test.ts' --reporter text --reporter lcov node --enable-source-maps --import tsx --test-timeout=10000 --test 'src/**/*.test.ts'",
"typecheck": "tsc --project tsconfig.json --noEmit"
},
"devDependencies": {
"@types/node": "^25.6.0",
"c8": "^11.0.0",
"tsx": "^4.21.0",
"typescript": "^6.0.2"
}
}
+7
View File
@@ -0,0 +1,7 @@
export {
BinaryReader,
BinaryWriter,
LengthPrefixedBinaryReader,
LengthPrefixedBinaryWriter,
} from "./binary.js"
export type { BinaryCursorOptions, BinaryEndian } from "./binary.js"
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"verbatimModuleSyntax": true,
"strict": true,
"skipLibCheck": true,
"lib": ["ES2022"],
"types": ["node"],
"outDir": "dist",
"rootDir": "src",
"declaration": true,
"declarationMap": true,
"inlineSourceMap": true,
"inlineSources": true
},
"include": ["src/**/*"],
"exclude": ["src/**/*.test.ts"]
}
+37
View File
@@ -0,0 +1,37 @@
# @webnet/browser-test-utils
Shared Playwright utilities for browser integration tests.
This is a private workspace, not a published library: it is used only by this repo's own `node:test` suites that need a real browser. `forBrowsers` wraps a test body in a `node:test` `suite` per configured browser (Chromium and Firefox), launching a headless `Browser` in `before` and closing it in `after`, and hands the test a `BrowserTestContext` with `newPage()`, `newContext()`, and `serve(dir)`. Both browser helpers inject an `__name` shim into pages, working around `tsx`'s esbuild `keepNames` output losing function names when `page.evaluate()` serializes callbacks via `toString()`. Use `newContext()` when a test needs multiple pages that share browser-context state. `serve(dir)` starts a local static file server (`TestServer`) rooted at `dir`, rejecting paths that escape it.
## Usage
```ts
import { forBrowsers } from "@webnet/browser-test-utils"
forBrowsers(({ browserName, newPage, serve }) => {
test(`loads in ${browserName}`, async () => {
const server = await serve("dist")
const page = await newPage()
await page.goto(server.url)
await server.close()
})
})
```
Use `newContext()` for multiple pages in one browser context:
```ts
forBrowsers(({ newContext }) => {
test("shares context state", async () => {
const context = await newContext()
const firstPage = await context.newPage()
const secondPage = await context.newPage()
await context.close()
})
})
```
## See also
- [`@webnet/test-app`](../test-app) — one of the apps commonly served under test via `serve()`
+3
View File
@@ -7,6 +7,9 @@
"exports": {
".": "./src/index.ts"
},
"scripts": {
"typecheck": "tsc --project tsconfig.json --noEmit"
},
"dependencies": {
"playwright": "^1.61.0"
},
+16 -1
View File
@@ -3,7 +3,7 @@ import { createServer } from "node:http"
import { createReadStream, statSync } from "node:fs"
import { extname, resolve as resolvePath, relative } from "node:path"
import { chromium, firefox } from "playwright"
import type { Browser, Page, BrowserType } from "playwright"
import type { Browser, BrowserContext, Page, BrowserType } from "playwright"
export type { Page }
@@ -15,6 +15,7 @@ export interface TestServer {
export interface BrowserTestContext {
readonly browserName: string
newPage(): Promise<Page>
newContext(): Promise<BrowserContext>
serve(dir: string): Promise<TestServer>
}
@@ -46,6 +47,20 @@ export function forBrowsers(fn: (ctx: BrowserTestContext) => void): void {
fn({
browserName: name,
newContext: async () => {
const context = await browser.newContext()
// tsx uses esbuild keepNames:true, injecting __name at module scope; page.evaluate() serializes callbacks via .toString(), losing the helper.
await context.addInitScript(() => {
;(globalThis as unknown as Record<string, unknown>).__name = (
target: unknown,
value: string,
) => {
Object.defineProperty(target, "name", { value, configurable: true })
return target
}
})
return context
},
newPage: async () => {
const page = await browser.newPage()
// tsx uses esbuild keepNames:true, injecting __name at module scope; page.evaluate() serializes callbacks via .toString(), losing the helper.
+19
View File
@@ -0,0 +1,19 @@
# @webnet/example-app
A React demo app for the tsconnect browser stack.
It wires up `@webnet/tsconnect`'s IPN into a React tree via `@webnet/tsconnect-react` and `@webnet/tsconnect-redux` (`IpnBuilderProvider`, `CaCertProvider`, `IpnProvider`), and demonstrates dialing and listening from the browser: an HTTPS server behind a TCP and TLS listener, served over Tailscale Funnel using a cert obtained through the IPN, plus a debug panel. This is a private workspace in this repository, not a published package, and is a development and demo surface for the stack rather than the product application.
## Running
```bash
npm run dev --workspace=packages/example-app
```
`dev` starts the webpack dev server bound to `localhost` only. `dev:host` (`WEBNET_DEV_HOST=1 webpack serve --mode development`) additionally binds all interfaces and accepts forwarded hostnames, for use behind a trusted reverse proxy or on a Tailnet; see the root README's "Remote-host development" section before using it. `build` produces a production bundle (`NODE_ENV=production webpack --mode production`), and `typecheck` runs `tsc --noEmit`.
## See also
- [`@webnet/tailshare`](../tailshare) — the repository's flagship tailnet file-sharing application
- [`@webnet/test-app`](../test-app) — the other development/testing surface, covering the non-React protocol packages
- [`@webnet/tsconnect-react`](../tsconnect-react) — the React hooks and context this app builds on
+2 -52
View File
@@ -20,7 +20,7 @@ import {
} from "@webnet/tsconnect-redux"
import styles from "./Debug.scss"
import { download, fmtSize } from "@webnet/utils"
import { useSecureContext } from "../hooks/useSecureContext"
import { useSecureContext } from "@webnet/react"
import { CaCertContext } from "../contexts/CaCertContext"
import type { IPN } from "@webnet/tsconnect"
@@ -47,7 +47,6 @@ export function DebugStuff() {
</div>
{ipn && (
<div className={styles.row}>
<FetchDebug />
<SuggestExitNodeDebug />
</div>
)}
@@ -75,7 +74,7 @@ function BaseDebug() {
<br />
</>
)}
ipnBuilder: {ipnBuilder ? "present" : "loading..."}
ipnBuilder: {ipnBuilder ? "present" : "unavailable"}
<br />
ipn:{" "}
{ipn ? (
@@ -399,55 +398,6 @@ function WaitingFileDebug({ id }: { id: string }) {
)
}
function FetchDebug() {
const ipc = use(IpnContext)
const [result, setResult] = useState<{
status: number
statusText: string
text: string | null
} | null>(null)
const [running, setRunning] = useState(false)
return (
<div>
<input type="text" defaultValue="" placeholder="URL" />{" "}
<button
disabled={!ipc || running}
onClick={async (e) => {
try {
if (running) return
if (!ipc) return
setRunning(true)
const input = (
e.target as HTMLButtonElement
).parentElement!.querySelector<HTMLInputElement>('[type="text"]')!
const result = await ipc.fetch(input.value)
setResult({ status: result.status, statusText: result.statusText, text: null })
const text = await result.text()
setResult({ status: result.status, statusText: result.statusText, text })
} finally {
setRunning(false)
}
}}
>
Fetch (tailscale)
</button>{" "}
<button disabled={!result} onClick={() => setResult(null)}>
Clear output
</button>
{running && <> (running)</>}
<br />
{result && (
<>
{result.status} {result.statusText}
<pre>{result.text}</pre>
</>
)}
</div>
)
}
function SuggestExitNodeDebug() {
const ipn = use(IpnContext)
const [suggested, setSuggested] = useState<Awaited<ReturnType<IPN["suggestExitNode"]>> | null>(
@@ -1,8 +1,9 @@
import { createContext, type ReactNode, use, useEffect, useState } from "react"
import { createContext, type ReactNode, use } from "react"
import { initIPN } from "@webnet/tsconnect"
import type { IPN } from "@webnet/tsconnect"
import wasmUrl from "@webnet/tsconnect/main.wasm"
export type IpnBuilder = Awaited<ReturnType<typeof initIPN>>
export type IpnBuilder = (config?: Parameters<typeof initIPN>[1]) => Promise<IPN>
export const IpnBuilderContext = createContext<IpnBuilder | null>(null)
@@ -10,11 +11,8 @@ export function useIpnBuilder(): IpnBuilder | null {
return use(IpnBuilderContext)
}
export function IpnBuilderProvider({ children }: { children: ReactNode }) {
const [ipnBuilder, setIpnBuilder] = useState<IpnBuilder | null>(null)
useEffect(() => {
initIPN(wasmUrl).then((ipnBuilder) => setIpnBuilder(() => ipnBuilder))
}, [])
const buildIpn: IpnBuilder = (config = {}) => initIPN(wasmUrl, config)
return <IpnBuilderContext value={ipnBuilder}>{children}</IpnBuilderContext>
export function IpnBuilderProvider({ children }: { children: ReactNode }) {
return <IpnBuilderContext value={buildIpn}>{children}</IpnBuilderContext>
}
@@ -11,7 +11,7 @@ import {
useRef,
useState,
} from "react"
import { useIpnBuilder } from "./IpnBuilderContext"
import { useIpnBuilder, type IpnBuilder } from "./IpnBuilderContext"
import { IpnContext, IpnStoreProvider } from "./tsconnect"
import { InMemoryFileOps, WebStorageState } from "@webnet/tsconnect"
import type { IpnClient } from "@webnet/tsconnect"
@@ -68,8 +68,8 @@ export function IpnProvider({
children: ReactNode
state?: "disable" | "localStorage" | "sessionStorage"
taildrop?: "disable" | "memory"
ipnBuilderParams?: Parameters<typeof useBuildIpn>[2]
runParams?: Parameters<typeof useBuildIpn>[3]
ipnBuilderParams?: Parameters<IpnBuilder>[0]
runParams?: Parameters<typeof useBuildIpn>[2]
defer?: boolean
autoLogin?: boolean
}) {
@@ -144,13 +144,14 @@ export function IpnProvider({
return builderParams
}, [ipnBuilderParams, state, taildrop, hostname, controlURL, authKey, client])
const mainIpn = useBuildIpn(
store,
!useWorker && client && willBuild ? ipnBuilder : null,
builderParams,
runParams,
const buildMainIpn = useMemo(
() =>
ipnBuilder && !useWorker && client && willBuild ? () => ipnBuilder(builderParams) : null,
[ipnBuilder, useWorker, client, willBuild, builderParams],
)
const mainIpn = useBuildIpn(store, buildMainIpn, runParams)
// ── Active client ─────────────────────────────────────────────────────────────
const ipn = useWorker ? workerClient : mainIpn
const activeStore = useWorker && workerClient ? workerClient.store : store
@@ -1,9 +0,0 @@
import { useSyncExternalStore } from "react"
const sub = () => () => void 0
const getClient = () => !!window.isSecureContext
const getServer = () => true
export function useSecureContext(): boolean {
return useSyncExternalStore(sub, getClient, getServer)
}
+52
View File
@@ -0,0 +1,52 @@
# @webnet/ftp
FTP/FTPS client and server built on `@webnet/transport` and `@webnet/vfs`.
`FTPClient` implements `AsyncVFS`, so it can be used anywhere an `AsyncVFS` is expected (`stat`, `readdir`, `readFile`/`readFileRange`, `writeFile`, `delete`, `mkdir`, `move`). It supports plaintext FTP, implicit FTPS (TLS from the first byte, via `dialer.dialTls`), and explicit FTPS (plaintext control connection upgraded with `AUTH TLS`/`PROT P`). `FTPServer` serves an `AsyncVFS` over FTP, with optional per-login `authenticate` to select a different VFS per user and optional TLS credentials for explicit `AUTH TLS`. Errors from both sides surface as `FTPError`, carrying the underlying `Reply`.
## Entry points
| Entry point | Description |
| -------------------- | ------------------------------------------------------------------- |
| `@webnet/ftp` | `FTPClient`, `FTPServer`, `FTPError`, and their option/reply types. |
| `@webnet/ftp/client` | `FTPClient` and its option types only. |
| `@webnet/ftp/server` | `FTPServer` and its option types only. |
`_internals` entry points are unstable and are not part of the public API.
## Usage
### Client
```ts
import { FTPClient } from "@webnet/ftp/client"
import type { RawDialer } from "@webnet/transport"
declare const dialer: RawDialer
const client = new FTPClient({ dialer, host: "ftp.example.com", user: "anonymous" })
const entries = await client.readdir("/pub")
const stream = await client.readFile("/pub/readme.txt")
await client.close()
```
### Server
```ts
import { FTPServer } from "@webnet/ftp/server"
import type { RawListener } from "@webnet/transport"
import type { AsyncVFS } from "@webnet/vfs"
declare const vfs: AsyncVFS
declare const controlListener: RawListener
declare const dataListen: (port: number) => Promise<RawListener>
const server = new FTPServer({ vfs, dataListen })
await server.listen(controlListener)
```
## See also
- [`@webnet/vfs`](../vfs) — the `AsyncVFS` interface this package implements and serves
- [`@webnet/transport`](../transport) — the `RawDialer`/`RawListener`/`RawTransport` primitives this package is built on
- [`@webnet/sftp`](../sftp) — an alternative file-transfer protocol over SSH instead of FTP
-1
View File
@@ -10,6 +10,5 @@ export {
export { formatPasv227, parsePasv227, formatEpsv229, parseEpsv229 } from "./common/addr.js"
export { quotePath } from "./common/paths.js"
export { formatTimeval, parseTimeval } from "./common/time.js"
export { transportToStream, writeAll, pumpToTransport, skipBytes } from "./common/stream.js"
export { ControlConnection } from "./client/_internals.js"
export { Session, commands, PREAUTH_COMMANDS, PassiveDataChannel } from "./server/_internals.js"
+27 -13
View File
@@ -1,8 +1,8 @@
import type { RawDialer, RawTransport } from "@webnet/transport"
import { pumpToWriter, transportToReadableStream } from "@webnet/transport/stream"
import { VFSError, baseName, parentPath, resolvePath, type AsyncVFS, type Stat } from "@webnet/vfs"
import { mlsxToStat, parseMLSX, parseUnixList } from "../common/listing.js"
import { FTPError, replyToVFSError, type Reply } from "../common/replies.js"
import { pumpToTransport, transportToStream } from "../common/stream.js"
import { parseTimeval } from "../common/time.js"
import { ControlConnection } from "./control.js"
import type { FTPClientOptions, FtpTransferState } from "./types.js"
@@ -247,7 +247,7 @@ export class FTPClient implements AsyncVFS {
}
const data = await conn.finishDataDialUnlocked(dial)
const chunks: Uint8Array[] = []
const reader = transportToStream(data).getReader()
const reader = transportToReadableStream(data).getReader()
for (;;) {
const { done, value } = await reader.read()
if (done) break
@@ -275,6 +275,13 @@ export class FTPClient implements AsyncVFS {
start: bigint,
end?: bigint,
): Promise<ReadableStream<Uint8Array>> {
// REST carries no end bound, so an inverted window would otherwise transfer from start to EOF.
if (end !== undefined && end < start)
return new ReadableStream({
start(controller) {
controller.close()
},
})
return this.#retrieve(resolvePath("/", path), start, end)
}
@@ -310,6 +317,7 @@ export class FTPClient implements AsyncVFS {
conn,
data,
release,
path,
end !== undefined ? end - offset + 1n : undefined,
)
}
@@ -318,21 +326,22 @@ export class FTPClient implements AsyncVFS {
conn: ControlConnection,
data: RawTransport,
release: () => void,
path: string,
limit: bigint | undefined,
): ReadableStream<Uint8Array> {
const reader = transportToStream(data).getReader()
const reader = transportToReadableStream(data).getReader()
let remaining = limit
let finished = false
const finish = async () => {
const finish = async (requireSuccess: boolean) => {
if (finished) return
finished = true
try {
await data.close()
// 226 after a full transfer, 426/450 after an early close: both are
// terminal for this transfer, real errors were already seen at RETR
await conn.readReplyUnlocked()
} catch {
// control-connection failures surface on the next command
const final = await conn.readReplyUnlocked()
if (requireSuccess && final.code !== 226 && final.code !== 250)
throw replyToVFSError(final, path, "RETR")
} catch (error) {
if (requireSuccess) throw error
} finally {
conn.finishDataTransfer()
release()
@@ -342,7 +351,7 @@ export class FTPClient implements AsyncVFS {
pull: async (controller) => {
const { done, value } = await reader.read()
if (done) {
await finish()
await finish(true)
controller.close()
return
}
@@ -354,13 +363,13 @@ export class FTPClient implements AsyncVFS {
if (chunk.length) controller.enqueue(chunk)
if (remaining !== undefined && remaining <= 0n) {
await reader.cancel()
await finish()
await finish(false)
controller.close()
}
},
cancel: async () => {
await reader.cancel()
await finish()
await finish(false)
},
})
}
@@ -380,7 +389,7 @@ export class FTPClient implements AsyncVFS {
const data = await conn.finishDataDialUnlocked(dial)
let pumpError: unknown
try {
await pumpToTransport(stream, data)
await pumpToWriter(stream, data)
} catch (e) {
pumpError = e
} finally {
@@ -405,6 +414,11 @@ export class FTPClient implements AsyncVFS {
return
}
if (recursive) {
// Kept rather than shared with the SFTP client: `delete` is a required operation, so there
// is no optional one to fall back for, and the recursion is protocol-shaped — DELE and RMD
// are different commands chosen from a stat this method already holds. A generic shim would
// restat every entry and still could not tell a symlink from what it points at, which is
// the difference that decides whether descending is correct at all.
for (const entry of await this.readdir(path)) await this.delete(entry.path, true)
}
const reply = await conn.exchange("RMD", path)
+8 -2
View File
@@ -10,6 +10,7 @@ export class ControlConnection {
readonly features: Set<string>
readonly #options: FTPClientOptions
readonly #transport: RawTransport
readonly #dataHost: string
readonly #reader: ControlReader
readonly #writer: ControlWriter
#tail: Promise<void> = Promise.resolve()
@@ -21,6 +22,11 @@ export class ControlConnection {
private constructor(options: FTPClientOptions, transport: RawTransport) {
this.#options = options
this.#transport = transport
const remoteAddr = transport.remoteAddr
const separator = remoteAddr?.lastIndexOf(":") ?? -1
let dataHost = separator > 0 ? remoteAddr!.slice(0, separator) : options.host
if (dataHost.startsWith("[") && dataHost.endsWith("]")) dataHost = dataHost.slice(1, -1)
this.#dataHost = dataHost
this.#reader = new ControlReader(transport)
this.#writer = new ControlWriter(transport)
this.features = new Set()
@@ -205,7 +211,7 @@ export class ControlConnection {
const port = parseEpsv229(reply.text)
if (port === undefined)
throw new FTPError(reply.code, `Unparseable EPSV reply: ${reply.text}`)
return { dial: this.#dialData(this.#options.host, port) }
return { dial: this.#dialData(this.#dataHost, port) }
}
if (reply.code !== 500 && reply.code !== 502) throw new FTPError(reply.code, reply.text)
this.#noEpsv = true
@@ -214,7 +220,7 @@ export class ControlConnection {
if (reply.code !== 227) throw new FTPError(reply.code, reply.text)
const addr = parsePasv227(reply.text)
if (!addr) throw new FTPError(reply.code, `Unparseable PASV reply: ${reply.text}`)
return { dial: this.#dialData(addr.host, addr.port) }
return { dial: this.#dialData(this.#dataHost, addr.port) }
}
async finishDataDialUnlocked(dial: Promise<RawTransport>): Promise<RawTransport> {
+13
View File
@@ -31,6 +31,15 @@ suite("parsePasv227", () => {
test("returns undefined for out-of-range port", () => {
assert.equal(parsePasv227("(192,168,1,2,999,1)"), undefined)
assert.equal(parsePasv227("(192,168,1,2,0,999)"), undefined)
})
test("returns undefined for port zero", () => {
assert.equal(parsePasv227("(192,168,1,2,0,0)"), undefined)
})
test("does not parse an unparenthesized address", () => {
assert.equal(parsePasv227("227 address 192,168,1,2,19,137"), undefined)
})
})
@@ -60,4 +69,8 @@ suite("parseEpsv229", () => {
test("returns undefined for out-of-range port", () => {
assert.equal(parseEpsv229("(|||99999|)"), undefined)
})
test("returns undefined for port zero", () => {
assert.equal(parseEpsv229("(|||0|)"), undefined)
})
})
+4 -4
View File
@@ -5,15 +5,15 @@ export function formatPasv227(host: string, port: number): string {
return `Entering Passive Mode (${octets.join(",")},${p1},${p2})`
}
const PASV_RE = /(\d+),(\d+),(\d+),(\d+),(\d+),(\d+)/
const PASV_RE = /\((\d+),(\d+),(\d+),(\d+),(\d+),(\d+)\)/
export function parsePasv227(text: string): { host: string; port: number } | undefined {
const m = PASV_RE.exec(text)
if (!m) return undefined
const [h1, h2, h3, h4, p1, p2] = m.slice(1, 7).map(Number)
if ([h1, h2, h3, h4].some((n) => n! > 255)) return undefined
if ([h1, h2, h3, h4, p1, p2].some((n) => n! > 255)) return undefined
const port = p1! * 256 + p2!
if (port > 65535) return undefined
if (port === 0) return undefined
return { host: `${h1}.${h2}.${h3}.${h4}`, port }
}
@@ -35,6 +35,6 @@ export function parseEpsv229(text: string): number | undefined {
}
if (digits === undefined) return undefined
const port = Number(digits)
if (port > 65535) return undefined
if (port === 0 || port > 65535) return undefined
return port
}
+7
View File
@@ -17,6 +17,7 @@ suite("vfsErrorToReply", () => {
["not-empty", 550, "Directory not empty"],
["forbidden", 550, "Permission denied"],
["locked", 450, "File busy"],
["unsupported", 502, "Command not supported"],
]
for (const [code, expectedCode, expectedText] of cases) {
test(`${code} => ${expectedCode}`, () => {
@@ -93,6 +94,12 @@ suite("replyToVFSError", () => {
assert.equal(e.code, "locked")
})
test("502 maps to unsupported", () => {
const e = replyToVFSError(reply(502, "Rename not supported"), "/a")
assert.ok(e instanceof VFSError)
assert.equal(e.code, "unsupported")
})
test("other codes map to FTPError", () => {
const e = replyToVFSError(reply(500, "Syntax error"), "/a")
assert.ok(e instanceof FTPError)
+3
View File
@@ -21,6 +21,7 @@ const CODE_BY_VFS_ERROR: Record<VFSErrorCode, number> = {
"not-empty": 550,
forbidden: 550,
locked: 450,
unsupported: 502,
}
// Conventional wording per code. 550 is FTP's catch-all, so relaying the backing filesystem's
@@ -34,6 +35,7 @@ const TEXT_BY_VFS_ERROR: Record<VFSErrorCode, string> = {
"not-empty": "Directory not empty",
forbidden: "Permission denied",
locked: "File busy",
unsupported: "Command not supported",
}
export function vfsErrorToReply(e: VFSError): { code: number; text: string } {
@@ -60,5 +62,6 @@ export function replyToVFSError(reply: Reply, path: string, verb?: string): VFSE
if (code === 553) return new VFSError("already-exists", text)
if (code === 530 || code === 532) return new VFSError("forbidden", text)
if (code === 450 || code === 452) return new VFSError("locked", text)
if (code === 502) return new VFSError("unsupported", text)
return new FTPError(code, text)
}
-70
View File
@@ -1,70 +0,0 @@
import type { RawTransport } from "@webnet/transport"
export function transportToStream(transport: RawTransport): ReadableStream<Uint8Array> {
return new ReadableStream({
async pull(controller) {
try {
controller.enqueue(await transport.read())
} catch {
// stream-mode FTP has no length framing: end-of-data and connection
// errors are indistinguishable, both terminate the stream
controller.close()
}
},
cancel() {
transport.close()
},
})
}
export async function writeAll(transport: RawTransport, data: Uint8Array): Promise<void> {
let written = (await transport.write(data)) ?? data.length
while (written < data.length) {
written += (await transport.write(data.subarray(written))) ?? data.length - written
}
}
export async function pumpToTransport(
stream: ReadableStream<Uint8Array>,
transport: RawTransport,
): Promise<void> {
const reader = stream.getReader()
try {
for (;;) {
const { done, value } = await reader.read()
if (done) break
await writeAll(transport, value)
}
} finally {
reader.releaseLock()
}
}
export function skipBytes(
stream: ReadableStream<Uint8Array>,
count: bigint,
): ReadableStream<Uint8Array> {
let remaining = count
const reader = stream.getReader()
return new ReadableStream({
async pull(controller) {
for (;;) {
const { done, value } = await reader.read()
if (done) {
controller.close()
return
}
if (remaining >= BigInt(value.length)) {
remaining -= BigInt(value.length)
continue
}
controller.enqueue(remaining > 0n ? value.subarray(Number(remaining)) : value)
remaining = 0n
return
}
},
cancel(reason) {
return reader.cancel(reason)
},
})
}
+313 -13
View File
@@ -7,7 +7,11 @@ import type { RawDialer, RawListener, RawTransport, TlsUpgradeOptions } from "@w
import type { StateTransferable } from "@webnet/state-transfer"
import { MemoryVFS } from "@webnet/vfs/memory"
import { VFSError, type AsyncVFS } from "@webnet/vfs"
import { testAsyncVFSConformance } from "@webnet/vfs/conformance"
import {
testAsyncVFSConformance,
withoutOptional,
unsupportedOptional,
} from "@webnet/vfs/conformance"
import { FTPServer } from "./server/server.js"
import type { FTPServerOptions } from "./server/types.js"
import { FTPClient } from "./client/client.js"
@@ -15,6 +19,7 @@ import type { FTPClientOptions } from "./client/types.js"
import { ControlReader, ControlWriter } from "./common/codec.js"
import { parseEpsv229 } from "./common/addr.js"
import { parseUnixList } from "./common/listing.js"
import { FTPError } from "./common/replies.js"
const certPem = readFileSync(
new URL("../../transport/src/node/fixtures/cert.pem", import.meta.url),
@@ -91,7 +96,23 @@ class TransferableTransport implements RawTransport, StateTransferable<RawTransp
// Control dials go to the control listener; dataListen mints a loopback pair
// per advertised port and the routing dialer sends data dials to it.
function makeNet() {
function shortWritingTransport(transport: RawTransport, maxWrite: number): RawTransport {
return new Proxy(transport, {
get(target, property) {
if (property === "write") {
return async (data: Uint8Array) => {
const chunk = data.subarray(0, Math.min(maxWrite, data.length))
const written = await target.write(chunk)
return typeof written === "number" ? written : chunk.length
}
}
const value = Reflect.get(target, property, target)
return typeof value === "function" ? value.bind(target) : value
},
})
}
function makeNet(wrapDataTransport?: (transport: RawTransport) => RawTransport) {
const [ctrlListener, ctrlDialer] = loopbackListener()
const dataDialers = new Map<number, RawDialer>()
const dataListeners: RawListener[] = []
@@ -109,7 +130,10 @@ function makeNet() {
dataDialers.delete(p)
return listener.close()
},
accept: () => listener.accept(),
accept: async () => {
const transport = await listener.accept()
return wrapDataTransport ? wrapDataTransport(transport) : transport
},
addr: `127.0.0.1:${p}`,
}
dataListeners.push(wrapped)
@@ -197,9 +221,12 @@ async function makeTlsPair(
return { vfs, client, upgrades, errors, close }
}
async function rawConnect(opts: Partial<FTPServerOptions> = {}) {
async function rawConnect(
opts: Partial<FTPServerOptions> = {},
wrapDataTransport?: (transport: RawTransport) => RawTransport,
) {
const vfs = new MemoryVFS()
const net = makeNet()
const net = makeNet(wrapDataTransport)
const server = new FTPServer({
vfs,
dataListen: net.dataListen,
@@ -228,6 +255,61 @@ async function rawConnect(opts: Partial<FTPServerOptions> = {}) {
return { vfs, net, greeting, cmd, login, close, reader, writer }
}
async function dataCommand(
conn: Awaited<ReturnType<typeof rawConnect>>,
verb: string,
arg?: string,
): Promise<string> {
const epsv = await conn.cmd("EPSV")
assert.equal(epsv.code, 229)
const port = parseEpsv229(epsv.text)!
const dataPromise = conn.net.dialer.dial("127.0.0.1", port)
assert.equal((await conn.cmd(verb, arg)).code, 150)
const data = await dataPromise
const decoder = new TextDecoder()
let text = ""
for (;;) {
const chunk = await data.read()
if (chunk === null) break
text += decoder.decode(chunk, { stream: true })
}
text += decoder.decode()
assert.equal((await conn.reader.readReply()).code, 226)
return text
}
function observedVfs(combined: boolean) {
const base = new MemoryVFS()
const calls = { stat: 0, readdir: 0, statAndReaddir: 0 }
const vfs = new Proxy(base, {
get(target, property, receiver) {
if (property === "statAndReaddir") {
if (!combined) return undefined
return async (path: string) => {
calls.statAndReaddir++
const self = await target.stat(path)
return { self, entries: self.isDirectory ? await target.readdir(path) : [] }
}
}
if (property === "stat") {
return async (path: string) => {
calls.stat++
return target.stat(path)
}
}
if (property === "readdir") {
return async (path: string) => {
calls.readdir++
return target.readdir(path)
}
}
const value = Reflect.get(target, property, receiver)
return typeof value === "function" ? value.bind(target) : value
},
}) as AsyncVFS
return { base, calls, vfs }
}
async function seed(vfs: AsyncVFS, files: Record<string, string>) {
for (const [path, content] of Object.entries(files)) {
await vfs.writeFile(path, streamOf(content))
@@ -244,6 +326,19 @@ testAsyncVFSConformance({
capabilities: { etag: false },
})
// The same client against a server that has only the required operations to work with, so the
// server's fallback paths are held to the same contract as its native ones.
testAsyncVFSConformance({
name: "FTPClient (server over a minimal filesystem)",
create: () => {
const { client, close } = makeTestPair({ vfs: withoutOptional(new MemoryVFS()) })
return { vfs: client, close }
},
// RNFR/RNTO answers 502 rather than renaming by hand when the filesystem cannot move, so the
// client reports the operation as unsupported. Whether to shim it instead is issue #179.
capabilities: { etag: false, move: "unsupported" },
})
// -- suites --
suite("FTPClient + FTPServer over loopback", () => {
@@ -369,6 +464,42 @@ suite("FTPClient + FTPServer over loopback", () => {
}
})
test("fails a download when the server aborts after sending data", async () => {
const base = new MemoryVFS()
await base.writeFile("/broken", streamOf("complete file"))
const vfs = new Proxy(base, {
get(target, property, receiver) {
if (property === "readFile") {
return async () => {
let sent = false
return new ReadableStream<Uint8Array>({
pull(controller) {
if (!sent) {
sent = true
controller.enqueue(new TextEncoder().encode("partial"))
} else {
controller.error(new Error("source failed"))
}
},
})
}
}
const value = Reflect.get(target, property, receiver)
return typeof value === "function" ? value.bind(target) : value
},
}) as AsyncVFS
const { client, close } = makeTestPair({ vfs })
try {
await assert.rejects(readAllText(await client.readFile("/broken")), (error) => {
assert.ok(error instanceof FTPError)
assert.equal(error.code, 426)
return true
})
} finally {
await close()
}
})
test("authenticate rejects bad logins and serves per-user vfs", async () => {
const vfsA = new MemoryVFS()
const vfsB = new MemoryVFS()
@@ -618,6 +749,37 @@ suite("fallback paths (no MLSD/MLST/EPSV)", () => {
await close()
}
})
test("PASV uses the control peer instead of the advertised host", async () => {
const pair = makeTestPair({ disableFeatures, passiveHost: "192.0.2.1" })
const dials: Array<{ host: string; port: number }> = []
const dialer: RawDialer = {
async dial(host, port) {
dials.push({ host, port })
const transport = await pair.net.dialer.dial(host, port)
if (port !== 21) return transport
return new Proxy(transport, {
get(target, property) {
if (property === "remoteAddr") return "198.51.100.2:21"
const value = Reflect.get(target, property, target)
return typeof value === "function" ? value.bind(target) : value
},
})
},
}
const client = new FTPClient({ dialer, host: "control.example" })
try {
await client.writeFile("/f.txt", streamOf("via pasv"))
assert.ok(dials.length >= 2)
assert.deepEqual(
dials.slice(0, 2).map(({ host }) => host),
["control.example", "198.51.100.2"],
)
} finally {
await client.close().catch(() => {})
await pair.close()
}
})
})
suite("FTP server protocol", () => {
@@ -746,6 +908,88 @@ suite("FTP server protocol", () => {
}
})
test("LIST, NLST, and STAT list a file as itself", async () => {
const conn = await rawConnect()
try {
await conn.login()
await conn.vfs.writeFile("/f.txt", streamOf("abc"))
const list = (await dataCommand(conn, "LIST", "/f.txt")).trim()
assert.equal(parseUnixList(list)?.name, "f.txt")
assert.equal((await dataCommand(conn, "NLST", "/f.txt")).trim(), "f.txt")
const stat = await conn.cmd("STAT", "/f.txt")
assert.equal(stat.code, 213)
assert.equal(parseUnixList(stat.lines[1].trim())?.name, "f.txt")
assert.equal((await conn.cmd("EPSV")).code, 229)
assert.equal((await conn.cmd("MLSD", "/f.txt")).code, 550)
} finally {
await conn.close()
}
})
test("listing commands prefer statAndReaddir while MLST remains stat-only", async () => {
const { base, calls, vfs } = observedVfs(true)
await base.mkdir("/dir")
await base.writeFile("/dir/child.txt", streamOf("child"))
const conn = await rawConnect({ vfs })
try {
await conn.login()
assert.match(await dataCommand(conn, "LIST", "/dir"), /child\.txt/)
assert.match(await dataCommand(conn, "MLSD", "/dir"), /child\.txt/)
assert.equal((await conn.cmd("STAT", "/dir")).code, 213)
assert.equal(calls.statAndReaddir, 3)
assert.equal(calls.stat, 0)
assert.equal(calls.readdir, 0)
assert.equal((await conn.cmd("MLST", "/dir/child.txt")).code, 250)
assert.equal(calls.statAndReaddir, 3)
assert.equal(calls.stat, 1)
} finally {
await conn.close()
}
})
test("listing commands fall back to stat and conditionally readdir", async () => {
const { base, calls, vfs } = observedVfs(false)
await base.mkdir("/dir")
await base.writeFile("/file.txt", streamOf("file"))
await base.writeFile("/dir/child.txt", streamOf("child"))
const conn = await rawConnect({ vfs })
try {
await conn.login()
assert.match(await dataCommand(conn, "LIST", "/file.txt"), /file\.txt/)
assert.match(await dataCommand(conn, "MLSD", "/dir"), /child\.txt/)
assert.equal(calls.stat, 2)
assert.equal(calls.readdir, 1)
assert.equal(calls.statAndReaddir, 0)
} finally {
await conn.close()
}
})
test("listing commands fall back when statAndReaddir is unsupported here", async () => {
const { base, calls, vfs } = observedVfs(true)
await base.mkdir("/dir")
await base.writeFile("/dir/child.txt", streamOf("child"))
const unsupported: AsyncVFS = new Proxy(vfs, {
get(target, property, receiver) {
if (property === "statAndReaddir") return () => Promise.reject(new VFSError("unsupported"))
return Reflect.get(target, property, receiver) as unknown
},
})
const conn = await rawConnect({ vfs: unsupported })
try {
await conn.login()
assert.match(await dataCommand(conn, "LIST", "/dir"), /child\.txt/)
assert.equal(calls.stat, 1)
assert.equal(calls.readdir, 1)
} finally {
await conn.close()
}
})
test("REST + RETR resumes at offset", async () => {
const conn = await rawConnect()
try {
@@ -760,10 +1004,36 @@ suite("FTP server protocol", () => {
assert.equal((await conn.cmd("RETR", "/f.txt")).code, 150)
const data = await dataPromise
const chunks: Uint8Array[] = []
try {
for (;;) chunks.push(await data.read())
} catch {
// socket ended
for (;;) {
const chunk = await data.read()
if (chunk === null) break
chunks.push(chunk)
}
assert.equal(new TextDecoder().decode(await readAll(streamOf(chunks[0]))), "world")
assert.equal((await conn.reader.readReply()).code, 226)
} finally {
await conn.close()
}
})
test("REST + RETR resumes at offset when readFileRange rejects unsupported", async () => {
const base = new MemoryVFS()
await base.writeFile("/f.txt", streamOf("hello world"))
const conn = await rawConnect({ vfs: unsupportedOptional(base, ["readFileRange"]) })
try {
await conn.login()
assert.equal((await conn.cmd("REST", "6")).code, 350)
const epsv = await conn.cmd("EPSV")
assert.equal(epsv.code, 229)
const port = parseEpsv229(epsv.text)!
const dataPromise = conn.net.dialer.dial("127.0.0.1", port)
assert.equal((await conn.cmd("RETR", "/f.txt")).code, 150)
const data = await dataPromise
const chunks: Uint8Array[] = []
for (;;) {
const chunk = await data.read()
if (chunk === null) break
chunks.push(chunk)
}
assert.equal(new TextDecoder().decode(await readAll(streamOf(chunks[0]))), "world")
assert.equal((await conn.reader.readReply()).code, 226)
@@ -783,10 +1053,10 @@ suite("FTP server protocol", () => {
assert.equal((await conn.cmd("LIST")).code, 150)
const data = await dataPromise
let text = ""
try {
for (;;) text += new TextDecoder().decode(await data.read())
} catch {
// socket ended
for (;;) {
const chunk = await data.read()
if (chunk === null) break
text += new TextDecoder().decode(chunk)
}
assert.equal((await conn.reader.readReply()).code, 226)
const lines = text.split("\r\n").filter(Boolean)
@@ -800,6 +1070,21 @@ suite("FTP server protocol", () => {
}
})
test("LIST preserves its output across short data writes", async () => {
const conn = await rawConnect({}, (transport) => shortWritingTransport(transport, 2))
try {
await conn.login()
await conn.vfs.writeFile("/first.txt", streamOf("a"))
await conn.vfs.writeFile("/second.txt", streamOf("b"))
const listing = await dataCommand(conn, "NLST")
assert.equal(listing, "first.txt\r\nsecond.txt\r\n")
} finally {
await conn.close()
}
})
test("STOR stores data and reports vfs errors after 150", async () => {
const conn = await rawConnect()
try {
@@ -841,6 +1126,21 @@ suite("FTP server protocol", () => {
}
})
test("RNFR/RNTO answers 502 when move rejects unsupported, matching an absent move", async () => {
const base = new MemoryVFS()
await base.writeFile("/a", streamOf("x"))
const conn = await rawConnect({ vfs: unsupportedOptional(base, ["move"]) })
try {
await conn.login()
assert.equal((await conn.cmd("RNFR", "/a")).code, 350)
assert.equal((await conn.cmd("RNTO", "/b")).code, 502)
assert.equal(await readAllText(await base.readFile("/a")), "x")
await assert.rejects(base.stat("/b"), rejectsVfs("not-found"))
} finally {
await conn.close()
}
})
test("passive replies and ABOR", async () => {
const conn = await rawConnect()
try {
+23 -11
View File
@@ -1,8 +1,9 @@
import { transportToReadableStream } from "@webnet/transport/stream"
import { parentPath, resolvePath, type Stat } from "@webnet/vfs"
import { moveFallback, readFileRangeFallback, statAndReaddirFallback } from "@webnet/vfs/fallback"
import { formatEpsv229, formatPasv227 } from "../common/addr.js"
import { formatMLSX, formatUnixList } from "../common/listing.js"
import { quotePath } from "../common/paths.js"
import { skipBytes, transportToStream } from "../common/stream.js"
import { formatTimeval } from "../common/time.js"
import type { Session } from "./session.js"
@@ -39,8 +40,10 @@ function listArg(session: Session, arg: string): string {
}
async function listEntries(session: Session, path: string): Promise<Stat[]> {
const stat = await session.vfs.stat(path)
return stat.isDirectory ? await session.vfs.readdir(path) : [stat]
const { self, entries } = await statAndReaddirFallback(session.vfs, path, {
policy: "on-unsupported",
})
return self.isDirectory ? entries : [self]
}
function encodeLines(lines: string[]): Uint8Array {
@@ -52,9 +55,11 @@ async function openRetrStream(
path: string,
offset: bigint,
): Promise<ReadableStream<Uint8Array>> {
// REST carries no end bound, so this is always a read to the end of the file. A zero offset is
// the whole file and does not need a ranged read; above that the shared fallback windows the
// stream, including when the backend has readFileRange but rejects it for this path.
if (offset === 0n) return session.vfs.readFile(path)
if (session.vfs.readFileRange) return session.vfs.readFileRange(path, offset)
return skipBytes(await session.vfs.readFile(path), offset)
return readFileRangeFallback(session.vfs, path, offset, undefined, { policy: "on-unsupported" })
}
export const commands = new Map<string, CommandHandler>([
@@ -242,9 +247,10 @@ export const commands = new Map<string, CommandHandler>([
const channel = s.state.passive
if (!channel) return s.reply(425, "Use PASV or EPSV first")
const path = resolvePath(s.state.cwd, arg)
const stat = await s.vfs.stat(path)
if (!stat.isDirectory) return s.reply(550, "Not a directory")
const entries = await s.vfs.readdir(path)
const { self, entries } = await statAndReaddirFallback(s.vfs, path, {
policy: "on-unsupported",
})
if (!self.isDirectory) return s.reply(550, "Not a directory")
s.takePassive()
await s.sendOverData(channel, encodeLines(entries.map((e) => formatMLSX(e))))
},
@@ -321,7 +327,7 @@ export const commands = new Map<string, CommandHandler>([
if (!channel) return s.reply(425, "Use PASV or EPSV first")
const path = resolvePath(s.state.cwd, arg)
await s.runTransfer(channel, (transport) =>
s.vfs.writeFile(path, transportToStream(transport)),
s.vfs.writeFile(path, transportToReadableStream(transport)),
)
},
],
@@ -369,8 +375,14 @@ export const commands = new Map<string, CommandHandler>([
const from = s.state.renameFrom
s.state.renameFrom = null
if (!from) return s.reply(503, "RNFR required first")
if (!s.vfs.move) return s.reply(502, "Rename not supported")
await s.vfs.move(from, resolvePath(s.state.cwd, arg), { overwrite: true })
// native-only: RENAME is expected to be cheap and roughly atomic, and a shim would turn a
// rename of a large directory into a recursive copy and delete with no way for the client
// to know. The fallback is here so a backend without move and one that rejects unsupported
// for this path both reach the session's error mapping as 502.
await moveFallback(s.vfs, from, resolvePath(s.state.cwd, arg), {
overwrite: true,
policy: "native-only",
})
await s.reply(250, "Rename successful")
},
],
+4 -3
View File
@@ -1,8 +1,9 @@
import type { RawTransport } from "@webnet/transport"
import { writeAll } from "@webnet/transport/operation"
import { pumpToWriter } from "@webnet/transport/stream"
import { VFSError, type AsyncVFS } from "@webnet/vfs"
import { ControlReader, ControlWriter } from "../common/codec.js"
import { vfsErrorToReply } from "../common/replies.js"
import { pumpToTransport } from "../common/stream.js"
import { PassiveDataChannel, type DataChannel } from "./data.js"
import { commands, PREAUTH_COMMANDS } from "./commands.js"
import type { FTPServerOptions } from "./types.js"
@@ -198,14 +199,14 @@ export class Session {
async sendOverData(channel: DataChannel, data: Uint8Array): Promise<void> {
await this.runTransfer(channel, async (transport) => {
if (data.length) await transport.write(data)
await writeAll(transport, data)
})
}
async streamOverData(channel: DataChannel, stream: ReadableStream<Uint8Array>): Promise<void> {
await this.runTransfer(
channel,
(transport) => pumpToTransport(stream, transport),
(transport) => pumpToWriter(stream, transport),
() => stream.cancel(),
)
}
+32
View File
@@ -0,0 +1,32 @@
# @webnet/http-static
Static file HTTP handler backed by an `@webnet/vfs` `AsyncVFS`.
`createStaticHandler` turns an `AsyncVFS` into a `@webnet/http` `Handler`: it resolves request paths against the VFS, serves files with `ETag`/`Last-Modified` conditional requests and `Range` support, and can render directory listings (HTML or JSON, based on `Accept`) or fall back to a single file (e.g. an SPA's `index.html`) when nothing matches.
## Usage
```ts
import { createStaticHandler } from "@webnet/http-static"
import { Server } from "@webnet/http/server"
import type { AsyncVFS } from "@webnet/vfs"
import type { RawListener } from "@webnet/transport"
declare const vfs: AsyncVFS
declare const listener: RawListener
const handler = createStaticHandler(vfs, {
prefix: "/static",
index: ["index.html"],
fallback: "/index.html",
})
const server = new Server(handler)
await server.listen(listener)
```
## See also
- [`@webnet/http`](../http) — `Handler`/`Server` this package's output is used with
- [`@webnet/vfs`](../vfs) — the `AsyncVFS` interface this package serves
- [`@webnet/webdav`](../webdav) — a read/write, protocol-level alternative to serving a VFS statically
+14
View File
@@ -3,6 +3,7 @@ import { suite, test } from "node:test"
import type { Body } from "@webnet/http"
import { MemoryVFS } from "@webnet/vfs/memory"
import type { AsyncVFS } from "@webnet/vfs"
import { unsupportedOptional } from "@webnet/vfs/conformance"
import { createStaticHandler } from "./handler.js"
function streamOf(value: string): ReadableStream<Uint8Array> {
@@ -391,6 +392,19 @@ suite("createStaticHandler", () => {
assert.equal(await readBody(context.res.body), "bcd")
})
test("falls back like an absent readFileRange when it rejects unsupported at runtime", async () => {
const base = new MemoryVFS()
await write(base, "/data", "abcdef")
const vfs = unsupportedOptional(base, ["readFileRange"])
const handler = createStaticHandler(vfs)
const context = makeContext("GET", "/data", { range: "bytes=1-3" })
await handler(context)
assert.equal(context.res.status, 206)
assert.equal(await readBody(context.res.body), "bcd")
assert.equal(context.res.getHeader("content-range"), "bytes 1-3/6")
assert.equal(context.res.getHeader("content-length"), "3")
})
test("returns 400 for malformed encoded paths and respects unacceptable listings", async () => {
const vfs = new MemoryVFS()
const handler = createStaticHandler(vfs)
+8 -43
View File
@@ -1,5 +1,6 @@
import type { Context, Handler } from "@webnet/http"
import { VFSError, type AsyncVFS, type Stat } from "@webnet/vfs"
import { readFileRangeFallback } from "@webnet/vfs/fallback"
export type StaticPathResolver = (ctx: Context) => string | Promise<string>
@@ -99,46 +100,6 @@ function setMetadata(ctx: Context, stat: Stat): void {
if (stat.modifiedAt) ctx.res.setHeader("Last-Modified", formatHttpDate(stat.modifiedAt))
}
function boundedStream(stream: ReadableStream<Uint8Array>, start: bigint, length: bigint) {
const reader = stream.getReader()
let offset = 0n
let remaining = length
return new ReadableStream<Uint8Array>({
async pull(controller) {
while (remaining > 0n) {
const result = await reader.read()
if (result.done) {
controller.close()
reader.releaseLock()
return
}
const chunk = result.value
const chunkStart = offset
offset += BigInt(chunk.byteLength)
const chunkEnd = offset
if (chunkEnd <= start) continue
const from = Number(start > chunkStart ? start - chunkStart : 0n)
const count = Math.min(Number(remaining), chunk.byteLength - from)
if (count > 0) {
controller.enqueue(chunk.slice(from, from + count))
remaining -= BigInt(count)
}
if (remaining === 0n) {
await reader.cancel()
reader.releaseLock()
return
}
}
controller.close()
reader.releaseLock()
},
cancel() {
void reader.cancel()
reader.releaseLock()
},
})
}
function quality(value: string, type: string): number {
let best = -1
for (const part of value.split(",")) {
@@ -264,9 +225,13 @@ async function serveFile(ctx: Context, vfs: AsyncVFS, lookup: Lookup): Promise<v
if (ctx.req.method === "HEAD") {
ctx.res.body = null
} else {
ctx.res.body = vfs.readFileRange
? await vfs.readFileRange(path, range.start, range.end)
: boundedStream(await vfs.readFile(path), range.start, length)
// A file server that could serve the range should not refuse it, so a backend that says
// unsupported for this path is answered the same way as one that has no readFileRange at
// all. The shim discards the prefix, which is wasted transfer the client cannot see; the
// alternative is a 501 many clients handle worse than a slow response.
ctx.res.body = await readFileRangeFallback(vfs, path, range.start, range.end, {
policy: "on-unsupported",
})
}
return
}
+84
View File
@@ -0,0 +1,84 @@
# @webnet/http
HTTP/1.1 client and server built on `@webnet/transport`.
The client provides `fetch`/`fetchStream` and a connection pool (`PooledDialer`/`UnpooledDialer`) with redirect following via `ClientConnection`. The server provides `Server`, which drives a `Handler` over accepted connections, and a `Router` for method/path dispatch with typed middleware. TLS is not handled by this package directly: the client dials TLS through `RawDialer.dialTls` (from `@webnet/transport`) when a `https:` URL is used, and `fetch` throws if the dialer has no `dialTls`; the server listens on whatever `RawListener` it is given, so plain vs. TLS listening is determined by the listener/transport passed to `Server.listen`.
## Entry points
| Entry point | Description |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `@webnet/http` | The `client`, `server`, and `common` entry points re-exported together. `Router` is not included; import it from `@webnet/http/router`. |
| `@webnet/http/client` | `fetch`, `fetchStream`, `makeFetch`, `ClientConnection`, `PooledDialer`, `UnpooledDialer`, and client types. |
| `@webnet/http/server` | `Server`, `ServerConnection`, and server types (`Context`, `Handler`, `ServerRequest`, `ServerResponse`). |
| `@webnet/http/router` | `Router` and `RouteProxy` for path/method dispatch. |
| `@webnet/http/common` | `Headers`, `MutableHeaders`, `methods`, `statusCodes`. |
`_internals` entry points are unstable and are not part of the public API.
## Usage
### Client
```ts
import { makeFetch } from "@webnet/http/client"
import type { RawDialer } from "@webnet/transport"
declare const dialer: RawDialer
const fetch = makeFetch(dialer)
const res = await fetch("http://example.com/")
console.log(res.status, await res.text())
```
### Server
```ts
import { Server } from "@webnet/http/server"
import { Router } from "@webnet/http/router"
import type { RawListener } from "@webnet/transport"
declare const listener: RawListener
const router = new Router()
router.get("/hello/:name", async (ctx) => {
ctx.res.body = `hello ${ctx.keys.name}`
})
const server = new Server(router.handler)
await server.listen(listener)
```
## Server limits
A server faces untrusted peers, so every limit `Server.listen` and `ServerConnection` accept has a finite default. Pass `Infinity` for one to opt out of it.
| Option | Default | Applies to |
| -------------------- | ------- | --------------------------------------------------------- |
| `headersTimeout` | 30 s | receiving the request line and all headers |
| `keepAliveTimeout` | 60 s | idle wait for the next request on a keep-alive connection |
| `bodyTimeout` | 5 min | reading an entire request body |
| `maxBodyLength` | 64 MiB | request body, whether declared or streamed |
| `maxTotalHeaderSize` | 64 KiB | all header lines of one request |
| `maxHeaderCount` | 256 | number of header lines of one request |
| `maxTargetLength` | 16 KiB | request target |
| `maxLineLength` | 128 KiB | any single request or header line |
The first five were `Infinity` before: a server that accepts large uploads, slow bodies or long idle periods now has to raise them explicitly. The client keeps its unbounded defaults, and a limit passed to an individual body read can only lower the connection's, never raise it.
## Message framing
Framing is decided from headers on the way in and enforced on the way out, in both the client and the server.
A message is rejected rather than parsed when it carries both `Transfer-Encoding` and `Content-Length`, repeats either of them, names a transfer-coding other than `chunked`, gives a `Content-Length` that is not a bare decimal, or has a field name that is not an RFC 9110 token or a value containing a control character. On the server the peer gets a `400` (or `413`, `414` or `431` where one of the limits above is what failed) and the connection closes; `ServerConnection.handle` no longer drops the connection without an answer. These are the shapes a peer uses to make us disagree with a proxy in front of us about where one message ends and the next begins, so guessing is worse than refusing.
`writeHeaders` throws on CR, LF or NUL in a header name or value, so caller-supplied metadata cannot split the message. Header values are routinely built from things the caller did not author, such as a `Content-Type` off a VFS `stat`, so this is checked in one place rather than at every call site.
Exactly one framing header reaches the wire, on any message that may carry a body. A body of known size is framed by `Content-Length`; a stream is chunked, unless the caller set a valid `Content-Length` itself, in which case that framing is kept and the stream is held to exactly that many bytes. A `Content-Length` that cannot frame anything is dropped in favour of chunked. On a status that forbids a body there is no framing to decide, so a header the handler set there is left as it is.
## See also
- [`@webnet/transport`](../transport) — the `RawDialer`/`RawListener`/`RawTransport` primitives this package is built on
- [`@webnet/http-static`](../http-static) — static file serving on top of a `Handler`
- [`@webnet/websocket`](../websocket) — WebSocket upgrade on top of `@webnet/http`
- [`@webnet/webdav`](../webdav) — WebDAV client and server on top of `@webnet/http`
@@ -115,6 +115,41 @@ suite("ClientConnection", () => {
assert.strictEqual(res.version, "1.1")
})
test("rejects a response carrying both framing headers", async () => {
const { conn, serverWrite, serverBuf } = await makeClientServer()
const req = new ClientRequestImpl({ target: "/" })
const responsePromise = conn.request(req)
await drainHeaders(serverBuf)
await serverWrite(
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nContent-Length: 6\r\n\r\n0\r\n\r\n",
)
await assert.rejects(
() => responsePromise,
/Both Transfer-Encoding and Content-Length/,
"a malicious response frames as badly as a malicious request",
)
})
test("rejects a response header name that is not a token", async () => {
const { conn, serverWrite, serverBuf } = await makeClientServer()
const req = new ClientRequestImpl({ target: "/" })
const responsePromise = conn.request(req)
await drainHeaders(serverBuf)
await serverWrite("HTTP/1.1 200 OK\r\nContent-Length : 6\r\n\r\nhello\n")
await assert.rejects(() => responsePromise, /Invalid header name/)
})
test("throws rather than writing a request header that would split the message", async () => {
const { conn, clientTransport, serverTransport } = await makeClientServer()
const req = new ClientRequestImpl({
target: "/",
headers: { "X-File": "a\r\nX-Injected: yes" },
})
await assert.rejects(() => conn.send(req), /must not contain CR, LF or NUL/)
clientTransport.close()
assert.strictEqual(await serverTransport.read(), null, "nothing should have been written")
})
test("parses a 404 Not Found response", async () => {
const { conn, serverWrite, serverBuf } = await makeClientServer()
const req = new ClientRequestImpl({ target: "/missing" })
+2 -2
View File
@@ -15,7 +15,7 @@ import {
} from "../common/reader.js"
import { methods, statusCodeProperties, statusCodes, type Method } from "../common/spec.js"
import { hasOwnProperty, withTimeout } from "../common/utils.js"
import { formatBody, sendBody, writeHeaders } from "../common/writer.js"
import { declaredContentLength, formatBody, sendBody, writeHeaders } from "../common/writer.js"
import { ClientResponseImpl } from "./objects.js"
import type { ClientInformationalResponse, ClientRequest, ClientResponse } from "./types.js"
@@ -95,7 +95,7 @@ export class ClientConnection {
this.#writeBuffer.write(headers)
// if we have a body, send it
if (body) await sendBody(this.#writeBuffer, this.#encoder, body)
if (body) await sendBody(this.#writeBuffer, this.#encoder, body, declaredContentLength(request))
// and flush to make sure we sent everything
await this.#writeBuffer.flushAll()
+115 -8
View File
@@ -1,7 +1,7 @@
import test, { suite } from "node:test"
import assert from "node:assert"
import { fetch, fetchStream, makeFetch } from "./fetch.js"
import { UnpooledDialer } from "./pool.js"
import { UnpooledDialer, type ConnectionPool } from "./pool.js"
import { loopbackListener, loopbackTransportPair } from "@webnet/transport/loopback"
import { ReadBuffer } from "@webnet/transport/buffer"
@@ -74,6 +74,57 @@ function redirectServer(
return { dialer, serverDone }
}
function truncatedRedirectServer(): {
dialer: ReturnType<typeof loopbackListener>[1]
serverDone: Promise<void>
} {
const [listener, dialer] = loopbackListener()
const serverDone = (async () => {
const first = await listener.accept()
await collectRequest(new ReadBuffer(first))
await first.write(
enc.encode(
"HTTP/1.1 302 Redirect\r\n" +
"Location: http://other.example/\r\n" +
"Content-Length: 10\r\n" +
"Connection: close\r\n\r\n" +
"ab",
),
)
first.close()
const second = await listener.accept()
await collectRequest(new ReadBuffer(second))
await second.write(enc.encode(ok200))
second.close()
listener.close()
})()
return { dialer, serverDone }
}
function trackRejections(dialer: ReturnType<typeof loopbackListener>[1]): {
pool: ConnectionPool
count: () => number
} {
const inner = new UnpooledDialer(dialer)
let count = 0
return {
pool: {
tlsSupported: inner.tlsSupported,
getConnection: inner.getConnection.bind(inner),
releaseConnection: inner.releaseConnection.bind(inner),
rejectConnection(...args) {
count++
inner.rejectConnection(...args)
},
exportIdle: inner.exportIdle.bind(inner),
seed: inner.seed.bind(inner),
shutdown: inner.shutdown.bind(inner),
},
count: () => count,
}
}
// Single-connection server that handles two requests (for same-connection redirect testing).
function sameConnRedirectServer(
redirectStatus: number,
@@ -258,16 +309,11 @@ suite("fetch()", () => {
await serverDone
})
test("stream() swallows body error when shouldClose closes transport (body.closed=true path)", async () => {
// fetch() without Connection:upgrade → keepAlive=false adds Connection:close → shouldClose=true
// Server sends 2 of 10 promised bytes then closes. Body read fails, conn.close() anyHandler fires
// (because shouldClose=true), which sets body.closed=true. stream() catch sees body.closed=true
// → `continue` → loop exits cleanly → bytes() resolves with partial data rather than rejecting.
test("truncated response body rejects even when the connection should close", async () => {
const { dialer, serverDone } = simpleServer("HTTP/1.1 200 OK\r\nContent-Length: 10\r\n\r\nab")
const res = await fetch(dialer, "http://localhost/")
assert.strictEqual(res.status, 200)
const body = await res.bytes()
assert.strictEqual(body.length, 2)
await assert.rejects(() => res.bytes(), /Unexpected end of stream/)
await serverDone
})
@@ -479,6 +525,42 @@ suite("makeFetch()", () => {
await serverDone
})
test("an empty body chunk does not desync a reused connection", async () => {
const [listener, dialer] = loopbackListener()
const requestLines: string[] = []
const serverDone = (async () => {
const transport = await listener.accept()
const buf = new ReadBuffer(transport)
for (let i = 0; i < 2; i++) {
requestLines.push(await buf.readLine())
while ((await buf.readLine()) !== "") {
/* drain headers */
}
if (i === 0) {
assert.strictEqual(await buf.readLine(), "0", "body should be a bare last-chunk")
assert.strictEqual(await buf.readLine(), "")
}
await transport.write(enc.encode(ok200ka))
}
transport.close()
listener.close()
})()
const f = makeFetch(dialer, { keepAlive: true })
const empty = new ReadableStream<Uint8Array>({
start(c) {
c.enqueue(new Uint8Array(0))
c.close()
},
})
await f("http://localhost/first", { method: "PUT", body: empty })
await f("http://localhost/second")
await f.pool.shutdown()
await serverDone
assert.deepStrictEqual(requestLines, ["PUT /first HTTP/1.1", "GET /second HTTP/1.1"])
})
test("f.pool is accessible", () => {
const [, dialer] = loopbackListener()
const f = makeFetch(dialer, { keepAlive: false })
@@ -636,6 +718,31 @@ suite("fetch() — redirect error handling", () => {
await assert.rejects(() => fetch(dialer, "http://localhost/", { redirect: { mode: "follow" } }))
await serverDone
})
test("rejects a truncated redirect connection and continues on a new one", async () => {
const { dialer, serverDone } = truncatedRedirectServer()
const tracked = trackRejections(dialer)
const f = makeFetch(tracked.pool)
const res = await f("http://localhost/", { redirect: { mode: "follow" } })
assert.strictEqual(res.status, 200)
assert.strictEqual(tracked.count(), 1)
await serverDone
})
test("fetchStream rejects a truncated redirect connection and continues on a new one", async () => {
const { dialer, serverDone } = truncatedRedirectServer()
const tracked = trackRejections(dialer)
const f = makeFetch(tracked.pool)
const statuses: number[] = []
for await (const res of f.stream("http://localhost/", {
redirect: { mode: "follow" },
})) {
statuses.push(res.status)
}
assert.deepStrictEqual(statuses, [302, 200])
assert.strictEqual(tracked.count(), 1)
await serverDone
})
})
suite("fetch() — redirect.mode: follow", () => {
-8
View File
@@ -361,17 +361,12 @@ async function rawFetch(
if (!reuseConn) {
// Drain the redirect body so the connection can be cleanly released.
let drainOk = true
// drain() exits via the while-condition rather than throwing: BasicBodyReader sets
// closed=true via buffer.ended before read() is re-entered, so the catch and the
// rejectConnection branch are unreachable with any standard transport.
/* c8 ignore start */
try {
await conn.drain()
} catch {
drainOk = false
}
if (!drainOk) pool.rejectConnection(hostname, port, isTls, conn)
/* c8 ignore stop */
if (drainOk) pool.releaseConnection(hostname, port, isTls, conn)
hostname = location.hostname
@@ -498,15 +493,12 @@ async function* rawFetchStream(
if (!reuseConn) {
let drainOk = true
// Same reasoning as above: drain() cannot throw with standard transports.
/* c8 ignore start */
try {
await conn.drain()
} catch {
drainOk = false
}
if (!drainOk) done(true)
/* c8 ignore stop */
if (drainOk) done(false)
hostname = location.hostname
+8 -2
View File
@@ -6,5 +6,11 @@ export {
readHeaders,
type BodyReader,
} from "./reader.js"
export { TimeoutError, hasOwnProperty, withTimeout } from "./utils.js"
export { formatBody, sendBody, writeChunkedBody, writeHeaders } from "./writer.js"
export { HttpProtocolError, TimeoutError, hasOwnProperty, withTimeout } from "./utils.js"
export {
declaredContentLength,
formatBody,
sendBody,
writeChunkedBody,
writeHeaders,
} from "./writer.js"
+7
View File
@@ -109,6 +109,13 @@ suite("MutableHeaders", () => {
assert.deepStrictEqual(h.get("set-cookie"), ["a=1", "b=2"])
})
test("delete() removes a header, case-insensitively", () => {
const h = new MutableHeaders({ "X-Foo": "bar" })
assert.strictEqual(h.delete("x-FOO"), true)
assert.strictEqual(h.has("x-foo"), false)
assert.strictEqual(h.delete("x-foo"), false)
})
test("add() creates header when absent", () => {
const h = new MutableHeaders({})
h.add("X-New", "first")
+3
View File
@@ -48,6 +48,9 @@ export class MutableHeaders extends Headers {
set(header: string, value: string | readonly string[]): void {
this._normalized.set(header.toLowerCase(), value)
}
delete(header: string): boolean {
return this._normalized.delete(header.toLowerCase())
}
add(header: string, value: string): void {
let next = this.get(header)
if (typeof next === "string") {
+1
View File
@@ -1,2 +1,3 @@
export { Headers, MutableHeaders, normalizeHeaders } from "./headers.js"
export { HttpProtocolError } from "./utils.js"
export { methods, statusCodeProperties, statusCodes, type Method } from "./spec.js"
+62 -6
View File
@@ -44,6 +44,12 @@ const hangingReader: Reader = {
},
}
// yields to the macrotask queue, so a body read started above has armed its
// timeout by the time the mocked clock is ticked
function armed(): Promise<void> {
return new Promise((ok) => setImmediate(ok))
}
function makeReadable(data: string, defaultOptions = {}) {
return new ReadableHttpImpl({
headers: new Headers({}),
@@ -161,46 +167,58 @@ suite("ReadableHttpImpl", () => {
)
})
test("fires TimeoutError when bodyTimeout expires", async () => {
test("fires TimeoutError when bodyTimeout expires", async (t) => {
t.mock.timers.enable({ apis: ["setTimeout"] })
const impl = new ReadableHttpImpl({
headers: new Headers({}),
bodyStream: hangingReader,
})
await assert.rejects(
const rejected = assert.rejects(
() => drain(impl.iter("binary", { bodyTimeout: 1 })),
(e) => {
assert.ok(e instanceof TimeoutError)
return true
},
)
await armed()
t.mock.timers.tick(1)
await rejected
})
test("fires TimeoutError from defaultBodyReaderOptions", async () => {
test("fires TimeoutError from defaultBodyReaderOptions", async (t) => {
t.mock.timers.enable({ apis: ["setTimeout"] })
const impl = new ReadableHttpImpl({
headers: new Headers({}),
bodyStream: hangingReader,
defaultBodyReaderOptions: { bodyTimeout: 1 },
})
await assert.rejects(
const rejected = assert.rejects(
() => drain(impl.iter("binary")),
(e) => {
assert.ok(e instanceof TimeoutError)
return true
},
)
await armed()
t.mock.timers.tick(1)
await rejected
})
test("takes minimum of options and default bodyTimeout", async () => {
test("takes minimum of options and default bodyTimeout", async (t) => {
// options has 10000ms (permissive), default has 1ms (strict) → 1ms wins
t.mock.timers.enable({ apis: ["setTimeout"] })
const impl = new ReadableHttpImpl({
headers: new Headers({}),
bodyStream: hangingReader,
defaultBodyReaderOptions: { bodyTimeout: 1 },
})
await assert.rejects(
const rejected = assert.rejects(
() => drain(impl.iter("binary", { bodyTimeout: 10000 })),
(e) => e instanceof TimeoutError,
)
await armed()
t.mock.timers.tick(1)
await rejected
})
})
@@ -235,6 +253,44 @@ suite("ReadableHttpImpl", () => {
assert.throws(() => impl.stream(), /doesn't have one/)
})
// Enqueueing on a byte stream detaches the chunk's ArrayBuffer, and a body reader
// hands out views onto the read buffer's own storage. Reading a body large enough
// to arrive in several transport chunks used to leave the buffer pointing at
// detached memory partway through.
test("reading a multi-chunk body does not detach the read buffer", async () => {
const size = 64 * 1024
const chunkSize = 8 * 1024
let sent = 0
const reader: Reader = {
get closed() {
return sent >= size
},
async read() {
if (sent >= size) return null
const n = Math.min(chunkSize, size - sent)
sent += n
return new Uint8Array(n).fill(7)
},
}
const impl = new ReadableHttpImpl({
headers: new Headers({}),
bodyStream: new BasicBodyReader(new ReadBuffer(reader), size),
defaultBodyReaderOptions: {},
})
const streamReader = impl.stream().getReader()
let total = 0
for (;;) {
const { done, value } = await streamReader.read()
if (done) break
assert.ok(
value.every((b) => b === 7),
"body bytes should survive the read",
)
total += value.byteLength
}
assert.strictEqual(total, size)
})
test("throws when called twice", async () => {
const impl = makeReadable("hello")
const s = impl.stream()
+12 -4
View File
@@ -57,7 +57,7 @@ export class ReadableHttpImpl implements ReadableHttp {
const deadline =
bodyTimeout !== undefined && isFinite(bodyTimeout) ? Date.now() + bodyTimeout : undefined
while (!body.closed) {
let chunk: Uint8Array
let chunk: Uint8Array | null
try {
const remaining = deadline !== undefined ? Math.max(0, deadline - Date.now()) : undefined
chunk = await withTimeout(body.read(), remaining, "Body timeout")
@@ -65,6 +65,7 @@ export class ReadableHttpImpl implements ReadableHttp {
if (body.closed) continue
throw e
}
if (chunk === null) break
if (!chunk.length) continue
bodyLength += chunk.length
if (bodyLength > maxBodyLength)
@@ -89,7 +90,7 @@ export class ReadableHttpImpl implements ReadableHttp {
const pull = async (controller: ReadableByteStreamController) => {
while (!body.closed) {
let chunk: Uint8Array
let chunk: Uint8Array | null
try {
const remaining = deadline !== undefined ? Math.max(0, deadline - Date.now()) : undefined
chunk = await withTimeout(body.read(), remaining, "Body timeout")
@@ -98,6 +99,7 @@ export class ReadableHttpImpl implements ReadableHttp {
controller.error(e)
return
}
if (chunk === null) break
if (!chunk.length) continue
bodyLength += chunk.length
if (bodyLength > maxBodyLength) {
@@ -113,10 +115,13 @@ export class ReadableHttpImpl implements ReadableHttp {
} else {
view.set(chunk.subarray(0, view.byteLength))
byob.respond(view.byteLength)
controller.enqueue(chunk.subarray(view.byteLength) as Uint8Array<ArrayBuffer>)
// slice, not subarray: enqueueing on a byte stream detaches the chunk's
// ArrayBuffer, and a body reader hands out views onto the read buffer's
// own storage, which it goes on using.
controller.enqueue(chunk.slice(view.byteLength) as Uint8Array<ArrayBuffer>)
}
} else {
controller.enqueue(chunk as Uint8Array<ArrayBuffer>)
controller.enqueue(chunk.slice() as Uint8Array<ArrayBuffer>)
}
return
}
@@ -191,4 +196,7 @@ export class WritableHttpImpl implements WritableHttp {
addHeader(header: string, value: string): void {
this.#headers.add(header, value)
}
removeHeader(header: string): void {
this.#headers.delete(header)
}
}
+106 -4
View File
@@ -15,7 +15,7 @@ function makeReader(data: string | Uint8Array, chunkSize = Infinity): Reader {
return offset >= bytes.length
},
async read() {
if (offset >= bytes.length) return new Uint8Array(0)
if (offset >= bytes.length) return null
const end = isFinite(chunkSize) ? Math.min(offset + chunkSize, bytes.length) : bytes.length
const chunk = bytes.slice(offset, end)
offset = end
@@ -59,6 +59,32 @@ suite("readHeaders", () => {
await assert.rejects(() => readHeaders(buf), /no colon/)
})
test("throws on whitespace between the field name and the colon", async () => {
const buf = rb("Content-Length : 5\r\n\r\n")
await assert.rejects(() => readHeaders(buf), /Invalid header name/)
})
test("throws on a non-token character in the field name", async () => {
const buf = rb("X(Foo): bar\r\n\r\n")
await assert.rejects(() => readHeaders(buf), /Invalid header name/)
})
test("throws on an empty field name", async () => {
const buf = rb(": bar\r\n\r\n")
await assert.rejects(() => readHeaders(buf), /Invalid header name/)
})
test("throws on a NUL in the field value", async () => {
const buf = rb("X-Foo: ba\0r\r\n\r\n")
await assert.rejects(() => readHeaders(buf), /Invalid header value/)
})
test("allows a HTAB in the field value", async () => {
const buf = rb("X-Foo: ba\tr\r\n\r\n")
const headers = await readHeaders(buf)
assert.strictEqual(headers.get("x-foo"), "ba\tr")
})
test("throws when maxHeaderCount exceeded", async () => {
const buf = rb("A: 1\r\nB: 2\r\nC: 3\r\n\r\n")
await assert.rejects(() => readHeaders(buf, { maxHeaderCount: 2 }), /Too many headers/)
@@ -343,6 +369,33 @@ suite("ChunkedBodyReader", () => {
const reader = new ChunkedBodyReader(buf)
await assert.rejects(() => reader.read(), /Chunk declared/)
})
test("throws when EOF arrives before the final chunk terminator", async () => {
const reader = new ChunkedBodyReader(rb("0\r\n"))
let ko = false
reader.onFinish(undefined, () => {
ko = true
})
await assert.rejects(() => reader.read(), /Unexpected end of stream/)
assert.strictEqual(reader.closed, false)
assert.strictEqual(ko, true)
})
test("throws when EOF arrives during a chunk terminator", async () => {
const reader = new ChunkedBodyReader(rb("1\r\na\r"))
await assert.rejects(() => reader.read(), /Unexpected end of stream/)
})
test("throws when a chunk has an invalid terminator", async () => {
const reader = new ChunkedBodyReader(rb("1\r\naXX"))
await assert.rejects(() => reader.read(), /Invalid chunk terminator/)
})
test("throws when the final chunk has an invalid terminator", async () => {
const reader = new ChunkedBodyReader(rb("0\r\nXX"))
await assert.rejects(() => reader.read(), /Invalid chunk terminator/)
})
})
suite("bodyReader", () => {
@@ -366,11 +419,60 @@ suite("bodyReader", () => {
assert.strictEqual(bodyReader(buf, headers), null)
})
test("Transfer-Encoding takes precedence over Content-Length", () => {
test("throws when both Transfer-Encoding and Content-Length are present", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": "chunked", "Content-Length": "10" })
const reader = bodyReader(buf, headers)
assert.ok(reader instanceof ChunkedBodyReader)
assert.throws(() => bodyReader(buf, headers), /Both Transfer-Encoding and Content-Length/)
})
test("matches the transfer-coding name case-insensitively", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": "Chunked" })
assert.ok(bodyReader(buf, headers) instanceof ChunkedBodyReader)
})
test("ignores optional whitespace around the transfer-coding name", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": " chunked " })
assert.ok(bodyReader(buf, headers) instanceof ChunkedBodyReader)
})
test("throws on a repeated Transfer-Encoding", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": ["chunked", "chunked"] })
assert.throws(() => bodyReader(buf, headers), /must not be repeated/)
})
test("throws on a transfer-coding we do not implement", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": "gzip, chunked" })
assert.throws(() => bodyReader(buf, headers), /Unsupported Transfer-Encoding/)
})
test("throws on a transfer-coding whose name only starts with chunked", () => {
const buf = rb("")
const headers = new Headers({ "Transfer-Encoding": "chunkedx" })
assert.throws(() => bodyReader(buf, headers), /Unsupported Transfer-Encoding/)
})
test("throws on a repeated Content-Length", () => {
const buf = rb("")
const headers = new Headers({ "Content-Length": ["5", "5"] })
assert.throws(() => bodyReader(buf, headers), /must not be repeated/)
})
test("throws on a Content-Length that is not a bare decimal", () => {
for (const value of ["0x10", "+5", "1e3", "5.0", " 5", "-1"]) {
const buf = rb("")
const headers = new Headers({ "Content-Length": value })
assert.throws(() => bodyReader(buf, headers), /Content-Length is invalid/, value)
}
})
test("throws on a Content-Length beyond the safe integer range", () => {
const buf = rb("")
const headers = new Headers({ "Content-Length": "9".repeat(20) })
assert.throws(() => bodyReader(buf, headers), /Content-Length is invalid/)
})
test("throws for invalid Content-Length", () => {
+70 -21
View File
@@ -1,9 +1,17 @@
import type { Reader } from "@webnet/transport"
import type { ReadBuffer } from "@webnet/transport/buffer"
import { Headers } from "./headers.js"
import { hasOwnProperty } from "./utils.js"
import { HttpProtocolError, hasOwnProperty } from "./utils.js"
function assertChunkTerminator(buffer: ReadBuffer, offset: number): void {
const parts = buffer.slice(offset, 2)
const last = parts[parts.length - 1]
if (parts[0][0] !== 0x0d || last[last.length - 1] !== 0x0a)
throw new Error("Invalid chunk terminator")
}
export interface BodyReader extends Reader {
read(): Promise<Uint8Array>
onFinish(
okHandler?: () => void | Promise<void>,
koHandler?: (err: unknown) => void | Promise<void>,
@@ -13,12 +21,12 @@ export interface BodyReader extends Reader {
export type BodyReaderOptions = {
/**
* @defaultValue Infinity
* @defaultValue Infinity, but finite on a server connection: see `ServerConnectionOptions`
*/
maxBodyLength?: number
/**
* Maximum total time in ms to read the entire body.
* @defaultValue Infinity
* @defaultValue Infinity, but finite on a server connection: see `ServerConnectionOptions`
*/
bodyTimeout?: number
}
@@ -84,7 +92,6 @@ export class BasicBodyReader extends BaseBodyReader implements BodyReader {
}
get closed(): boolean {
if (this.#buffer.ended) return true
return this.#remaining === 0
}
@@ -96,7 +103,8 @@ export class BasicBodyReader extends BaseBodyReader implements BodyReader {
return chunk
}
if (this.closed) throw new Error("BasicBodyReader is already closed")
while (!this.#buffer.len) await this.#buffer.readOnce()
while (!this.#buffer.len && !this.#buffer.ended) await this.#buffer.readOnce()
if (!this.#buffer.len) throw new Error("Unexpected end of stream")
const len = Math.min(this.#buffer.len, this.#remaining)
this.#remaining -= len
const data = this.#buffer.slice(0, len)
@@ -131,7 +139,7 @@ export class ChunkedBodyReader extends BaseBodyReader implements BodyReader {
}
get closed(): boolean {
return this.#buffer.ended || this.#finished
return this.#finished
}
async read(): Promise<Uint8Array> {
@@ -146,15 +154,19 @@ export class ChunkedBodyReader extends BaseBodyReader implements BodyReader {
if (this.#bodyLength > this.#maxBodyLength)
throw new Error(`Body too large: ${this.#bodyLength} (max: ${this.#maxBodyLength})`)
if (!len) {
this.#finished = true
await this.#buffer.read(2)
if (this.#buffer.len < 2) throw new Error("Unexpected end of stream")
assertChunkTerminator(this.#buffer, 0)
this.#buffer.forward(2)
this.#finished = true
await this._triggerFinishOk()
return new Uint8Array(0)
}
await this.#buffer.read(len + 2)
if (len > this.#buffer.len)
throw new Error(`Chunk declared ${len} bytes but buffer only has ${this.#buffer.len}`)
if (this.#buffer.len < len + 2) throw new Error("Unexpected end of stream")
assertChunkTerminator(this.#buffer, len)
const data = this.#buffer.slice(0, len)
this.#buffer.forward(len + 2)
for (const chunk of data) this.#chunks.push(chunk)
@@ -172,11 +184,20 @@ export type ReadHeadersOptions = {
*/
maxHeaderCount?: number
/**
* @defaultValue Infinity
* @defaultValue Infinity, but finite on a server connection: see `ServerConnectionOptions`
*/
maxTotalHeaderSize?: number
}
// RFC 9110 §5.6.2 token: the only shape a field name may take. Rejecting everything
// else also rejects `Name : value`, whose space would otherwise make the header a
// distinct name we silently ignore while a proxy in front of us reads it normally.
const TOKEN = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/
// CTLs other than HTAB are forbidden in a field value. CR and LF cannot reach here,
// since readLine has already split the message on them, but a NUL can.
// eslint-disable-next-line no-control-regex
const FORBIDDEN_IN_VALUE = /[\0-\x08\x0a-\x1f\x7f]/
export async function readHeaders(
buffer: ReadBuffer,
{ maxHeaderCount = 256, maxTotalHeaderSize = Infinity }: ReadHeadersOptions = {},
@@ -192,17 +213,23 @@ export async function readHeaders(
headerCount++
if (headerCount > maxHeaderCount)
throw new Error(`Too many headers: ${headerCount} (max: ${maxHeaderCount})`)
throw new HttpProtocolError(431, `Too many headers: ${headerCount} (max: ${maxHeaderCount})`)
totalHeaderSize += headerLine.length
if (totalHeaderSize > maxTotalHeaderSize)
throw new Error(
throw new HttpProtocolError(
431,
`Total max header size exceeded: ${totalHeaderSize} (max: ${maxTotalHeaderSize})`,
)
const colon = headerLine.indexOf(":")
if (colon === -1) throw new Error(`Invalid header line (no colon): ${headerLine.slice(0, 80)}`)
if (colon === -1)
throw new HttpProtocolError(400, `Invalid header line (no colon): ${headerLine.slice(0, 80)}`)
const key = headerLine.slice(0, colon)
const value = headerLine.slice(colon + 1).trim()
if (!TOKEN.test(key))
throw new HttpProtocolError(400, `Invalid header name: ${JSON.stringify(key.slice(0, 80))}`)
if (FORBIDDEN_IN_VALUE.test(value))
throw new HttpProtocolError(400, `Invalid header value for ${key}`)
if (!hasOwnProperty(rawHeaders, key)) rawHeaders[key] = []
rawHeaders[key].push(value)
}
@@ -216,16 +243,38 @@ export function bodyReader(
headers: Headers,
{ maxBodyLength = Infinity }: BodyReaderOptions = {},
): BodyReader | null {
if (headers.get("Transfer-Encoding") === "chunked") {
const transferEncoding = headers.get("Transfer-Encoding")
const contentLength = headers.get("Content-Length")
// Preferring either one is the request smuggling primitive: whichever we ignore is
// the one a proxy in front of us may have used to decide where the body ends.
if (transferEncoding !== undefined && contentLength !== undefined)
throw new HttpProtocolError(400, "Both Transfer-Encoding and Content-Length are present")
if (transferEncoding !== undefined) {
// A repeated field is a list split across lines, and the halves can disagree.
if (typeof transferEncoding !== "string")
throw new HttpProtocolError(400, "Transfer-Encoding must not be repeated")
const codings = transferEncoding.split(",").map((coding) => coding.trim().toLowerCase())
if (codings.length !== 1 || codings[0] !== "chunked")
throw new HttpProtocolError(400, `Unsupported Transfer-Encoding: ${transferEncoding}`)
return new ChunkedBodyReader(buffer, { maxBodyLength })
} else if (typeof headers.get("Content-Length") === "string") {
const len = +(headers.get("Content-Length") as string)
if (!Number.isInteger(len) || len < 0)
throw new Error(`Content-Length is invalid: ${headers.get("Content-Length")}`)
if (len > maxBodyLength)
throw new Error(`Content-Length is too big: ${len} (max: ${maxBodyLength})`)
return new BasicBodyReader(buffer, len)
} else {
return null
}
if (contentLength !== undefined) {
if (typeof contentLength !== "string")
throw new HttpProtocolError(400, "Content-Length must not be repeated")
// DIGIT only: `+`, `0x` and exponent forms all parse to a different length than
// a peer reading the field as decimal would get.
if (!/^[0-9]+$/.test(contentLength))
throw new HttpProtocolError(400, `Content-Length is invalid: ${contentLength}`)
const len = +contentLength
if (!Number.isSafeInteger(len))
throw new HttpProtocolError(400, `Content-Length is invalid: ${contentLength}`)
if (len > maxBodyLength)
throw new HttpProtocolError(413, `Content-Length is too big: ${len} (max: ${maxBodyLength})`)
return new BasicBodyReader(buffer, len)
}
return null
}
+1
View File
@@ -27,6 +27,7 @@ export interface WritableHttp {
hasHeader(header: string): boolean
setHeader(header: string, value: string | readonly string[]): void
addHeader(header: string, value: string): void
removeHeader(header: string): void
body: Body
}
+18
View File
@@ -2,6 +2,21 @@ export function hasOwnProperty(obj: object, prop: string): boolean {
return Object.prototype.hasOwnProperty.call(obj, prop)
}
/**
* Thrown when a peer sends a message we refuse to parse. `status` is the response
* a server should send before closing the connection; on the client side it only
* describes the fault.
*/
export class HttpProtocolError extends Error {
status: number
constructor(status: number, message: string) {
super(message)
this.name = "HttpProtocolError"
this.status = status
}
}
export class TimeoutError extends Error {
constructor(message: string) {
super(message)
@@ -17,6 +32,9 @@ export function withTimeout<T>(
if (ms === undefined || !isFinite(ms)) return promise
return new Promise((ok, ko) => {
const t = setTimeout(() => ko(new TimeoutError(message)), ms)
// a watchdog must never be the only thing keeping a Node process alive;
// in browsers setTimeout returns a number and there is nothing to unref
if (typeof t !== "number") t.unref?.()
promise.then(
(v) => {
clearTimeout(t)
+164 -1
View File
@@ -2,7 +2,13 @@ import test, { suite } from "node:test"
import assert from "node:assert"
import { WriteBuffer } from "@webnet/transport/buffer"
import { WritableHttpImpl } from "./objects.js"
import { formatBody, sendBody, writeChunkedBody, writeHeaders } from "./writer.js"
import {
declaredContentLength,
formatBody,
sendBody,
writeChunkedBody,
writeHeaders,
} from "./writer.js"
import type { Writer } from "@webnet/transport"
const enc = new TextEncoder()
@@ -52,6 +58,75 @@ suite("writeHeaders", () => {
test("returns empty string for empty headers", () => {
assert.strictEqual(writeHeaders({}), "")
})
test("throws on CR, LF or NUL in a value", () => {
for (const injected of ["a\r\nX-Injected: yes", "a\nX-Injected: yes", "a\rb", "a\0b"]) {
assert.throws(
() => writeHeaders({ "X-File": injected }),
/must not contain CR, LF or NUL/,
JSON.stringify(injected),
)
}
})
test("throws on CR, LF or NUL in a name", () => {
assert.throws(
() => writeHeaders({ "X-File\r\nX-Injected": "yes" }),
/Header name must not contain CR, LF or NUL/,
)
})
test("throws on CR, LF or NUL in any value of a multi-value header", () => {
assert.throws(
() => writeHeaders({ "Set-Cookie": ["a=1", "b=2\r\nX-Injected: yes"] }),
/must not contain CR, LF or NUL/,
)
})
})
suite("sendBody with a declared Content-Length", () => {
async function* gen(...chunks: string[]) {
for (const chunk of chunks) yield chunk
}
test("writes the stream unframed", async () => {
const { wb, output } = makeWriteBuffer()
await sendBody(wb, enc, gen("hel", "lo"), 5)
await wb.flushAll()
assert.strictEqual(output(), "hello")
})
test("throws when the stream runs short of what the peer was told", async () => {
const { wb } = makeWriteBuffer()
await assert.rejects(
() => sendBody(wb, enc, gen("hi"), 5),
/Body is 2 bytes but Content-Length/,
)
})
test("throws before an overrun reaches the wire", async () => {
const { wb, output } = makeWriteBuffer()
await assert.rejects(
() => sendBody(wb, enc, gen("hello", " world"), 5),
/longer than the declared Content-Length/,
)
await wb.flushAll()
assert.strictEqual(output(), "hello", "the overrunning chunk is not written")
})
test("writes a ReadableStream unframed", async () => {
const { wb, output } = makeWriteBuffer()
const stream = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(enc.encode("hel"))
controller.enqueue(enc.encode("lo"))
controller.close()
},
})
await sendBody(wb, enc, stream, 5)
await wb.flushAll()
assert.strictEqual(output(), "hello")
})
})
suite("formatBody", () => {
@@ -95,6 +170,53 @@ suite("formatBody", () => {
assert.strictEqual(w.getHeader("transfer-encoding"), "chunked")
})
test("only one framing header survives, whichever the body picks", () => {
async function* gen() {
yield "chunk"
}
const streamed = new WritableHttpImpl({
body: gen(),
headers: { "Transfer-Encoding": "chunked" },
})
formatBody(streamed, enc, true)
assert.strictEqual(streamed.getHeader("content-length"), undefined)
const sized = new WritableHttpImpl({
body: "hello",
headers: { "Transfer-Encoding": "chunked" },
})
formatBody(sized, enc, true)
assert.strictEqual(sized.getHeader("transfer-encoding"), undefined)
assert.strictEqual(sized.getHeader("content-length"), "5")
const empty = new WritableHttpImpl({ headers: { "Transfer-Encoding": "chunked" } })
formatBody(empty, enc, true)
assert.strictEqual(empty.getHeader("transfer-encoding"), undefined)
assert.strictEqual(empty.getHeader("content-length"), "0")
})
test("a stream body keeps a Content-Length the caller already knows", () => {
async function* gen() {
yield "chunk"
}
const w = new WritableHttpImpl({ body: gen(), headers: { "Content-Length": "5" } })
formatBody(w, enc, true)
assert.strictEqual(w.getHeader("transfer-encoding"), undefined)
assert.strictEqual(declaredContentLength(w), 5)
})
test("a Content-Length that cannot frame anything is dropped for chunked", () => {
async function* gen() {
yield "chunk"
}
for (const value of ["-1", "0x10", "1e3", "nonsense"]) {
const w = new WritableHttpImpl({ body: gen(), headers: { "Content-Length": value } })
formatBody(w, enc, true)
assert.strictEqual(w.getHeader("content-length"), undefined, value)
assert.strictEqual(w.getHeader("transfer-encoding"), "chunked", value)
}
})
test("ReadableStream body sets Transfer-Encoding: chunked", () => {
const stream = new ReadableStream({
start(c) {
@@ -186,6 +308,19 @@ suite("writeChunkedBody", () => {
await wb.flushAll()
assert.strictEqual(output(), "2\r\nab\r\n3\r\ncde\r\n0\r\n\r\n")
})
test("skips empty chunks instead of ending the message early", async () => {
const { wb, output } = makeWriteBuffer()
async function* gen() {
yield ""
yield enc.encode("")
yield "ab"
yield new Uint8Array(0)
}
await writeChunkedBody(wb, enc, gen())
await wb.flushAll()
assert.strictEqual(output(), "2\r\nab\r\n0\r\n\r\n")
})
})
const FLUSH_THRESHOLD = 65536
@@ -398,4 +533,32 @@ suite("sendBody", () => {
await wb.flushAll()
assert.strictEqual(output(), "5\r\nhello\r\n0\r\n\r\n")
})
test("skips a ReadableStream's empty chunks instead of ending the message early", async () => {
const { wb, output } = makeWriteBuffer()
const stream = new ReadableStream<Uint8Array>({
start(c) {
c.enqueue(new Uint8Array(0))
c.enqueue(enc.encode("hi"))
c.enqueue(new Uint8Array(0))
c.close()
},
})
await sendBody(wb, enc, stream)
await wb.flushAll()
assert.strictEqual(output(), "2\r\nhi\r\n0\r\n\r\n")
})
test("frames an empty ReadableStream as a bare terminator", async () => {
const { wb, output } = makeWriteBuffer()
const stream = new ReadableStream<Uint8Array>({
start(c) {
c.enqueue(new Uint8Array(0))
c.close()
},
})
await sendBody(wb, enc, stream)
await wb.flushAll()
assert.strictEqual(output(), "0\r\n\r\n")
})
})
+98 -1
View File
@@ -14,6 +14,9 @@ export async function writeChunkedBody(
await buffer.flushAll()
continue
}
// A zero-length chunk is the last-chunk. Writing one for an empty piece of the body would
// end the message early and leave the real terminator to be read as the next one.
if (!chunk.length) continue
buffer.write(`${chunk.length.toString(16)}\r\n`)
buffer.write(chunk)
buffer.write("\r\n")
@@ -58,6 +61,7 @@ async function writeChunkedBodyFromStream(
const result = await reader.read()
if (result.done) break
const chunk = result.value
if (!chunk.byteLength) continue
buffer.write(`${chunk.byteLength.toString(16)}\r\n`)
buffer.write(chunk)
buffer.write("\r\n")
@@ -75,6 +79,17 @@ async function writeChunkedBodyFromStream(
buffer.write("0\r\n\r\n")
}
/**
* The length `formatBody` framed the body by, if it framed it that way. Pass it to
* `sendBody` so a stream cannot silently under- or overrun what the peer was told.
*/
export function declaredContentLength(writable: WritableHttp): number | undefined {
const value = writable.getHeader("Content-Length")
if (typeof value !== "string" || !/^[0-9]+$/.test(value)) return undefined
const len = +value
return Number.isSafeInteger(len) ? len : undefined
}
export function formatBody(
writable: WritableHttp,
encoder: TextEncoder,
@@ -87,25 +102,52 @@ export function formatBody(
writable.body = JSON.stringify(writable.body.json)
}
if (typeof writable.body === "string") writable.body = encoder.encode(writable.body)
// Exactly one framing header survives: a message carrying both is the one shape a
// peer and an intermediary can read two different ways.
if (writable.body instanceof Uint8Array) {
writable.removeHeader("Transfer-Encoding")
writable.setHeader("Content-Length", "" + writable.body.length)
} else if (writable.body) {
writable.setHeader("Transfer-Encoding", "chunked")
// A caller that already knows the length keeps it, so the peer can still show
// progress; sendBody then holds the stream to exactly that many bytes. One it
// got wrong frames nothing, so it goes rather than reaching the wire.
if (declaredContentLength(writable) !== undefined) {
writable.removeHeader("Transfer-Encoding")
} else {
writable.removeHeader("Content-Length")
writable.setHeader("Transfer-Encoding", "chunked")
}
} else if (bodyAllowed) {
writable.removeHeader("Transfer-Encoding")
writable.setHeader("Content-Length", "0")
}
return writable.body as null | Uint8Array | StreamBody | ReadableStream<Uint8Array>
}
// CR, LF or NUL anywhere in a field ends the line early, so the rest of the value
// reaches the peer's parser as headers of its own or as a second message. Header
// values routinely carry metadata we did not author, such as a content type off a
// VFS stat, so the check belongs here where no call site can forget it.
const FORBIDDEN_ON_WIRE = /[\r\n\0]/
function assertWritableField(header: string, value: string): void {
if (FORBIDDEN_ON_WIRE.test(header))
throw new TypeError(`Header name must not contain CR, LF or NUL: ${JSON.stringify(header)}`)
if (FORBIDDEN_ON_WIRE.test(value))
throw new TypeError(`Header ${header} must not contain CR, LF or NUL in its value`)
}
export function writeHeaders(
headers: Readonly<Record<string, string | readonly string[]>>,
): string {
let output = ""
for (const [header, value] of Object.entries(headers)) {
if (typeof value === "string") {
assertWritableField(header, value)
output += `${header}: ${value}\r\n`
} else {
for (const item of value) {
assertWritableField(header, item)
output += `${header}: ${item}\r\n`
}
}
@@ -113,13 +155,68 @@ export function writeHeaders(
return output
}
async function writeCountedBody(
buffer: WriteBuffer,
encoder: TextEncoder,
body: StreamBody | ReadableStream<Uint8Array>,
contentLength: number,
): Promise<void> {
let written = 0
let unflushed = 0
const write = async (chunk: Uint8Array) => {
written += chunk.byteLength
// Overrunning the declared length would leave the extra bytes to be read as the
// start of the next message, so stop before they reach the wire.
if (written > contentLength)
throw new Error(`Body is longer than the declared Content-Length of ${contentLength}`)
buffer.write(chunk)
unflushed += chunk.byteLength
if (unflushed >= FLUSH_THRESHOLD) {
await buffer.flushAll()
unflushed = 0
}
}
if (body instanceof ReadableStream) {
const reader = body.getReader()
try {
for (;;) {
const result = await reader.read()
if (result.done) break
await write(result.value)
}
} finally {
reader.releaseLock()
}
} else {
for await (let chunk of body) {
if (typeof chunk === "string") chunk = encoder.encode(chunk)
if ("flush" in chunk) {
await buffer.flushAll()
unflushed = 0
continue
}
await write(chunk)
}
}
// A short body leaves the peer waiting for bytes that will never come; failing here
// means the connection is torn down instead, which it can tell apart.
if (written !== contentLength)
throw new Error(`Body is ${written} bytes but Content-Length declared ${contentLength}`)
}
export async function sendBody(
buffer: WriteBuffer,
encoder: TextEncoder,
body: Uint8Array | StreamBody | ReadableStream<Uint8Array>,
contentLength?: number,
): Promise<void> {
if (body instanceof Uint8Array) {
buffer.write(body)
} else if (contentLength !== undefined) {
await buffer.flush()
await writeCountedBody(buffer, encoder, body, contentLength)
} else if (body instanceof ReadableStream) {
await buffer.flush()
await writeChunkedBodyFromStream(buffer, body)
+1
View File
@@ -37,6 +37,7 @@ export {
} from "./client/index.js"
export {
Headers,
HttpProtocolError,
MutableHeaders,
methods,
normalizeHeaders,
+152 -13
View File
@@ -251,8 +251,8 @@ suite("ServerConnection", () => {
res.body = "hello world"
await conn.send(res)
// Close both sides so the ReadBuffer can detect EOF after draining headers
serverTransport.close()
clientTransport.close()
serverTransport.halfClose()
clientTransport.halfClose()
const clientBuf = new ReadBuffer(clientTransport)
await clientBuf.readLine() // status line
let hasContentLength = false
@@ -311,11 +311,24 @@ suite("ServerConnection", () => {
const clientBuf = new ReadBuffer(clientTransport)
const req = `GET / HTTP/1.1\r\n${requestHeaders}Connection: close\r\n\r\n`
await clientTransport.write(enc.encode(req))
clientTransport.close()
clientTransport.halfClose()
await conn.handle(handler)
return await clientBuf.readLine()
}
// Helper: send one malformed request, run handle(), return the status line the
// server answered with before closing.
async function rejectedStatusLine(request: string) {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const clientBuf = new ReadBuffer(clientTransport)
await clientTransport.write(enc.encode(request))
await conn.handle(async () => {
throw new Error("handler must not run for a malformed request")
})
return await clientBuf.readLine()
}
test("calls handler and sends response", async () => {
const statusLine = await requestResponse("Host: localhost\r\n", async ({ res }) => {
res.setStatus(200)
@@ -351,6 +364,48 @@ suite("ServerConnection", () => {
)
})
test("default headersTimeout closes an idle connection", async (t) => {
t.mock.timers.enable({ apis: ["setTimeout"] })
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const handled = conn.handle(async () => {})
t.mock.timers.tick(30_000)
await handled
assert.strictEqual(await clientTransport.read(), null, "peer should see a closed connection")
})
test("default keepAliveTimeout closes an idle keep-alive connection", async (t) => {
t.mock.timers.enable({ apis: ["setTimeout"] })
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const clientBuf = new ReadBuffer(clientTransport)
await clientTransport.write(enc.encode("GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"))
const handled = conn.handle(async ({ res }) => {
res.setStatus(204)
})
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 204 No Content")
// let the connection go idle before the keep-alive timer is armed
await new Promise((ok) => setImmediate(ok))
t.mock.timers.tick(60_000)
await handled
assert.strictEqual(await clientTransport.read(), null, "peer should see a closed connection")
})
test("default maxBodyLength rejects an oversized Content-Length", async () => {
const req = "POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: 1073741824\r\n\r\n"
assert.strictEqual(await rejectedStatusLine(req), "HTTP/1.1 413 Content Too Large")
})
test("default maxTotalHeaderSize rejects a header flood", async () => {
const pad = "a".repeat(1024)
let req = "GET / HTTP/1.1\r\nHost: localhost\r\n"
for (let i = 0; i < 100; i++) req += `X-Pad-${i}: ${pad}\r\n`
assert.strictEqual(
await rejectedStatusLine(req + "\r\n"),
"HTTP/1.1 431 Header Field Too Large",
)
})
test("unconsumed request body is drained before next keep-alive request", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
@@ -358,7 +413,7 @@ suite("ServerConnection", () => {
const req1 = `POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: ${body.length}\r\n\r\n${body}`
const req2 = "GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"
await clientTransport.write(enc.encode(req1 + req2))
clientTransport.close()
clientTransport.halfClose()
let count = 0
await conn.handle(async ({ res }) => {
count++
@@ -368,12 +423,96 @@ suite("ServerConnection", () => {
assert.strictEqual(count, 2, "both requests should be processed")
})
test("non-timeout parse error in handle() propagates to caller", async () => {
test("a malformed request line is answered with 400 and the connection closed", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const clientBuf = new ReadBuffer(clientTransport)
await clientTransport.write(enc.encode("BADLINE\r\n"))
clientTransport.close()
await assert.rejects(() => conn.handle(async () => {}), /Invalid request line/)
await assert.doesNotReject(() => conn.handle(async () => {}))
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 400 Bad Request")
assert.strictEqual(clientTransport.readEnded, true, "peer should see a closed connection")
})
test("an error that is not a protocol violation still propagates to the caller", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport, { maxLineLength: 8 })
await clientTransport.write(enc.encode("GET /aaaaaaaaaaaaaaaaaaaa HTTP/1.1\r\n"))
await assert.rejects(() => conn.handle(async () => {}), /Line too long/)
})
suite("malformed requests are rejected on the wire", () => {
test("both Transfer-Encoding and Content-Length", async () => {
const req =
"POST / HTTP/1.1\r\nHost: x\r\nTransfer-Encoding: chunked\r\nContent-Length: 5\r\n\r\n"
assert.strictEqual(await rejectedStatusLine(req), "HTTP/1.1 400 Bad Request")
})
test("a repeated Transfer-Encoding", async () => {
const req =
"POST / HTTP/1.1\r\nHost: x\r\nTransfer-Encoding: chunked\r\nTransfer-Encoding: chunked\r\n\r\n"
assert.strictEqual(await rejectedStatusLine(req), "HTTP/1.1 400 Bad Request")
})
test("whitespace between a field name and its colon", async () => {
const req = "POST / HTTP/1.1\r\nHost: x\r\nContent-Length : 5\r\n\r\nhello"
assert.strictEqual(await rejectedStatusLine(req), "HTTP/1.1 400 Bad Request")
})
test("the rejection closes the connection instead of reading a smuggled request", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const clientBuf = new ReadBuffer(clientTransport)
const targets: string[] = []
await clientTransport.write(
enc.encode(
"POST / HTTP/1.1\r\nHost: x\r\nTransfer-Encoding: chunked\r\nContent-Length: 6\r\n\r\n" +
"0\r\n\r\nGET /smuggled HTTP/1.1\r\nHost: x\r\n\r\n",
),
)
await conn.handle(async ({ req, res }) => {
targets.push(req.target)
res.setStatus(204)
})
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 400 Bad Request")
assert.deepStrictEqual(targets, [], "no request should reach the handler")
assert.strictEqual(clientTransport.readEnded, true, "peer should see a closed connection")
})
})
test("Transfer-Encoding: Chunked is framed as chunked", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
const clientBuf = new ReadBuffer(clientTransport)
await clientTransport.write(
enc.encode(
"POST / HTTP/1.1\r\nHost: x\r\nTransfer-Encoding: Chunked\r\nConnection: close\r\n\r\n" +
"5\r\nhello\r\n0\r\n\r\n",
),
)
let body = ""
await conn.handle(async ({ req, res }) => {
body = await req.text()
res.setStatus(204)
})
assert.strictEqual(body, "hello")
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 204 No Content")
})
test("a header value that would split the response never reaches the wire", async () => {
const [serverTransport, clientTransport] = loopbackTransportPair()
const conn = new ServerConnection(serverTransport)
await clientTransport.write(
enc.encode("GET / HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n"),
)
await assert.rejects(
() =>
conn.handle(async ({ res }) => {
res.setHeader("X-File", "a\r\nX-Injected: yes")
res.body = "OK"
}),
/must not contain CR, LF or NUL/,
)
assert.strictEqual(await clientTransport.read(), null, "nothing should have been written")
})
test("processes two requests on the same connection", async () => {
@@ -383,7 +522,7 @@ suite("ServerConnection", () => {
const req1 = "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"
const req2 = "GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"
await clientTransport.write(enc.encode(req1 + req2))
clientTransport.close()
clientTransport.halfClose()
let count = 0
await conn.handle(async ({ res }) => {
count++
@@ -532,7 +671,7 @@ suite("ServerConnection", () => {
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 100 Continue")
await clientBuf.readLine() // blank line after 100 Continue
await clientTransport.write(enc.encode(body))
clientTransport.close()
clientTransport.halfClose()
})(),
])
@@ -547,7 +686,7 @@ suite("ServerConnection", () => {
`POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: ${body.length}\r\n` +
`Expect: 100-continue\r\nConnection: close\r\n\r\n`
await clientTransport.write(enc.encode(req))
clientTransport.close()
clientTransport.halfClose()
const clientBuf = new ReadBuffer(clientTransport)
await conn.handle(async ({ res }) => {
@@ -569,7 +708,7 @@ suite("ServerConnection", () => {
`POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: ${body.length}\r\n` +
`Expect: 100-continue\r\n\r\n`
await clientTransport.write(enc.encode(req))
clientTransport.close()
clientTransport.halfClose()
await conn.handle(async ({ res }) => {
res.setStatus(400)
@@ -587,7 +726,7 @@ suite("ServerConnection", () => {
await clientTransport.write(
enc.encode("GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"),
)
clientTransport.close()
clientTransport.halfClose()
const clientBuf = new ReadBuffer(clientTransport)
await conn.handle(async ({ res }) => {
@@ -607,7 +746,7 @@ suite("ServerConnection", () => {
await clientTransport.write(
enc.encode("GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"),
)
clientTransport.close()
clientTransport.halfClose()
const clientBuf = new ReadBuffer(clientTransport)
await conn.handle(async ({ res }) => {
+51 -12
View File
@@ -1,6 +1,6 @@
import { ServerRequestImpl, ServerResponseImpl } from "./objects.js"
import type { RawTransport, Reader } from "@webnet/transport"
import { statusCodeProperties } from "../common/spec.js"
import { statusCodeProperties, statusCodes } from "../common/spec.js"
import {
prependTransport,
ReadBuffer,
@@ -14,11 +14,16 @@ import {
type BodyReaderOptions,
type ReadHeadersOptions,
} from "../common/reader.js"
import { formatBody, sendBody, writeHeaders } from "../common/writer.js"
import { declaredContentLength, formatBody, sendBody, writeHeaders } from "../common/writer.js"
import type { Handler, ServerHijackOptions, ServerRequest, ServerResponse } from "./types.js"
import { shouldClose } from "../common/connection.js"
import { TimeoutError, withTimeout } from "../common/utils.js"
import { HttpProtocolError, TimeoutError, withTimeout } from "../common/utils.js"
/**
* A server faces untrusted peers, so the limits inherited from the client-side
* option types (`bodyTimeout`, `maxBodyLength`, `maxTotalHeaderSize`) default to
* finite values here instead of `Infinity`. Pass `Infinity` explicitly to opt out.
*/
export type ServerConnectionOptions = ReadBufferOptions &
ReadHeadersOptions &
BodyReaderOptions & {
@@ -28,17 +33,25 @@ export type ServerConnectionOptions = ReadBufferOptions &
maxTargetLength?: number
/**
* Maximum time in ms to receive the request line and all headers.
* @defaultValue Infinity
* @defaultValue 30_000
*/
headersTimeout?: number
/**
* Maximum idle time in ms to wait for the next request on a keep-alive connection.
* Once the request line arrives, `headersTimeout` applies independently.
* @defaultValue Infinity
* @defaultValue 60_000
*/
keepAliveTimeout?: number
}
const DEFAULT_LIMITS = {
headersTimeout: 30_000,
keepAliveTimeout: 60_000,
bodyTimeout: 300_000,
maxBodyLength: 64 * 1024 * 1024,
maxTotalHeaderSize: 64 * 1024,
} satisfies ServerConnectionOptions
class ContinueBodyReader implements BodyReader {
#inner: BodyReader
#sendContinue: () => Promise<void>
@@ -91,15 +104,22 @@ export class ServerConnection {
#options: ServerConnectionOptions
constructor(transport: RawTransport, options: ServerConnectionOptions = {}) {
this.#options = {
...options,
headersTimeout: options.headersTimeout ?? DEFAULT_LIMITS.headersTimeout,
keepAliveTimeout: options.keepAliveTimeout ?? DEFAULT_LIMITS.keepAliveTimeout,
bodyTimeout: options.bodyTimeout ?? DEFAULT_LIMITS.bodyTimeout,
maxBodyLength: options.maxBodyLength ?? DEFAULT_LIMITS.maxBodyLength,
maxTotalHeaderSize: options.maxTotalHeaderSize ?? DEFAULT_LIMITS.maxTotalHeaderSize,
}
this.#transport = transport
this.#readBuffer = new ReadBuffer(transport, options)
this.#readBuffer = new ReadBuffer(transport, this.#options)
this.#writeBuffer = new WriteBuffer(transport)
this.#encoder = new TextEncoder()
this.#prevBody = null
this.#hijacked = false
this.#pendingContinue = null
this.#options = options
}
#takeTransport(): RawTransport {
@@ -115,17 +135,19 @@ export class ServerConnection {
async #parseFrom(requestLine: string): Promise<ServerRequestImpl> {
const parts = requestLine.split(" ")
if (parts.length !== 2 && parts.length !== 3)
throw new Error(`Invalid request line: ${requestLine.slice(0, 80)}`)
throw new HttpProtocolError(400, `Invalid request line: ${requestLine.slice(0, 80)}`)
const method = parts[0].toUpperCase()
const target = parts[1]
let version: "1.0" | "1.1" = "1.0"
if (parts.length === 3) {
if (parts[2] === "HTTP/1.1") version = "1.1"
else if (parts[2] !== "HTTP/1.0") throw new Error(`Invalid HTTP version: ${parts[2]}`)
else if (parts[2] !== "HTTP/1.0")
throw new HttpProtocolError(400, `Invalid HTTP version: ${parts[2]}`)
}
if (target.length >= this.#maxTargetLength)
throw new Error(
throw new HttpProtocolError(
414,
`Target max length exceeded: ${target.length} (max: ${this.#maxTargetLength})`,
)
@@ -193,7 +215,7 @@ export class ServerConnection {
// if we need to send a body, send it
if (response.request.method !== "HEAD" && scp.responseBody !== false && body) {
await sendBody(this.#writeBuffer, this.#encoder, body)
await sendBody(this.#writeBuffer, this.#encoder, body, declaredContentLength(response))
}
// and flush to make sure we sent everything
@@ -222,6 +244,14 @@ export class ServerConnection {
}
} catch (e) {
if (e instanceof TimeoutError) return
// A malformed message earns an answer before the connection goes away:
// dropping it silently leaves the peer unable to tell a rejection from a
// dead link, and leaves an intermediary free to keep its own reading of
// the bytes we refused.
if (e instanceof HttpProtocolError) {
await this.#reject(e.status)
return
}
throw e
}
first = false
@@ -265,6 +295,7 @@ export class ServerConnection {
const pendingContinue = this.#pendingContinue
if (shouldClose(req, res) || (pendingContinue !== null && !pendingContinue.sent)) {
await this.close()
return
} else if (this.#prevBody) {
// drain the request body to allow reuse
while (!this.#prevBody.closed) await this.#prevBody.read()
@@ -277,6 +308,14 @@ export class ServerConnection {
}
}
async #reject(status: number): Promise<void> {
this.#writeBuffer.write(
`HTTP/1.1 ${status} ${statusCodes[status] ?? "Bad Request"}\r\n` +
"Content-Length: 0\r\nConnection: close\r\n\r\n",
)
await this.#writeBuffer.flushAll()
}
async close(): Promise<void> {
if (this.#transport.closed) return
await this.#transport.close()
+49
View File
@@ -0,0 +1,49 @@
import test, { suite } from "node:test"
import assert from "node:assert"
import { ReadBuffer } from "@webnet/transport/buffer"
import { loopbackListener } from "@webnet/transport/loopback"
import { Server } from "./server.js"
const enc = new TextEncoder()
suite("Server", () => {
suite("listen()", () => {
test("onConnect returning false closes the transport", async () => {
const [listener, dialer] = loopbackListener()
let handled = false
const server = new Server(async ({ res }) => {
handled = true
res.body = "OK"
})
const listening = server.listen(listener, { onConnect: () => false })
const client = await dialer.dial("localhost", 80)
await client.write(enc.encode("GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"))
assert.strictEqual(await client.read(), null, "peer should see a closed connection")
assert.strictEqual(handled, false, "handler should not run for a rejected connection")
listener.close()
await listening
})
test("onConnect returning nothing accepts the connection", async () => {
const [listener, dialer] = loopbackListener()
const server = new Server(async ({ res }) => {
res.body = "OK"
})
const listening = server.listen(listener, { onConnect: () => {} })
const client = await dialer.dial("localhost", 80)
const clientBuf = new ReadBuffer(client)
await client.write(
enc.encode("GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"),
)
assert.strictEqual(await clientBuf.readLine(), "HTTP/1.1 200 OK")
listener.close()
await listening
})
})
})
+4 -1
View File
@@ -34,7 +34,10 @@ export class Server {
const handle = async (transport: RawTransport) => {
const connection = new ServerConnection(transport, options)
try {
if (onConnect(transport) === false) return
if (onConnect(transport) === false) {
await transport.close()
return
}
await connection.handle(this.#handler)
} catch (e) {
onError(connection, transport, e)
+34
View File
@@ -0,0 +1,34 @@
# @webnet/react
Shared React hooks and components for webnet apps.
It provides SSR-safe capability hooks built on `useSyncExternalStore` (`useClient`, `useSecureContext`, `useSharedWorkerAvailable`), a `useLocalStorage` hook that syncs a string value across tabs via the `storage` event, and error-serialization and display helpers (`serializeError`, `ErrorDetails`) for preserving and rendering caught errors, including causes and aggregate errors. `ErrorDetails` keeps the technical details collapsed by default; callers can pass a redactor to `serializeError` when error content may be sensitive.
## Entry points
| Entry point | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `@webnet/react` | Hooks and components: `useClient`, `useLocalStorage`, `useSecureContext`, `useSharedWorkerAvailable`, `serializeError`, `ErrorDetails` |
## Usage
```tsx
import { useClient, useLocalStorage, serializeError, ErrorDetails } from "@webnet/react"
function Panel() {
const isClient = useClient()
const [theme, setTheme] = useLocalStorage("theme", "light")
if (!isClient) return null
return <button onClick={() => setTheme("dark")}>Current theme: {theme}</button>
}
function Fallback({ error }: { error: unknown }) {
return <ErrorDetails error={serializeError(error)} />
}
```
## See also
- [`@webnet/utils`](../utils) — browser utilities without a React dependency
+41
View File
@@ -0,0 +1,41 @@
import type { ReactNode } from "react"
import type { SerializedError } from "./error.js"
function ErrorTrace({ error, label }: { error: SerializedError; label?: string }) {
return (
<>
<strong>{label ?? error.name}</strong>
<pre>{error.stack ?? `${error.name}: ${error.message}`}</pre>
{error.cause && <ErrorTrace error={error.cause} label="Cause" />}
{error.errors?.map((nested, index) => (
<ErrorTrace key={index} error={nested} label={`Error ${index + 1}`} />
))}
</>
)
}
export function ErrorDetails({
error,
reactComponentStack,
summary = "Technical details",
children,
}: {
error: SerializedError
reactComponentStack?: string | null
summary?: ReactNode
children?: ReactNode
}) {
return (
<details>
<summary>{summary}</summary>
<ErrorTrace error={error} />
{reactComponentStack && (
<>
<strong>React component stack</strong>
<pre>{reactComponentStack}</pre>
</>
)}
{children}
</details>
)
}
+37
View File
@@ -0,0 +1,37 @@
export type SerializedError = {
name: string
message: string
stack?: string
cause?: SerializedError
errors?: SerializedError[]
}
export type ErrorRedactor = (value: string) => string
function isErrorLike(error: unknown): error is {
name?: unknown
message?: unknown
stack?: unknown
cause?: unknown
errors?: unknown
} {
return typeof error === "object" && error !== null
}
export function serializeError(
error: unknown,
redact: ErrorRedactor = (value) => value,
): SerializedError {
if (!isErrorLike(error)) return { name: "Error", message: redact(String(error)) }
const serialized: SerializedError = {
name: typeof error.name === "string" ? redact(error.name) : "Error",
message: typeof error.message === "string" ? redact(error.message) : redact(String(error)),
}
if (typeof error.stack === "string") serialized.stack = redact(error.stack)
if ("cause" in error && error.cause !== undefined)
serialized.cause = serializeError(error.cause, redact)
if (Array.isArray(error.errors))
serialized.errors = error.errors.map((item) => serializeError(item, redact))
return serialized
}
+3
View File
@@ -1,3 +1,6 @@
export { ErrorDetails } from "./ErrorDetails.js"
export { type ErrorRedactor, serializeError, type SerializedError } from "./error.js"
export { useClient } from "./useClient.js"
export { useLocalStorage } from "./useLocalStorage.js"
export { useSecureContext } from "./useSecureContext.js"
export { useSharedWorkerAvailable } from "./useSharedWorkerAvailable.js"
+9
View File
@@ -0,0 +1,9 @@
import { useSyncExternalStore } from "react"
const subscribe = () => () => undefined
const getClientSnapshot = () => window.isSecureContext
const getServerSnapshot = () => true
export function useSecureContext(): boolean {
return useSyncExternalStore(subscribe, getClientSnapshot, getServerSnapshot)
}
+1
View File
@@ -4,6 +4,7 @@
"module": "ES2020",
"moduleResolution": "bundler",
"strict": true,
"jsx": "react-jsx",
"declaration": true,
"outDir": "dist",
"rootDir": "src",
+64
View File
@@ -0,0 +1,64 @@
# @webnet/sftp
SFTP (protocol version 3) client and server built on `@webnet/ssh` and `@webnet/vfs`.
`SFTPClient` implements `AsyncVFS` (`stat`, `readdir`, `readFile`/`readFileRange`, `writeFile`, `delete`, `mkdir`, `move`) over an SFTP session. It can be created three ways: `SFTPClient.connect` runs SFTP as a new subsystem channel on an `SSHClientConnection` the caller owns, `SFTPClient.fromChannel` runs it over a subsystem channel the caller already opened, or `new SFTPClient({ dialer, host, user, ... })` dials its own SSH connection. `SFTPServer` drives a full SSH server (via `@webnet/ssh`) and serves an `AsyncVFS` over the SFTP subsystem to accepted connections, with optional per-login `authenticate` to select a different VFS per credential; `serveSFTPChannel` runs the SFTP server protocol directly over a `Channel` for callers who already have an authenticated SSH connection. Errors surface as `SFTPError`, and `SSH_FX` holds the SFTP status codes.
## Entry points
| Entry point | Description |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| `@webnet/sftp` | `SFTPClient`, `SFTPServer`, `serveSFTPChannel`, `SFTPError`, `SSH_FX`, and their option types. |
| `@webnet/sftp/client` | `SFTPClient` and its option types only. |
| `@webnet/sftp/server` | `SFTPServer`, `serveSFTPChannel`, and their option types only. |
`_internals` entry points are unstable and are not part of the public API.
## Limits
The server bounds what one session can be made to hold. `SFTP_LIMITS` holds the defaults; `SFTPServerOptions` and the third argument to `serveSFTPChannel` override them per session. The SSH connection underneath applies [its own limits](../ssh#limits).
| What is bounded | Default | Option | When the peer exceeds it |
| -------------------------------- | ------- | --------------------- | ---------------------------------------------------------------- |
| Requests dispatched at once | 64 | `maxInflightRequests` | The server stops reading until a reply goes out. |
| Handles open per session | 256 | `maxOpenHandles` | `SSH_FXP_OPEN` and `SSH_FXP_OPENDIR` fail with `SSH_FX_FAILURE`. |
| Bytes one `SSH_FXP_READ` returns | 256 KiB | `maxReadLength` | The reply is clamped, which is a legal short read. |
| Bytes in one inbound packet | 1 MiB | — | The session fails. |
Pipelining is how SFTP clients are meant to reach full speed, so exceeding the in-flight cap throttles the peer through the SSH channel window instead of failing the session. The read cap matters because the packet-size limit bounds the request, not the reply it asks for: without it, a single 4 GiB `SSH_FXP_READ` is a 4 GiB allocation.
## Usage
### Client
```ts
import { SFTPClient } from "@webnet/sftp/client"
import type { RawDialer } from "@webnet/transport"
declare const dialer: RawDialer
const client = new SFTPClient({ dialer, host: "example.com", user: "u", password: "p" })
const entries = await client.readdir("/home/u")
const stream = await client.readFile("/home/u/file.txt")
await client.close()
```
### Server
```ts
import { SFTPServer } from "@webnet/sftp/server"
import type { RawListener } from "@webnet/transport"
import type { AsyncVFS } from "@webnet/vfs"
declare const vfs: AsyncVFS
declare const listener: RawListener
const server = new SFTPServer({ vfs })
await server.listen(listener)
```
## See also
- [`@webnet/ssh`](../ssh) — the SSH connections, channels, and subsystems this package runs over
- [`@webnet/vfs`](../vfs) — the `AsyncVFS` interface this package implements and serves
- [`@webnet/ftp`](../ftp) — an alternative file-transfer protocol using FTP/FTPS instead of SSH
+1 -1
View File
@@ -44,9 +44,9 @@
"typescript": "^6.0.2"
},
"dependencies": {
"@webnet/binary": "*",
"@webnet/ssh": "*",
"@webnet/transport": "*",
"@webnet/utils": "*",
"@webnet/vfs": "*"
}
}
+23 -7
View File
@@ -58,7 +58,7 @@ function streamOf(data: Uint8Array): ReadableStream<Uint8Array> {
async function serveFXP(
channel: Channel,
vfs: AsyncVFS,
opts: { shortRead?: number } = {},
opts: { shortRead?: number; posixRename?: boolean } = {},
): Promise<void> {
const reassembler = new PacketReassembler()
const handles = new Map<string, Handle>()
@@ -78,10 +78,9 @@ async function serveFXP(
const handle = async (type: number, payload: Uint8Array<ArrayBuffer>): Promise<void> => {
if (type === FXP.INIT) {
await send(
FXP.VERSION,
new Writer().u32(3).string("posix-rename@openssh.com").string("1").finish(),
)
const version = new Writer().u32(3)
if (opts.posixRename !== false) version.string("posix-rename@openssh.com").string("1")
await send(FXP.VERSION, version.finish())
return
}
const r = new Reader(payload)
@@ -255,7 +254,7 @@ async function serveFXP(
async function runServer(
listener: RawListener,
vfs: AsyncVFS,
opts: { shortRead?: number } = {},
opts: { shortRead?: number; posixRename?: boolean } = {},
): Promise<void> {
const raw = await listener.accept()
const hostKey = await generateHostKey()
@@ -274,7 +273,7 @@ async function runServer(
async function withClient(
fn: (client: SFTPClient, vfs: MemoryVFS) => Promise<void>,
opts: { shortRead?: number } = {},
opts: { shortRead?: number; posixRename?: boolean } = {},
): Promise<void> {
const [listener, dialer] = loopbackListener()
const vfs = new MemoryVFS()
@@ -438,6 +437,23 @@ suite("SFTPClient", () => {
})
})
test("move fallback preserves an overwrite destination when the source is missing", async () => {
await withClient(
async (client, vfs) => {
await seed(vfs, "/dst.txt", new TextEncoder().encode("keep"))
await assert.rejects(
client.move("/missing.txt", "/dst.txt", { overwrite: true }),
(e: unknown) => e instanceof VFSError && e.code === "not-found",
)
assert.deepEqual(
await readAll(await client.readFile("/dst.txt")),
new TextEncoder().encode("keep"),
)
},
{ posixRename: false },
)
})
test("utf8 filenames", async () => {
await withClient(async (client) => {
const name = "/文件-λ-📁.txt"
+49 -5
View File
@@ -1,5 +1,5 @@
import { VFSError, baseName, resolvePath, type AsyncVFS, type Stat } from "@webnet/vfs"
import { SSHAuthError, SSHClientConnection } from "@webnet/ssh"
import { SSHAuthError, SSHClientConnection, type Channel } from "@webnet/ssh"
import { Reader, Writer } from "../sftp/cursor.js"
import { FXP, SSH_FXF, attrsToStat, decodeAttrs, encodeAttrs } from "../sftp/packets.js"
import { SSH_FX, statusToVFSError } from "../sftp/status.js"
@@ -12,11 +12,27 @@ const MAX_INFLIGHT = 8
export class SFTPClient implements AsyncVFS {
readonly #options: SFTPClientOptions
#session: Promise<SftpSession> | null = null
// Only a connection this client dialed itself is closed by close().
#ownedConnection: SSHClientConnection | null = null
constructor(options: SFTPClientOptions) {
this.#options = options
}
/** Run SFTP over an SSH connection the caller owns and keeps. */
static async connect(connection: SSHClientConnection): Promise<SFTPClient> {
const client = new SFTPClient({ connection })
await client.#connect()
return client
}
/** Run SFTP over a subsystem channel the caller already opened. */
static async fromChannel(channel: Channel): Promise<SFTPClient> {
const client = new SFTPClient({ channel })
await client.#connect()
return client
}
#connect(): Promise<SftpSession> {
if (!this.#session) {
const session = this.#doConnect()
@@ -29,7 +45,20 @@ export class SFTPClient implements AsyncVFS {
}
async #doConnect(): Promise<SftpSession> {
const { dialer, host, port, user, password, privateKey, verifyHostKey } = this.#options
const options = this.#options
if ("channel" in options) return startSession(options.channel)
if ("connection" in options) {
// The connection is the caller's; only the channel we open is ours to
// clean up if the handshake fails.
const channel = await options.connection.openSubsystem("sftp")
try {
return await startSession(channel)
} catch (e) {
await channel.close().catch(() => {})
throw e
}
}
const { dialer, host, port, user, password, privateKey, verifyHostKey } = options
const raw = await dialer.dial(host, port ?? 22)
let connection: SSHClientConnection
try {
@@ -44,9 +73,8 @@ export class SFTPClient implements AsyncVFS {
throw e
}
try {
const channel = await connection.openSubsystem("sftp")
const session = new SftpSession(channel, connection)
await session.init()
const session = await startSession(await connection.openSubsystem("sftp"))
this.#ownedConnection = connection
return session
} catch (e) {
await connection.close().catch(() => {})
@@ -56,7 +84,9 @@ export class SFTPClient implements AsyncVFS {
async close(): Promise<void> {
const session = this.#session
const connection = this.#ownedConnection
this.#session = null
this.#ownedConnection = null
if (session) {
try {
await (await session).close()
@@ -64,6 +94,7 @@ export class SFTPClient implements AsyncVFS {
/* already closed or never connected */
}
}
await connection?.close().catch(() => {})
}
async stat(path: string): Promise<Stat> {
@@ -325,6 +356,11 @@ export class SFTPClient implements AsyncVFS {
return
}
if (recursive) {
// Kept rather than shared with the FTP client: `delete` is a required operation, so there is
// no optional one to fall back for, and the recursion is protocol-shaped — REMOVE and RMDIR
// are different requests chosen from the ATTRS reply this method already holds. A generic
// shim would restat every entry and still could not tell a symlink from what it points at,
// which is the difference that decides whether descending is correct at all.
for (const entry of await this.readdir(path)) await this.delete(entry.path, true)
}
expectOk(await session.request(FXP.RMDIR, new Writer().string(path).finish()), path, "rmdir")
@@ -351,6 +387,8 @@ export class SFTPClient implements AsyncVFS {
return
}
if (overwrite) {
const st = await session.request(FXP.STAT, new Writer().string(src).finish())
if (st.type !== FXP.ATTRS) throw statusError(st, src, "stat")
await session.request(FXP.REMOVE, new Writer().string(dest).finish())
}
const reply = await session.request(FXP.RENAME, new Writer().string(src).string(dest).finish())
@@ -358,6 +396,12 @@ export class SFTPClient implements AsyncVFS {
}
}
async function startSession(channel: Channel): Promise<SftpSession> {
const session = new SftpSession(channel)
await session.init()
return session
}
function expectOk(reply: SftpResponse, path: string, verb: string): void {
if (reply.type !== FXP.STATUS) throw statusError(reply, path, verb)
const r = new Reader(reply.payload)
+6 -1
View File
@@ -1,2 +1,7 @@
export { SFTPClient } from "./client.js"
export type { SFTPClientOptions } from "./types.js"
export type {
SFTPClientOptions,
SFTPDialOptions,
SFTPConnectionOptions,
SFTPChannelOptions,
} from "./types.js"
+4 -6
View File
@@ -1,4 +1,4 @@
import type { Channel, SSHClientConnection } from "@webnet/ssh"
import type { Channel } from "@webnet/ssh"
import { Reader, Writer } from "../sftp/cursor.js"
import { FXP, PacketReassembler, encodePacket, type ReassembledPacket } from "../sftp/packets.js"
@@ -8,7 +8,6 @@ type Resolver = { resolve: (v: SftpResponse) => void; reject: (e: unknown) => vo
export class SftpSession {
readonly #channel: Channel
readonly #connection: SSHClientConnection
readonly #reassembler = new PacketReassembler()
readonly #pending = new Map<number, Resolver>()
readonly #extensions = new Set<string>()
@@ -17,9 +16,8 @@ export class SftpSession {
#error: unknown = null
#versionResolver: { resolve: () => void; reject: (e: unknown) => void } | null = null
constructor(channel: Channel, connection: SSHClientConnection) {
constructor(channel: Channel) {
this.#channel = channel
this.#connection = connection
}
async init(): Promise<void> {
@@ -48,10 +46,10 @@ export class SftpSession {
return promise
}
// Closes only the SFTP channel. Whoever owns the SSH connection closes that.
async close(): Promise<void> {
this.#fail(new Error("sftp session closed"))
this.#channel.close().catch(() => {})
await this.#connection.close().catch(() => {})
await this.#channel.close().catch(() => {})
}
#allocId(): number {
+10 -1
View File
@@ -1,6 +1,7 @@
import type { RawDialer } from "@webnet/transport"
import type { Channel, SSHClientConnection } from "@webnet/ssh"
export type SFTPClientOptions = {
export type SFTPDialOptions = {
dialer: RawDialer
host: string
port?: number
@@ -13,3 +14,11 @@ export type SFTPClientOptions = {
fingerprint: string
}) => boolean | Promise<boolean>
}
/** Use an SSH connection the caller owns; SFTP opens its own subsystem channel. */
export type SFTPConnectionOptions = { connection: SSHClientConnection }
/** Use a subsystem channel the caller already opened. */
export type SFTPChannelOptions = { channel: Channel }
export type SFTPClientOptions = SFTPDialOptions | SFTPConnectionOptions | SFTPChannelOptions
+13 -3
View File
@@ -1,5 +1,15 @@
export { SFTPClient } from "./client/index.js"
export type { SFTPClientOptions } from "./client/index.js"
export { SFTPServer } from "./server/index.js"
export type { SFTPServerOptions, ListenOptions, Credential } from "./server/index.js"
export type {
SFTPClientOptions,
SFTPDialOptions,
SFTPConnectionOptions,
SFTPChannelOptions,
} from "./client/index.js"
export { SFTPServer, serveSFTPChannel, SFTP_LIMITS } from "./server/index.js"
export type {
SFTPServerLimits,
SFTPServerOptions,
ListenOptions,
Credential,
} from "./server/index.js"
export { SFTPError, SSH_FX } from "./sftp/status.js"
@@ -0,0 +1,196 @@
import { after, before, suite, test } from "node:test"
import assert from "node:assert/strict"
import { spawn, spawnSync } from "node:child_process"
import { mkdtemp, rm, writeFile } from "node:fs/promises"
import { tmpdir } from "node:os"
import { join } from "node:path"
import { nodeListen, type NodeListener } from "@webnet/transport/node"
import { MemoryVFS } from "@webnet/vfs/memory"
import type { AsyncVFS } from "@webnet/vfs"
import { generateHostKey, serveSession, SSHServerConnection, type Channel } from "@webnet/ssh"
import { serveSFTPChannel } from "./server/session.js"
const enc = new TextEncoder()
const dec = new TextDecoder()
const available = spawnSync("sftp", [], { stdio: "ignore" }).status !== null
function streamOf(text: string): ReadableStream<Uint8Array> {
return new ReadableStream({
start(controller) {
controller.enqueue(enc.encode(text))
controller.close()
},
})
}
async function readAll(stream: ReadableStream<Uint8Array>): Promise<string> {
const chunks: Uint8Array[] = []
const reader = stream.getReader()
for (;;) {
const { done, value } = await reader.read()
if (done) break
chunks.push(value)
}
return dec.decode(Buffer.concat(chunks))
}
type Fixture = {
port: number
keyPath: string
dir: string
vfs: MemoryVFS
listener: NodeListener
execCommands: string[]
stop: () => Promise<void>
}
let fixture: Fixture
// A server that routes session channels the way #108 asks: the `sftp`
// subsystem goes to serveSFTPChannel, exec to a command handler, on one
// connection with a single accept loop.
async function start(): Promise<Fixture> {
const dir = await mkdtemp(join(tmpdir(), "webnet-sftp-routed-"))
const keyPath = join(dir, "id_ed25519")
await writeFile(keyPath, await generateHostKey(), { mode: 0o600 })
const hostKey = await generateHostKey()
const listener = await nodeListen(0)
const port = Number(listener.addr.slice(listener.addr.lastIndexOf(":") + 1))
const vfs = new MemoryVFS()
const execCommands: string[] = []
const accept = async (): Promise<void> => {
for (;;) {
let raw
try {
raw = await listener.accept()
} catch {
return
}
void (async () => {
const connection = await SSHServerConnection.accept<AsyncVFS>(raw, {
hostKey,
authenticate: async (_user, credential) => (credential.kind === "publickey" ? vfs : null),
})
for (;;) {
let channel: Channel
try {
channel = await connection.acceptSession()
} catch {
return
}
void serveSession(channel, {
authorize: (start) => start.type !== "subsystem" || start.name === "sftp",
subsystem: (_name, s) => serveSFTPChannel(s.channel, connection.context),
exec: async (command, s) => {
execCommands.push(command)
await s.send(enc.encode(`ran:${command}\n`))
await s.exit(0)
},
})
}
})().catch(() => {})
}
}
void accept()
return {
port,
keyPath,
dir,
vfs,
listener,
execCommands,
stop: async () => {
await listener.close()
await rm(dir, { recursive: true, force: true })
},
}
}
function run(bin: string, args: string[], stdin?: string): Promise<string> {
const child = spawn(
bin,
[
"-F",
"/dev/null",
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
"IdentitiesOnly=yes",
"-o",
"BatchMode=yes",
"-i",
fixture.keyPath,
...args,
],
{ stdio: ["pipe", "pipe", "pipe"] },
)
const out: Buffer[] = []
const err: Buffer[] = []
child.stdout.on("data", (c: Buffer) => out.push(c))
child.stderr.on("data", (c: Buffer) => err.push(c))
child.stdin.end(stdin ?? "")
return new Promise((resolve, reject) => {
child.on("error", reject)
child.on("close", (code) => {
const stdout = dec.decode(Buffer.concat(out))
const stderr = dec.decode(Buffer.concat(err))
if (code !== 0) reject(new Error(`${bin} exited ${code}: ${stderr || stdout}`))
else resolve(stdout)
})
})
}
const sftp = (batch: string) =>
run("sftp", ["-P", String(fixture.port), "-b", "-", "u@127.0.0.1"], batch)
suite(
"routed sftp server against the OpenSSH client",
{ skip: !available && "the sftp binary is not installed" },
() => {
before(async () => {
fixture = await start()
})
after(async () => {
await fixture.stop()
})
test("the OpenSSH sftp client lists a routed server", async () => {
await fixture.vfs.writeFile("/listed.txt", streamOf("listed"))
const out = await sftp("ls\n")
assert.ok(out.includes("listed.txt"), out)
})
test("the OpenSSH sftp client uploads and downloads through the router", async () => {
const local = join(fixture.dir, "upload.txt")
await writeFile(local, "uploaded through the router")
await sftp(`put ${local} /uploaded.txt\n`)
assert.equal(
await readAll(await fixture.vfs.readFile("/uploaded.txt")),
"uploaded through the router",
)
await fixture.vfs.writeFile("/download.txt", streamOf("downloaded"))
const target = join(fixture.dir, "download.txt")
await sftp(`get /download.txt ${target}\n`)
const { readFile } = await import("node:fs/promises")
assert.equal(await readFile(target, "utf8"), "downloaded")
})
test("exec still works on the same routed server", async () => {
const out = await run("ssh", ["-p", String(fixture.port), "u@127.0.0.1", "uptime"])
assert.equal(out, "ran:uptime\n")
assert.ok(fixture.execCommands.includes("uptime"))
})
test("a non-sftp subsystem is refused without breaking the server", async () => {
await assert.rejects(run("ssh", ["-p", String(fixture.port), "-s", "u@127.0.0.1", "rexec"]))
const out = await run("ssh", ["-p", String(fixture.port), "u@127.0.0.1", "after-refusal"])
assert.equal(out, "ran:after-refusal\n")
})
},
)
+25 -13
View File
@@ -1,8 +1,10 @@
import { VFSError, normalizePath, type AsyncVFS } from "@webnet/vfs"
import { moveFallback } from "@webnet/vfs/fallback"
import { Reader } from "../sftp/cursor.js"
import { FXP, SSH_FXF, decodeAttrs, statToAttrs, type Attrs } from "../sftp/packets.js"
import { SSH_FX, vfsErrorToStatus } from "../sftp/status.js"
import { formatLongname } from "../sftp/longname.js"
import { SFTP_LIMITS, type SFTPServerLimits } from "./limits.js"
import {
HandleTable,
WriteQueue,
@@ -25,12 +27,15 @@ export type Responder = {
export class Handlers {
readonly #vfs: AsyncVFS
readonly #handles = new HandleTable()
readonly #handles: HandleTable
readonly #reply: Responder
readonly #maxReadLength: number
constructor(vfs: AsyncVFS, reply: Responder) {
constructor(vfs: AsyncVFS, reply: Responder, limits: SFTPServerLimits = {}) {
this.#vfs = vfs
this.#reply = reply
this.#handles = new HandleTable(limits.maxOpenHandles ?? SFTP_LIMITS.openHandles)
this.#maxReadLength = limits.maxReadLength ?? SFTP_LIMITS.readLength
}
async dispatch(type: number, id: number, r: Reader): Promise<void> {
@@ -89,6 +94,7 @@ export class Handlers {
const path = normalizePath(r.utf8())
const pflags = r.u32()
decodeAttrs(r)
if (this.#handles.full) throw new Error("too many open handles")
if (pflags & SSH_FXF.WRITE) {
if (!(pflags & SSH_FXF.CREAT)) await this.#vfs.stat(path)
const handle: WriteHandle = {
@@ -152,7 +158,10 @@ export class Handlers {
#read(id: number, r: Reader): Promise<void> {
const key = r.utf8()
const offset = r.u64()
const len = r.u32()
// A READ names the size of the reply, which MAX_PACKET_LENGTH does not
// bound; a short read is legal, so oversized asks are clamped rather than
// refused.
const len = Math.min(r.u32(), this.#maxReadLength)
const handle = this.#handles.get(key)
if (!handle || handle.kind !== "read") {
this.#reply.status(id, SSH_FX.NO_SUCH_FILE, "invalid handle")
@@ -235,6 +244,9 @@ export class Handlers {
async #opendir(id: number, r: Reader): Promise<void> {
const path = normalizePath(r.utf8())
// Checked before the readdir so a full table does not still pay for the
// enumeration; add() re-checks, since two opens can race past this.
if (this.#handles.full) throw new Error("too many open handles")
const entries = await this.#vfs.readdir(path)
this.#reply.handle(id, this.#handles.add({ kind: "dir", path, entries, cursor: 0 }))
}
@@ -285,10 +297,6 @@ export class Handlers {
async #rename(id: number, r: Reader): Promise<void> {
const oldPath = normalizePath(r.utf8())
const newPath = normalizePath(r.utf8())
if (!this.#vfs.move) {
this.#reply.status(id, SSH_FX.OP_UNSUPPORTED, "rename not supported")
return
}
let exists = true
try {
await this.#vfs.stat(newPath)
@@ -300,20 +308,24 @@ export class Handlers {
this.#reply.status(id, SSH_FX.FAILURE, `destination already exists: ${newPath}`)
return
}
await this.#vfs.move(oldPath, newPath)
await this.#move(oldPath, newPath, false)
this.#reply.status(id, SSH_FX.OK)
}
// native-only: RENAME is expected to be cheap and roughly atomic, and a shim would turn a
// rename of a large directory into a recursive copy and delete with no way for the client to
// know. The fallback is here so a backend without move and one that rejects unsupported for
// this path both reach the handler's error mapping as SSH_FX_OP_UNSUPPORTED.
#move(oldPath: string, newPath: string, overwrite: boolean): Promise<void> {
return moveFallback(this.#vfs, oldPath, newPath, { overwrite, policy: "native-only" })
}
async #extended(id: number, r: Reader): Promise<void> {
const extension = r.utf8()
if (extension === "posix-rename@openssh.com") {
const oldPath = normalizePath(r.utf8())
const newPath = normalizePath(r.utf8())
if (!this.#vfs.move) {
this.#reply.status(id, SSH_FX.OP_UNSUPPORTED, "rename not supported")
return
}
await this.#vfs.move(oldPath, newPath, { overwrite: true })
await this.#move(oldPath, newPath, true)
this.#reply.status(id, SSH_FX.OK)
return
}
+60 -1
View File
@@ -1,9 +1,23 @@
import { suite, test } from "node:test"
import assert from "node:assert/strict"
import { WriteQueue } from "./handles.js"
import { MemoryVFS } from "@webnet/vfs/memory"
import { withoutOptional, unsupportedOptional } from "@webnet/vfs/conformance"
import { HandleTable, WriteQueue, readFromHandle, type ReadHandle } from "./handles.js"
const bytes = (n: number) => new Uint8Array(n).fill(1) as Uint8Array<ArrayBuffer>
function newReadHandle(path: string): ReadHandle {
return {
kind: "read",
path,
leftover: new Uint8Array(0),
pos: 0n,
started: false,
eof: false,
mutex: Promise.resolve(),
}
}
suite("WriteQueue", () => {
test("streams chunks in order to the consumer", async () => {
const q = new WriteQueue()
@@ -43,3 +57,48 @@ suite("WriteQueue", () => {
await assert.rejects(blocked!, /writeFile failed/)
})
})
async function seeded(): Promise<MemoryVFS> {
const vfs = new MemoryVFS()
await vfs.writeFile(
"/data",
new ReadableStream({
start(c) {
c.enqueue(new TextEncoder().encode("abcdefghij"))
c.close()
},
}),
)
return vfs
}
suite("readFromHandle", () => {
test("reads the correct suffix at a nonzero offset when readFileRange is absent", async () => {
const vfs = withoutOptional(await seeded())
const out = await readFromHandle(vfs, newReadHandle("/data"), 3n, 4)
assert.equal(new TextDecoder().decode(out), "defg")
})
test("reads the correct suffix at a nonzero offset when readFileRange rejects unsupported", async () => {
const vfs = unsupportedOptional(await seeded(), ["readFileRange"])
const out = await readFromHandle(vfs, newReadHandle("/data"), 3n, 4)
assert.equal(new TextDecoder().decode(out), "defg")
})
})
suite("HandleTable", () => {
test("keys are eight hex digits and do not repeat", () => {
const table = new HandleTable(4)
const keys = [0, 1, 2, 3].map(() => table.add(newReadHandle("/f")))
assert.deepEqual(keys, ["00000000", "00000001", "00000002", "00000003"])
})
test("add() refuses to grow past the cap", () => {
const table = new HandleTable(1)
const key = table.add(newReadHandle("/f"))
assert.throws(() => table.add(newReadHandle("/g")), /too many open handles/)
table.remove(key)
assert.equal(table.full, false)
assert.ok(table.add(newReadHandle("/g")))
})
})
+23 -25
View File
@@ -1,4 +1,5 @@
import type { AsyncVFS, Stat } from "@webnet/vfs"
import { readFileRangeFallback } from "@webnet/vfs/fallback"
export type DirHandle = {
kind: "dir"
@@ -34,14 +35,24 @@ export type Handle = DirHandle | ReadHandle | WriteHandle
export class HandleTable {
readonly #map = new Map<string, Handle>()
readonly #max: number
#counter = 0
constructor(max: number) {
this.#max = max
}
add(handle: Handle): string {
const key = ((this.#counter++ >>> 0) >>> 0).toString(16).padStart(8, "0")
if (this.#map.size >= this.#max) throw new Error("too many open handles")
const key = (this.#counter++ >>> 0).toString(16).padStart(8, "0")
this.#map.set(key, handle)
return key
}
get full(): boolean {
return this.#map.size >= this.#max
}
get(key: string): Handle | undefined {
return this.#map.get(key)
}
@@ -119,33 +130,20 @@ export async function openReadStream(
if (handle.reader) await handle.reader.cancel().catch(() => {})
handle.leftover = new Uint8Array(0)
handle.eof = false
let stream: ReadableStream<Uint8Array>
let discard = 0n
if (offset === 0n) {
stream = await vfs.readFile(handle.path)
} else if (vfs.readFileRange) {
stream = await vfs.readFileRange(handle.path, offset)
} else {
stream = await vfs.readFile(handle.path)
discard = offset
}
// A zero offset wants the whole file, so it does not ask a backend that charges for a ranged
// read to open one. Above that the shared fallback windows the stream lazily: the discard used
// to run here, before the reader was handed back, holding the SSH_FXP_READ reply open for the
// whole prefix. The handle needs no bookkeeping for it — leftover starts empty either way, and
// a source shorter than the offset reads done on the first pull.
const stream =
offset === 0n
? await vfs.readFile(handle.path)
: await readFileRangeFallback(vfs, handle.path, offset, undefined, {
policy: "on-unsupported",
})
handle.reader = stream.getReader()
handle.started = true
handle.pos = offset
while (discard > 0n) {
const { value, done } = await handle.reader.read()
if (done) {
handle.eof = true
break
}
const chunk = value as Uint8Array<ArrayBuffer>
if (BigInt(chunk.length) <= discard) {
discard -= BigInt(chunk.length)
} else {
handle.leftover = chunk.subarray(Number(discard))
discard = 0n
}
}
}
export async function readFromHandle(
+2
View File
@@ -1,2 +1,4 @@
export { SFTPServer } from "./server.js"
export { serveSFTPChannel } from "./session.js"
export { SFTP_LIMITS, type SFTPServerLimits } from "./limits.js"
export type { SFTPServerOptions, ListenOptions, Credential } from "./types.js"
+14
View File
@@ -0,0 +1,14 @@
export const SFTP_LIMITS = {
/** Requests dispatched concurrently for one session before the peer is throttled. */
inflightRequests: 64,
/** Handles one session may hold open. */
openHandles: 256,
/** Largest reply a single SSH_FXP_READ can ask for, matching OpenSSH. */
readLength: 256 * 1024,
} as const
export type SFTPServerLimits = {
maxInflightRequests?: number
maxOpenHandles?: number
maxReadLength?: number
}
+195 -2
View File
@@ -4,10 +4,12 @@ import { loopbackListener, loopbackTransportPair } from "@webnet/transport/loopb
import type { RawDialer, RawListener } from "@webnet/transport"
import { MemoryVFS } from "@webnet/vfs/memory"
import type { AsyncVFS } from "@webnet/vfs"
import { withoutOptional, unsupportedOptional } from "@webnet/vfs/conformance"
import {
Channel,
generateHostKey,
SSHClientConnection,
SSHServerConnection,
SSHAuthError,
type HostKeyVerifier,
} from "@webnet/ssh"
@@ -23,7 +25,7 @@ import {
} from "../sftp/packets.js"
import { SSH_FX } from "../sftp/status.js"
import { SFTPServer } from "./server.js"
import { Session } from "./session.js"
import { TransportSession, serveSFTPChannel } from "./session.js"
import type { SFTPServerOptions } from "./types.js"
type Response = { type: number; r: Reader }
@@ -216,6 +218,16 @@ function encoder(s: string): Uint8Array {
return new TextEncoder().encode(s)
}
function streamOf(s: string): ReadableStream<Uint8Array> {
const data = encoder(s)
return new ReadableStream({
start(c) {
c.enqueue(data)
c.close()
},
})
}
async function withServer(
options: Partial<SFTPServerOptions> & { vfs: AsyncVFS },
fn: (dialer: RawDialer, listener: RawListener) => Promise<void>,
@@ -370,6 +382,60 @@ suite("SFTPServer", () => {
})
})
test("RENAME reports OP_UNSUPPORTED the same way for an absent move and one that rejects unsupported", async () => {
const base = new MemoryVFS()
await base.writeFile("/from.txt", streamOf("data"))
await withServer({ vfs: withoutOptional(base) }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const res = await client.request(
FXP.RENAME,
new Writer().string("/from.txt").string("/to.txt"),
)
assert.equal(statusCode(res), SSH_FX.OP_UNSUPPORTED)
await client.close()
})
const base2 = new MemoryVFS()
await base2.writeFile("/from.txt", streamOf("data"))
await withServer({ vfs: unsupportedOptional(base2, ["move"]) }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const res = await client.request(
FXP.RENAME,
new Writer().string("/from.txt").string("/to.txt"),
)
assert.equal(statusCode(res), SSH_FX.OP_UNSUPPORTED)
await client.close()
})
})
test("posix-rename extension reports OP_UNSUPPORTED the same way for an absent move and one that rejects unsupported", async () => {
const base = new MemoryVFS()
await base.writeFile("/src.txt", streamOf("new"))
await base.writeFile("/dst.txt", streamOf("old"))
await withServer({ vfs: withoutOptional(base) }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const res = await client.request(
FXP.EXTENDED,
new Writer().string("posix-rename@openssh.com").string("/src.txt").string("/dst.txt"),
)
assert.equal(statusCode(res), SSH_FX.OP_UNSUPPORTED)
await client.close()
})
const base2 = new MemoryVFS()
await base2.writeFile("/src.txt", streamOf("new"))
await base2.writeFile("/dst.txt", streamOf("old"))
await withServer({ vfs: unsupportedOptional(base2, ["move"]) }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const res = await client.request(
FXP.EXTENDED,
new Writer().string("posix-rename@openssh.com").string("/src.txt").string("/dst.txt"),
)
assert.equal(statusCode(res), SSH_FX.OP_UNSUPPORTED)
await client.close()
})
})
test("REALPATH of . resolves to /", async () => {
await withServer({ vfs: new MemoryVFS() }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
@@ -514,7 +580,7 @@ suite("SFTPServer", () => {
test("an authentication failure closes the connection instead of leaking it", async () => {
const [clientTransport, serverTransport] = loopbackTransportPair()
const running = new Session(serverTransport, {
const running = new TransportSession(serverTransport, {
vfs: new MemoryVFS(),
hostKey: await generateHostKey(),
authenticate: async () => null,
@@ -530,4 +596,131 @@ suite("SFTPServer", () => {
await assert.rejects(running)
assert.equal(serverTransport.closed, true)
})
test("an oversized READ is clamped instead of allocating what it asks for", async () => {
const vfs = new MemoryVFS()
await vfs.writeFile("/f", streamOf("hello"))
await withServer({ vfs }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const handle = await openReadHandle(client, "/f")
// MAX_PACKET_LENGTH bounds the request, not the reply it asks for.
const res = await client.request(
FXP.READ,
new Writer().string(handle).u64(0n).u32(0xffffffff),
)
assert.equal(res.type, FXP.DATA)
assert.equal(new TextDecoder().decode(res.r.string()), "hello")
await client.close()
})
})
test("maxReadLength caps the reply size", async () => {
const vfs = new MemoryVFS()
await vfs.writeFile("/f", streamOf("abcdefghij"))
await withServer({ vfs, maxReadLength: 4 }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const handle = await openReadHandle(client, "/f")
const res = await client.request(FXP.READ, new Writer().string(handle).u64(0n).u32(64))
assert.equal(res.type, FXP.DATA)
assert.equal(new TextDecoder().decode(res.r.string()), "abcd")
await client.close()
})
})
test("opens past maxOpenHandles fail instead of growing the table", async () => {
const vfs = new MemoryVFS()
await vfs.writeFile("/f", streamOf("x"))
await withServer({ vfs, maxOpenHandles: 2 }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
await openReadHandle(client, "/f")
await openReadHandle(client, "/f")
const res = await client.request(
FXP.OPEN,
new Writer().string("/f").u32(SSH_FXF.READ).bytes(encodeAttrs({})),
)
assert.equal(statusCode(res), SSH_FX.FAILURE)
const opendir = await client.request(FXP.OPENDIR, new Writer().string("/"))
assert.equal(statusCode(opendir), SSH_FX.FAILURE)
await client.close()
})
})
test("a pipelining client is throttled, not failed, above the in-flight cap", async () => {
const vfs = new MemoryVFS()
await vfs.writeFile("/f", streamOf("abcdefghij"))
await withServer({ vfs, maxInflightRequests: 4 }, async (dialer) => {
const client = await MockClient.connect(dialer, { user: "u", password: "p" })
const handle = await openReadHandle(client, "/f")
const reads = []
for (let i = 0; i < 200; i++) {
reads.push(client.request(FXP.READ, new Writer().string(handle).u64(0n).u32(10)))
}
for (const res of await Promise.all(reads)) {
assert.equal(res.type, FXP.DATA)
assert.equal(new TextDecoder().decode(res.r.string()), "abcdefghij")
}
await client.close()
})
})
test("the in-flight cap stalls the peer's writes instead of buffering them", async () => {
let release = (): void => {}
const gate = new Promise<void>((r) => (release = r))
const vfs = new MemoryVFS()
// OPEN stats the path, so every request parks in the handler until released.
const gated = new Proxy(vfs, {
get(target, prop) {
if (prop === "stat") return async (path: string) => (await gate, target.stat(path))
const value = Reflect.get(target, prop, target)
return typeof value === "function" ? value.bind(target) : value
},
}) as AsyncVFS
const [a, b] = loopbackTransportPair()
const hostKey = await generateHostKey()
const [client, server] = await Promise.all([
SSHClientConnection.connect(a, { user: "u", password: "p" }),
SSHServerConnection.accept<AsyncVFS>(b, { hostKey, authenticate: async () => gated }),
])
const serving = (async () => {
const channel = await server.acceptSession()
await channel.acceptSubsystem("sftp")
await serveSFTPChannel(channel, server.context, { maxInflightRequests: 1 })
})()
const channel = await client.openSubsystem("sftp")
try {
// Each request is a quarter of the 1 MiB channel window, so a server that
// has stopped reading runs the window down within a handful of them.
const request = (id: number): Uint8Array<ArrayBuffer> =>
encodePacket(
FXP.OPEN,
new Writer()
.u32(id)
.string("/" + "a".repeat(256 * 1024))
.u32(SSH_FXF.READ)
.bytes(encodeAttrs({}))
.finish(),
)
let stalled: Promise<void> | null = null
for (let i = 0; i < 10 && !stalled; i++) {
const send = channel.send(request(i))
let settled = false
void send.then(
() => (settled = true),
() => (settled = true),
)
await new Promise((r) => setTimeout(r, 50))
if (!settled) stalled = send
}
assert.ok(stalled, "expected the client's send to block on the channel window")
// Backpressure, not failure: releasing the handlers lets the write finish.
release()
await stalled
} finally {
release()
await client.close().catch(() => {})
await server.close().catch(() => {})
await serving.catch(() => {})
}
})
})
+2 -2
View File
@@ -1,6 +1,6 @@
import type { RawListener, RawTransport } from "@webnet/transport"
import { generateHostKey } from "@webnet/ssh"
import { Session } from "./session.js"
import { TransportSession } from "./session.js"
import type { ListenOptions, SFTPServerOptions } from "./types.js"
const defaultOnError: NonNullable<ListenOptions["onError"]> = (transport, error) => {
@@ -32,7 +32,7 @@ export class SFTPServer {
return
}
const hostKey = await this.#getHostKey()
await new Session(transport, { ...this.#options, hostKey }).run()
await new TransportSession(transport, { ...this.#options, hostKey }).run()
} catch (e) {
onError(transport, e)
}
+78 -35
View File
@@ -1,41 +1,47 @@
import type { RawTransport } from "@webnet/transport"
import type { AsyncVFS } from "@webnet/vfs"
import { SSHServerConnection, type Channel } from "@webnet/ssh"
import { SSHServerConnection, WorkLimit, type Channel } from "@webnet/ssh"
import { Reader, Writer } from "../sftp/cursor.js"
import { FXP, PacketReassembler, encodeAttrs, encodePacket } from "../sftp/packets.js"
import { Handlers, type NameEntry, type Responder } from "./handlers.js"
import { SFTP_LIMITS, type SFTPServerLimits } from "./limits.js"
import type { SFTPServerOptions } from "./types.js"
const VERSION = 3
/**
* Serve SFTP on an already-accepted `sftp` subsystem channel, using a VFS the
* caller has already authorized. Performs no SSH authentication and never
* touches the underlying connection, so one SSH connection can carry this
* alongside exec, shell, and forwarding channels.
*/
export async function serveSFTPChannel(
channel: Channel,
vfs: AsyncVFS,
limits: SFTPServerLimits = {},
): Promise<void> {
await new Session(channel, vfs, limits).serve()
}
export class Session {
readonly #transport: RawTransport
readonly #options: SFTPServerOptions
readonly #channel: Channel
readonly #vfs: AsyncVFS
readonly #limits: SFTPServerLimits
readonly #inflight: WorkLimit
readonly #reassembler = new PacketReassembler()
#sendChain: Promise<void> = Promise.resolve()
#closed = false
constructor(transport: RawTransport, options: SFTPServerOptions) {
this.#transport = transport
this.#options = options
constructor(channel: Channel, vfs: AsyncVFS, limits: SFTPServerLimits = {}) {
this.#channel = channel
this.#vfs = vfs
this.#limits = limits
this.#inflight = new WorkLimit(limits.maxInflightRequests ?? SFTP_LIMITS.inflightRequests)
}
async run(): Promise<void> {
let connection: SSHServerConnection<AsyncVFS> | undefined
let channel: Channel | undefined
async serve(): Promise<void> {
const ch = this.#channel
try {
connection = await SSHServerConnection.accept<AsyncVFS>(this.#transport, {
hostKey: this.#options.hostKey,
authenticate: async (user, credential) =>
this.#options.authenticate
? this.#options.authenticate(user, credential)
: this.#options.vfs,
})
const vfs = connection.context
channel = await connection.acceptSession()
await channel.acceptSubsystem("sftp")
const ch = channel
const enqueue = (type: number, body: Writer): void => {
if (this.#closed) return
const packet = encodePacket(type, body.finish())
@@ -54,13 +60,20 @@ export class Session {
name: (id, entries) => enqueue(FXP.NAME, encodeName(id, entries)),
attrs: (id, attrs) => enqueue(FXP.ATTRS, new Writer().u32(id).bytes(encodeAttrs(attrs))),
}
const handlers = new Handlers(vfs, responder)
const handlers = new Handlers(this.#vfs, responder, this.#limits)
const closed = ch.whenClosed
for (;;) {
const chunk = await ch.read()
if (chunk === null) break
for (const packet of this.#reassembler.feed(chunk)) {
if (this.#closed) break
// Pipelining is how SFTP clients are meant to behave, so the cap
// stops reading and lets the channel window slow the peer down
// rather than failing the session. A peer that vanishes mid-stall
// must not leave the loop waiting for a slot forever.
await Promise.race([this.#inflight.acquire(), closed])
if (ch.closed) break
this.#handle(handlers, enqueue, packet.type, packet.payload)
}
if (this.#closed) break
@@ -68,14 +81,8 @@ export class Session {
} finally {
this.#closed = true
await this.#drainSends()
if (channel && connection) await this.#teardown(channel, connection)
else if (connection) await connection.close().catch(() => {})
else
try {
await this.#transport.close()
} catch {
/* already closed */
}
await ch.closeSend().catch(() => {})
await ch.close().catch(() => {})
}
}
@@ -87,21 +94,57 @@ export class Session {
): void {
if (type === FXP.INIT) {
enqueue(FXP.VERSION, new Writer().u32(VERSION).string("posix-rename@openssh.com").string("1"))
this.#inflight.release()
return
}
const r = new Reader(payload)
const id = r.u32()
void handlers.dispatch(type, id, r)
// The slot is held until the reply has actually gone out. Releasing it when
// the handler returns would only bound the handlers, letting #sendChain
// grow instead while the peer withholds window credit.
void handlers
.dispatch(type, id, r)
.then(() => this.#sendChain)
.finally(() => this.#inflight.release())
}
async #drainSends(): Promise<void> {
await this.#sendChain.catch(() => {})
}
}
async #teardown(channel: Channel, connection: SSHServerConnection<AsyncVFS>): Promise<void> {
await channel.closeSend().catch(() => {})
await channel.close().catch(() => {})
await connection.close().catch(() => {})
/** Owns the whole SSH lifecycle for one accepted transport: the listen() path. */
export class TransportSession {
readonly #transport: RawTransport
readonly #options: SFTPServerOptions
constructor(transport: RawTransport, options: SFTPServerOptions) {
this.#transport = transport
this.#options = options
}
async run(): Promise<void> {
let connection: SSHServerConnection<AsyncVFS> | undefined
try {
connection = await SSHServerConnection.accept<AsyncVFS>(this.#transport, {
hostKey: this.#options.hostKey,
authenticate: async (user, credential) =>
this.#options.authenticate
? this.#options.authenticate(user, credential)
: this.#options.vfs,
})
const channel = await connection.acceptSession()
await channel.acceptSubsystem("sftp")
await serveSFTPChannel(channel, connection.context, this.#options)
} finally {
if (connection) await connection.close().catch(() => {})
else
try {
await this.#transport.close()
} catch {
/* already closed */
}
}
}
}
+2 -1
View File
@@ -1,8 +1,9 @@
import type { RawTransport } from "@webnet/transport"
import type { AsyncVFS } from "@webnet/vfs"
import type { Credential } from "@webnet/ssh"
import type { SFTPServerLimits } from "./limits.js"
export type SFTPServerOptions = {
export type SFTPServerOptions = SFTPServerLimits & {
vfs: AsyncVFS
authenticate?: (user: string, credential: Credential) => Promise<AsyncVFS | null>
hostKey?: string
+46 -24
View File
@@ -3,7 +3,11 @@ import assert from "node:assert/strict"
import { loopbackListener } from "@webnet/transport/loopback"
import { MemoryVFS } from "@webnet/vfs/memory"
import { VFSError, type AsyncVFS } from "@webnet/vfs"
import { testAsyncVFSConformance } from "@webnet/vfs/conformance"
import {
testAsyncVFSConformance,
withoutOptional,
type ConformanceCapabilities,
} from "@webnet/vfs/conformance"
import { SFTPClient } from "./client/client.js"
import { SFTPServer } from "./server/server.js"
import type { SFTPServerOptions } from "./server/types.js"
@@ -16,29 +20,47 @@ async function sharedHostKey(): Promise<string> {
return hostKey
}
testAsyncVFSConformance({
name: "SFTPClient",
capabilities: { etag: false },
// SFTP v3's RENAME has no overwrite semantics; the server pre-checks the destination and
// fails the same way (SSH_FX_FAILURE) whether or not the source itself would also conflict,
// so the client can only report "already-exists" for this case.
errorCodes: { "precondition-failed": ["precondition-failed", "already-exists"] },
create: async () => {
const vfs = new MemoryVFS()
const [listener, dialer] = loopbackListener()
const server = new SFTPServer({ vfs, hostKey: await sharedHostKey() })
const stopped = server.listen(listener, { onError: () => {} })
const client = new SFTPClient({ dialer, host: "test", user: "user", password: "pw" })
return {
vfs: client,
close: async () => {
await client.close().catch(() => {})
listener.close()
await stopped.catch(() => {})
},
}
},
})
function conformanceRun(
name: string,
backing: () => AsyncVFS,
capabilities: ConformanceCapabilities = {},
): void {
testAsyncVFSConformance({
name,
capabilities: { etag: false, ...capabilities },
// SFTP v3's RENAME has no overwrite semantics; the server pre-checks the destination and
// fails the same way (SSH_FX_FAILURE) whether or not the source itself would also conflict,
// so the client can only report "already-exists" for this case.
errorCodes: { "precondition-failed": ["precondition-failed", "already-exists"] },
create: async () => {
const vfs = backing()
const [listener, dialer] = loopbackListener()
const server = new SFTPServer({ vfs, hostKey: await sharedHostKey() })
const stopped = server.listen(listener, { onError: () => {} })
const client = new SFTPClient({ dialer, host: "test", user: "user", password: "pw" })
return {
vfs: client,
close: async () => {
await client.close().catch(() => {})
listener.close()
await stopped.catch(() => {})
},
}
},
})
}
conformanceRun("SFTPClient", () => new MemoryVFS())
// The same client against a server that has only the required operations to work with, so the
// server's fallback paths are held to the same contract as its native ones.
// RENAME answers SSH_FX_OP_UNSUPPORTED rather than renaming by hand when the filesystem cannot
// move, so the client reports the operation as unsupported. Whether to shim it instead is #179.
conformanceRun(
"SFTPClient (server over a minimal filesystem)",
() => withoutOptional(new MemoryVFS()),
{ move: "unsupported" },
)
type Harness = {
client: SFTPClient
+1 -1
View File
@@ -1,4 +1,4 @@
import { LengthPrefixedBinaryReader, LengthPrefixedBinaryWriter } from "@webnet/utils"
import { LengthPrefixedBinaryReader, LengthPrefixedBinaryWriter } from "@webnet/binary"
export class Writer extends LengthPrefixedBinaryWriter {
constructor(capacity = 256) {
+7 -2
View File
@@ -48,9 +48,14 @@ suite("statusToVFSError", () => {
const e = statusToVFSError(SSH_FX.FAILURE, "boom", "/a")
assert.ok(e instanceof SFTPError)
})
test("other codes -> SFTPError", () => {
test("OP_UNSUPPORTED -> unsupported", () => {
const e = statusToVFSError(SSH_FX.OP_UNSUPPORTED, "nope", "/a")
assert.ok(e instanceof VFSError)
assert.equal((e as VFSError).code, "unsupported")
})
test("other codes -> SFTPError", () => {
const e = statusToVFSError(SSH_FX.BAD_MESSAGE, "nope", "/a")
assert.ok(e instanceof SFTPError)
assert.equal((e as SFTPError).code, SSH_FX.OP_UNSUPPORTED)
assert.equal((e as SFTPError).code, SSH_FX.BAD_MESSAGE)
})
})
+2
View File
@@ -31,6 +31,7 @@ const CODE_BY_VFS_ERROR: Record<VFSErrorCode, number> = {
"not-empty": SSH_FX.FAILURE,
"precondition-failed": SSH_FX.FAILURE,
locked: SSH_FX.FAILURE,
unsupported: SSH_FX.OP_UNSUPPORTED,
}
const KNOWN_VFS_ERROR_CODES = new Set<string>(Object.keys(CODE_BY_VFS_ERROR))
@@ -53,6 +54,7 @@ export function statusToVFSError(
): VFSError | SFTPError {
if (code === SSH_FX.NO_SUCH_FILE) return new VFSError("not-found", `${message} (${path})`)
if (code === SSH_FX.PERMISSION_DENIED) return new VFSError("forbidden", message)
if (code === SSH_FX.OP_UNSUPPORTED) return new VFSError("unsupported", message)
if (code === SSH_FX.FAILURE) {
// vfsErrorToStatus embeds the original VFSErrorCode as `${code}: ${message}` for the
// FAILURE bucket since it covers several distinct VFS error codes. Recover it when the
+281
View File
@@ -0,0 +1,281 @@
import { suite, test } from "node:test"
import assert from "node:assert/strict"
import { loopbackTransportPair } from "@webnet/transport/loopback"
import { MemoryVFS } from "@webnet/vfs/memory"
import type { AsyncVFS } from "@webnet/vfs"
import {
generateHostKey,
serveSession,
SSHClientConnection,
SSHServerConnection,
type Channel,
type ServerSession,
} from "@webnet/ssh"
import { SFTPClient } from "./client/client.js"
import { serveSFTPChannel } from "./server/session.js"
const enc = new TextEncoder()
const dec = new TextDecoder()
function streamOf(text: string): ReadableStream<Uint8Array> {
return new ReadableStream({
start(controller) {
controller.enqueue(enc.encode(text))
controller.close()
},
})
}
async function readAll(stream: ReadableStream<Uint8Array>): Promise<string> {
const chunks: Uint8Array[] = []
const reader = stream.getReader()
for (;;) {
const { done, value } = await reader.read()
if (done) break
chunks.push(value)
}
return dec.decode(Buffer.concat(chunks))
}
type Routed = {
client: SSHClientConnection
server: SSHServerConnection<AsyncVFS>
vfs: MemoryVFS
close: () => Promise<void>
}
/**
* One SSH connection whose session channels are routed by `serveSession`:
* `sftp` goes to the SFTP server, `exec` to a trivial echo command. This is
* the dispatch shape #108 asks #103 to coordinate with.
*/
async function routed(): Promise<Routed> {
const [a, b] = loopbackTransportPair()
const hostKey = await generateHostKey()
const vfs = new MemoryVFS()
const [client, server] = await Promise.all([
SSHClientConnection.connect(a, { user: "u", password: "p" }),
SSHServerConnection.accept<AsyncVFS>(b, { hostKey, authenticate: async () => vfs }),
])
const loop = (async () => {
for (;;) {
let channel: Channel
try {
channel = await server.acceptSession()
} catch {
return
}
void serveSession(channel, {
authorize: (start) => start.type !== "subsystem" || start.name === "sftp",
subsystem: (_name, s: ServerSession) => serveSFTPChannel(s.channel, server.context),
exec: async (command, s) => {
await s.send(enc.encode(`ran:${command}`))
await s.exit(0)
},
})
}
})()
return {
client,
server,
vfs,
close: async () => {
await client.close().catch(() => {})
await server.close().catch(() => {})
await loop
},
}
}
suite("sftp over a shared ssh connection", () => {
test("SFTPClient.connect runs over a caller-owned connection", async () => {
const { client, vfs, close } = await routed()
try {
const sftp = await SFTPClient.connect(client)
await sftp.writeFile("/hello.txt", streamOf("shared"))
assert.equal(await readAll(await sftp.readFile("/hello.txt")), "shared")
assert.equal(await readAll(await vfs.readFile("/hello.txt")), "shared")
await sftp.close()
} finally {
await close()
}
})
test("SFTPClient.fromChannel runs over a pre-opened subsystem channel", async () => {
const { client, close } = await routed()
try {
const channel = await client.openSubsystem("sftp")
const sftp = await SFTPClient.fromChannel(channel)
await sftp.writeFile("/from-channel.txt", streamOf("preopened"))
assert.equal(await readAll(await sftp.readFile("/from-channel.txt")), "preopened")
await sftp.close()
} finally {
await close()
}
})
test("sftp and exec share one connection concurrently", async () => {
const { client, close } = await routed()
try {
const sftp = await SFTPClient.connect(client)
const [listing, execOut] = await Promise.all([
(async () => {
await sftp.writeFile("/a.txt", streamOf("a"))
return (await sftp.readdir("/")).map((s) => s.name)
})(),
(async () => {
const result = await client.run("uptime")
return dec.decode(result.stdout)
})(),
])
assert.deepEqual(listing, ["a.txt"])
assert.equal(execOut, "ran:uptime")
await sftp.close()
} finally {
await close()
}
})
test("multiple sftp channels share one connection independently", async () => {
const { client, close } = await routed()
try {
const [first, second] = await Promise.all([
SFTPClient.connect(client),
SFTPClient.connect(client),
])
await first.writeFile("/first.txt", streamOf("one"))
await second.writeFile("/second.txt", streamOf("two"))
// Closing one leaves the shared connection and the other client alive.
await first.close()
assert.equal(await readAll(await second.readFile("/first.txt")), "one")
await second.writeFile("/third.txt", streamOf("three"))
assert.deepEqual((await second.readdir("/")).map((s) => s.name).sort(), [
"first.txt",
"second.txt",
"third.txt",
])
await second.close()
// The connection itself is untouched by SFTP client teardown.
const result = await client.run("still-alive")
assert.equal(dec.decode(result.stdout), "ran:still-alive")
} finally {
await close()
}
})
test("closing an SFTP client never closes a caller-owned connection", async () => {
const { client, close } = await routed()
try {
const sftp = await SFTPClient.connect(client)
await sftp.close()
const again = await SFTPClient.connect(client)
await again.writeFile("/after.txt", streamOf("after"))
assert.equal(await readAll(await again.readFile("/after.txt")), "after")
await again.close()
} finally {
await close()
}
})
test("a failed init leaves the caller-owned connection usable", async () => {
const [a, b] = loopbackTransportPair()
const hostKey = await generateHostKey()
const vfs = new MemoryVFS()
const [client, server] = await Promise.all([
SSHClientConnection.connect(a, { user: "u", password: "p" }),
SSHServerConnection.accept<AsyncVFS>(b, { hostKey, authenticate: async () => vfs }),
])
let refuseSftp = true
const loop = (async () => {
for (;;) {
let channel: Channel
try {
channel = await server.acceptSession()
} catch {
return
}
void serveSession(channel, {
// Refuse the first subsystem request outright, then allow it.
authorize: (start) => {
if (start.type !== "subsystem") return true
const allow = !refuseSftp
refuseSftp = false
return allow
},
subsystem: (_name, s) => serveSFTPChannel(s.channel, vfs),
exec: async (command, s) => {
await s.send(enc.encode(`ran:${command}`))
await s.exit(0)
},
})
}
})()
try {
await assert.rejects(SFTPClient.connect(client))
// The refused subsystem must not have taken the connection down.
const result = await client.run("survived")
assert.equal(dec.decode(result.stdout), "ran:survived")
const sftp = await SFTPClient.connect(client)
await sftp.writeFile("/ok.txt", streamOf("ok"))
assert.equal(await readAll(await sftp.readFile("/ok.txt")), "ok")
await sftp.close()
} finally {
await client.close().catch(() => {})
await server.close().catch(() => {})
await loop
}
})
test("a non-sftp subsystem is refused by the router", async () => {
const { client, close } = await routed()
try {
await assert.rejects(client.openSubsystem("rexec"), /channel request failed/)
// The refusal is per-channel; the connection keeps working.
const sftp = await SFTPClient.connect(client)
await sftp.writeFile("/after-refusal.txt", streamOf("fine"))
await sftp.close()
} finally {
await close()
}
})
test("the authenticated context selects the per-user vfs", async () => {
const [a, b] = loopbackTransportPair()
const hostKey = await generateHostKey()
const alice = new MemoryVFS()
const bob = new MemoryVFS()
await alice.writeFile("/whose.txt", streamOf("alice"))
await bob.writeFile("/whose.txt", streamOf("bob"))
const [client, server] = await Promise.all([
SSHClientConnection.connect(a, { user: "bob", password: "p" }),
SSHServerConnection.accept<AsyncVFS>(b, {
hostKey,
authenticate: async (user) => (user === "alice" ? alice : bob),
}),
])
const loop = (async () => {
for (;;) {
let channel: Channel
try {
channel = await server.acceptSession()
} catch {
return
}
void serveSession(channel, {
subsystem: (_name, s) => serveSFTPChannel(s.channel, server.context),
})
}
})()
try {
const sftp = await SFTPClient.connect(client)
assert.equal(await readAll(await sftp.readFile("/whose.txt")), "bob")
await sftp.close()
} finally {
await client.close().catch(() => {})
await server.close().catch(() => {})
await loop
}
})
})
+34
View File
@@ -0,0 +1,34 @@
# @webnet/smb2
SMB2/3 client built on `@webnet/transport` and `@webnet/vfs`. There is no server.
`SMB2Client` implements `AsyncVFS` (`stat`, `readdir`, `statAndReaddir`, `readFile`/`readFileRange`, `writeFile`, `mkdir`, `delete`, `move`, `copy`, `setProps`) over a single share. It dials through a `RawDialer`, negotiates the highest of SMB 2.0.2, 2.1.0, or 3.1.1 that the server supports, and authenticates with NTLM wrapped in SPNEGO via `Credentials` (`username`/`password`/optional `domain`/`spn`). `SMB2Client` also implements `StateTransferable<SMB2TransferState>` from `@webnet/state-transfer`, so an established connection's state (dialect, session, tree, signing key) can be serialized and later resumed with `SMB2Client.adopt` — the transferred state carries no credentials, so `SMB2AdoptOptions` supplies them again for reconnect fallback. Errors surface as `Smb2Error`.
`_internals` entry points are unstable and are not part of the public API.
## Usage
```ts
import { SMB2Client } from "@webnet/smb2"
import type { RawDialer } from "@webnet/transport"
declare const dialer: RawDialer
const client = new SMB2Client({
dialer,
host: "example.com",
share: "shared",
username: "u",
password: "p",
})
await client.connect()
const entries = await client.readdir("/")
const stream = await client.readFile("/file.txt")
await client.disconnect()
```
## See also
- [`@webnet/vfs`](../vfs) — the `AsyncVFS` interface this package implements
- [`@webnet/transport`](../transport) — the `RawDialer` primitive this package dials through
- [`@webnet/state-transfer`](../state-transfer) — the `StateTransferable` interface this package implements for connection handoff
+1 -1
View File
@@ -28,7 +28,7 @@
"typescript": "^6.0.2"
},
"dependencies": {
"@webnet/utils": "*",
"@webnet/binary": "*",
"@webnet/state-transfer": "*",
"@webnet/transport": "*",
"@webnet/vfs": "*"
+51 -9
View File
@@ -1,6 +1,13 @@
import type { RawDialer, RawTransport } from "@webnet/transport"
import type { StateTransferable } from "@webnet/state-transfer"
import { VFSError, type AsyncVFS, type Stat } from "@webnet/vfs"
import {
normalizePath,
pathsOverlap,
VFSError,
type AsyncVFS,
type Stat,
type StatAndReaddirResult,
} from "@webnet/vfs"
import {
Access,
ShareAccess,
@@ -322,6 +329,33 @@ export class SMB2Client implements AsyncVFS, StateTransferable<SMB2TransferState
})
}
async statAndReaddir(path: string): Promise<StatAndReaddirResult> {
const name = smbPath(path)
return this.#guard(path, async (tree) => {
const r = await tree.create({
name,
desiredAccess: Access.FILE_LIST_DIRECTORY | Access.FILE_READ_ATTRIBUTES,
fileAttributes: 0,
shareAccess: SHARE_ALL,
createDisposition: CreateDisposition.OPEN,
createOptions: 0,
})
try {
const self = this.#statFromCreate(path, basename(path), r)
if (!self.isDirectory) return { self, entries: [] }
const entries = await tree.queryDirectory(r.fileId)
return {
self,
entries: entries
.filter((e) => e.name !== "." && e.name !== "..")
.map((e) => this.#statFromDirInfo(path, e)),
}
} finally {
await tree.close(r.fileId)
}
})
}
async readFile(path: string): Promise<ReadableStream<Uint8Array>> {
return this.readFileRange(path, 0n)
}
@@ -449,9 +483,13 @@ export class SMB2Client implements AsyncVFS, StateTransferable<SMB2TransferState
async delete(path: string, recursive?: boolean): Promise<void> {
if (smbPath(path) === "") throw new VFSError("forbidden", "cannot delete root")
const st = await this.stat(path)
await this.#delete(path, recursive ?? false, st)
}
async #delete(path: string, recursive: boolean, st: Stat): Promise<void> {
if (st.isDirectory && recursive) {
const entries = await this.readdir(path)
for (const e of entries) await this.delete(e.path, true)
const { entries } = await this.statAndReaddir(path)
for (const entry of entries) await this.#delete(entry.path, true, entry)
}
const name = smbPath(path)
return this.#guard(path, async (tree) => {
@@ -493,12 +531,16 @@ export class SMB2Client implements AsyncVFS, StateTransferable<SMB2TransferState
})
}
// Kept rather than delegated to copyFallback: this is the implementation of the optional
// operation, not a caller of it, and SMB paths are case-insensitive, so the overlap check has to
// fold case where the shared one compares normalized paths exactly. `\dir\A` copied into
// `\dir\a\sub` is the same non-terminating copy the shared check exists to reject.
async copy(src: string, dest: string, opts?: { overwrite?: boolean }): Promise<void> {
const st = await this.stat(src)
const srcKey = smbPath(src)
const destKey = smbPath(dest)
if (destKey === srcKey || destKey.startsWith(srcKey + "\\"))
throw new VFSError("forbidden", `cannot copy ${src} into itself`)
const { self: st, entries } = await this.statAndReaddir(src)
const from = normalizePath(src)
const to = normalizePath(dest)
if (pathsOverlap(from.toLowerCase(), to.toLowerCase()))
throw new VFSError("precondition-failed", `${from} and ${to} overlap`)
const existing = await this.stat(dest).then(
(s) => s,
(e) => {
@@ -512,7 +554,7 @@ export class SMB2Client implements AsyncVFS, StateTransferable<SMB2TransferState
}
if (st.isDirectory) {
await this.mkdir(dest)
for (const entry of await this.readdir(src)) {
for (const entry of entries) {
await this.copy(entry.path, joinPath(dest, entry.name), opts)
}
return
+2 -1
View File
@@ -1,6 +1,7 @@
import { isStateTransferable } from "@webnet/state-transfer"
import type { RawTransport } from "@webnet/transport"
import { ReadBuffer } from "@webnet/transport/buffer"
import { writeAll } from "@webnet/transport/operation"
import {
Command,
Dialect,
@@ -219,7 +220,7 @@ export class Smb2Connection {
const sig = await this.#signer.sign(message)
message.set(sig.subarray(0, 16), SIGNATURE_OFFSET)
}
await this.#transport.write(encodeTransport(message))
await writeAll(this.#transport, encodeTransport(message))
let recvMessage = await this.#readMessage()
let recvHeader = decodeHeader(recvMessage)
while (recvHeader.status >>> 0 === Status.PENDING) {
+1 -1
View File
@@ -1,4 +1,4 @@
import { BinaryReader, BinaryWriter } from "@webnet/utils"
import { BinaryReader, BinaryWriter } from "@webnet/binary"
export class Writer extends BinaryWriter {
constructor(capacity = 256) {
@@ -85,6 +85,15 @@ suite("smb2 integration", { skip: !configured && "set SMB2_TEST_* env vars to ru
const entries = await client.readdir(`/${dir}`)
assert.ok(entries.some((e) => e.name === "a.bin"))
const combinedDirectory = await client.statAndReaddir(`/${dir}`)
assert.equal(combinedDirectory.self.isDirectory, true)
assert.ok(combinedDirectory.entries.some((e) => e.name === "a.bin"))
const combinedFile = await client.statAndReaddir(`/${dir}/a.bin`)
assert.equal(combinedFile.self.isDirectory, false)
assert.equal(combinedFile.self.size, BigInt(payload.length))
assert.deepEqual(combinedFile.entries, [])
await client.move(`/${dir}/a.bin`, `/${dir}/b.bin`)
await assert.rejects(
client.stat(`/${dir}/a.bin`),
+176 -4
View File
@@ -20,6 +20,7 @@ import {
Command,
Status,
Dialect,
Access,
FileAttribute,
CreateDisposition,
CreateOptions,
@@ -175,6 +176,8 @@ class MockSmb2Server {
#tamperNext = false
#sendUnsignedNext = false
readonly messageIds: bigint[] = []
readonly commands: number[] = []
readonly creates: { path: string; desiredAccess: number }[] = []
negotiates = 0
badSignatures = 0
@@ -223,6 +226,7 @@ class MockSmb2Server {
const header = decodeHeader(msg)
const body = msg.slice(HEADER_SIZE)
this.messageIds.push(header.messageId)
this.commands.push(header.command)
if (conn.signer && (header.flags & HeaderFlags.SIGNED) !== 0) {
const expected = msg.slice(SIGNATURE_OFFSET, SIGNATURE_OFFSET + 16)
const actual = (await conn.signer.sign(zeroSig(msg))).subarray(0, 16)
@@ -516,7 +520,7 @@ class MockSmb2Server {
r.u32() // Impersonation
r.u64() // SmbCreateFlags
r.u64() // Reserved
r.u32() // DesiredAccess
const desiredAccess = r.u32()
r.u32() // FileAttributes
r.u32() // ShareAccess
const createDisposition = r.u32()
@@ -528,6 +532,7 @@ class MockSmb2Server {
const nameOff = nameOffset - HEADER_SIZE
const nameBytes = body.slice(nameOff, nameOff + nameLength)
const path = fromUtf16le(nameBytes)
this.creates.push({ path, desiredAccess })
const { parent } = splitPath(path)
const existing = this.#fs.nodes.get(path)
@@ -976,12 +981,19 @@ class MockSmb2Server {
// -- test harness --
function setup(): {
function setup(wrapClient?: (transport: RawTransport) => RawTransport): {
client: SMB2Client
server: MockSmb2Server
stop: () => Promise<void>
} {
const [listener, dialer] = loopbackListener()
const [listener, rawDialer] = loopbackListener()
const dialer: RawDialer = wrapClient
? {
async dial(host, port) {
return wrapClient(await rawDialer.dial(host, port))
},
}
: rawDialer
const server = new MockSmb2Server()
const acceptLoop = (async () => {
while (!listener.closed) {
@@ -1038,6 +1050,125 @@ testAsyncVFSConformance({
})
suite("smb2 client e2e", () => {
test("requests tolerate short transport writes", async () => {
const { client, stop } = setup(
(transport) =>
new Proxy(transport, {
get(target, property) {
if (property === "write") {
return async (data: Uint8Array) => {
const chunk = data.subarray(0, Math.min(3, data.length))
const written = await target.write(chunk)
return typeof written === "number" ? written : chunk.length
}
}
const value = Reflect.get(target, property, target)
return typeof value === "function" ? value.bind(target) : value
},
}),
)
try {
await client.mkdir("/short-writes")
assert.equal((await client.stat("/short-writes")).isDirectory, true)
} finally {
await stop()
}
})
test("statAndReaddir reuses one handle for directory metadata and entries", async () => {
const { client, server, stop } = setup()
try {
await client.mkdir("/dir")
await client.writeFile("/dir/file.txt", streamOf(new TextEncoder().encode("data")))
const before = server.commands.length
const { self, entries } = await client.statAndReaddir("/dir")
assert.equal(self.path, "/dir")
assert.equal(self.isDirectory, true)
assert.deepEqual(
entries.map((entry) => entry.name),
["file.txt"],
)
const commands = server.commands.slice(before)
assert.equal(commands[0], Command.CREATE)
assert.equal(commands.at(-1), Command.CLOSE)
assert.ok(commands.slice(1, -1).every((command) => command === Command.QUERY_DIRECTORY))
} finally {
await stop()
}
})
test("statAndReaddir returns a file without querying it as a directory", async () => {
const { client, server, stop } = setup()
try {
await client.writeFile("/file.txt", streamOf(new TextEncoder().encode("data")))
const before = server.commands.length
const { self, entries } = await client.statAndReaddir("/file.txt")
assert.equal(self.path, "/file.txt")
assert.equal(self.isDirectory, false)
assert.deepEqual(entries, [])
assert.deepEqual(server.commands.slice(before), [Command.CREATE, Command.CLOSE])
} finally {
await stop()
}
})
test("recursive delete combines directory metadata and enumeration", async () => {
const { client, server, stop } = setup()
try {
await client.mkdir("/tree")
await client.mkdir("/tree/sub")
await client.writeFile("/tree/sub/file.txt", streamOf(new TextEncoder().encode("data")))
const before = server.commands.length
await client.delete("/tree", true)
const commands = server.commands.slice(before)
assert.equal(commands.filter((command) => command === Command.CREATE).length, 6)
assert.equal(commands.filter((command) => command === Command.QUERY_DIRECTORY).length, 4)
assert.deepEqual(server.creates.slice(-6), [
{ path: "tree", desiredAccess: Access.FILE_READ_ATTRIBUTES },
{
path: "tree",
desiredAccess: Access.FILE_LIST_DIRECTORY | Access.FILE_READ_ATTRIBUTES,
},
{
path: "tree\\sub",
desiredAccess: Access.FILE_LIST_DIRECTORY | Access.FILE_READ_ATTRIBUTES,
},
{ path: "tree\\sub\\file.txt", desiredAccess: Access.DELETE },
{ path: "tree\\sub", desiredAccess: Access.DELETE },
{ path: "tree", desiredAccess: Access.DELETE },
])
await assert.rejects(
client.stat("/tree"),
(error: unknown) => error instanceof VFSError && error.code === "not-found",
)
} finally {
await stop()
}
})
test("recursive delete of a file does not request read-data access", async () => {
const { client, server, stop } = setup()
try {
await client.writeFile("/file.txt", streamOf(new TextEncoder().encode("data")))
const before = server.creates.length
await client.delete("/file.txt", true)
assert.deepEqual(server.creates.slice(before), [
{ path: "file.txt", desiredAccess: Access.FILE_READ_ATTRIBUTES },
{ path: "file.txt", desiredAccess: Access.DELETE },
])
} finally {
await stop()
}
})
test("client rejects a response with a tampered signature", async () => {
const { client, server, stop } = setup()
try {
@@ -1059,7 +1190,48 @@ suite("smb2 client e2e", () => {
await client.writeFile("/tree/a.txt", streamOf(new TextEncoder().encode("a")))
await assert.rejects(
() => client.copy("/tree", "/tree/inner"),
(e: unknown) => e instanceof VFSError && e.code === "forbidden",
(e: unknown) => e instanceof VFSError && e.code === "precondition-failed",
)
} finally {
await stop()
}
})
test("copy detects a mixed-case source inside its destination", async () => {
const { client, stop } = setup()
try {
await client.mkdir("/Tree")
await client.mkdir("/Tree/Inner")
await client.writeFile("/Tree/Inner/a.txt", streamOf(new TextEncoder().encode("a")))
await assert.rejects(
() => client.copy("/Tree/Inner", "/tree", { overwrite: true }),
(e: unknown) => e instanceof VFSError && e.code === "precondition-failed",
)
assert.equal(
new TextDecoder().decode(await readAll(await client.readFile("/Tree/Inner/a.txt"))),
"a",
)
} finally {
await stop()
}
})
test("recursive copy combines source directory metadata and enumeration", async () => {
const { client, server, stop } = setup()
try {
await client.mkdir("/source")
await client.mkdir("/source/sub")
await client.writeFile("/source/sub/file.txt", streamOf(new TextEncoder().encode("payload")))
const before = server.commands.length
await client.copy("/source", "/destination")
const commands = server.commands.slice(before)
assert.equal(commands.filter((command) => command === Command.CREATE).length, 10)
assert.equal(commands.filter((command) => command === Command.QUERY_DIRECTORY).length, 4)
assert.equal(
new TextDecoder().decode(await readAll(await client.readFile("/destination/sub/file.txt"))),
"payload",
)
} finally {
await stop()
+1
View File
@@ -18,6 +18,7 @@ const MAP: Record<number, VFSErrorCode> = {
[Status.FILE_LOCK_CONFLICT]: "locked",
[Status.LOCK_NOT_GRANTED]: "locked",
[Status.DELETE_PENDING]: "locked",
[Status.NOT_SUPPORTED]: "unsupported",
}
export function statusToVFSError(status: number, path: string): VFSError {

Some files were not shown because too many files have changed in this diff Show More