feat: add export/import streaming to the pluggable backends
CI / format (pull_request) Successful in 28s
CI / lint (pull_request) Successful in 28s
CI / install-and-build (pull_request) Successful in 53s
CI / typecheck-tests (pull_request) Successful in 28s
CI / typecheck-source (pull_request) Successful in 32s
CI / test (pull_request) Successful in 43s

Add backend-agnostic data export/import over an NDJSON wire format, plus an
inspect utility, exposed via three new CLIs and an HTTP export endpoint.

- ExportImport types + filter helpers (kind/record scoping, hard-intersection
  of filters) in src/db/{types/ExportImport,export/filter}.ts
- SqliteInterface implements Exportable + Importable: signal-checked async
  generator export (one query per table, per-record yield, trailing error
  sentinel on mid-stream failure) and a manually-driven import transaction
  that rolls back on any error/abort and never commits partial data
- ApiInterface implements Exportable via its own fetch({signal})
- computeForcedExportFilter enforces non-global-admin scope (resided-in abodes
  + co-resident users, intersected with apikey restrict_*); GET /export
  intersects it with the caller's filter and wires an AbortController to the
  response socket
- inspectExportStream reports kinds/counts from any stream without a db
- abode-export / abode-import / abode-inspect CLIs (import is sqlite-only)
- Secrets are not exported: imported users default to '#unset' passwords and
  apikeys are re-minted a token (ClientApikey view round-trips exactly)
- test/tools/export-import.test.ts: round-trip, filter narrowing, forced-scope,
  export/import cancellation, in-process Koa endpoint, inspect

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 22:29:03 +00:00
co-authored by Claude
parent 31d4636dde
commit ccb970f200
14 changed files with 1586 additions and 5 deletions
+58
View File
@@ -0,0 +1,58 @@
import type { Context } from "koa";
import type { BackendDbInterface } from "../db/types/DbInterface.js";
import type { ClientUser } from "../db/types/User.js";
import type { ExportFilter } from "../db/types/ExportImport.js";
/**
* Compute the export scope that must be *forced* on a caller, independent of
* anything they requested. Returns `null` when the caller is unrestricted (a
* global admin whose credential imposes no narrowing) — their own filter, if
* any, is then honored verbatim as a voluntary narrowing.
*
* Otherwise returns `{ abodes, users }`: the abodes the caller resides in, and
* the users needed to keep that data referentially whole (the caller plus every
* co-resident of those abodes). This is the maximum a non-admin may export; the
* route intersects it with any caller-supplied filter (never a union).
*/
export async function computeForcedExportFilter(
db: BackendDbInterface,
ctx: { user: ClientUser; session: NonNullable<Context["session"]> },
): Promise<ExportFilter | null> {
const { user, session } = ctx;
if (user.flags.admin) {
if (session.source !== "apikey") return null;
const p = session.key.permissions;
const unrestricted =
!!p.admin &&
!!p.all &&
!p.restrict_users?.length &&
!p.restrict_abodes?.length;
if (unrestricted) return null;
}
const residencies = await db.listResidentsByUserId(user.uid);
const abodeSet = new Set(residencies.map((r) => r.aid));
const userSet = new Set<string>([user.uid]);
for (const aid of abodeSet) {
for (const u of await db.listUsersByAbodeId(aid)) userSet.add(u.uid);
}
let abodes = [...abodeSet];
let users = [...userSet];
// An apikey can only narrow what its owning user could otherwise export.
if (session.source === "apikey") {
const p = session.key.permissions;
if (p.restrict_abodes?.length) {
const allow = new Set(p.restrict_abodes);
abodes = abodes.filter((a) => allow.has(a));
}
if (p.restrict_users?.length) {
const allow = new Set(p.restrict_users);
users = users.filter((u) => allow.has(u));
}
}
return { abodes, users };
}