feat: add export/import streaming to the pluggable backends
CI / format (pull_request) Successful in 28s
CI / lint (pull_request) Successful in 28s
CI / install-and-build (pull_request) Successful in 53s
CI / typecheck-tests (pull_request) Successful in 28s
CI / typecheck-source (pull_request) Successful in 32s
CI / test (pull_request) Successful in 43s
CI / format (pull_request) Successful in 28s
CI / lint (pull_request) Successful in 28s
CI / install-and-build (pull_request) Successful in 53s
CI / typecheck-tests (pull_request) Successful in 28s
CI / typecheck-source (pull_request) Successful in 32s
CI / test (pull_request) Successful in 43s
Add backend-agnostic data export/import over an NDJSON wire format, plus an
inspect utility, exposed via three new CLIs and an HTTP export endpoint.
- ExportImport types + filter helpers (kind/record scoping, hard-intersection
of filters) in src/db/{types/ExportImport,export/filter}.ts
- SqliteInterface implements Exportable + Importable: signal-checked async
generator export (one query per table, per-record yield, trailing error
sentinel on mid-stream failure) and a manually-driven import transaction
that rolls back on any error/abort and never commits partial data
- ApiInterface implements Exportable via its own fetch({signal})
- computeForcedExportFilter enforces non-global-admin scope (resided-in abodes
+ co-resident users, intersected with apikey restrict_*); GET /export
intersects it with the caller's filter and wires an AbortController to the
response socket
- inspectExportStream reports kinds/counts from any stream without a db
- abode-export / abode-import / abode-inspect CLIs (import is sqlite-only)
- Secrets are not exported: imported users default to '#unset' passwords and
apikeys are re-minted a token (ClientApikey view round-trips exactly)
- test/tools/export-import.test.ts: round-trip, filter narrowing, forced-scope,
export/import cancellation, in-process Koa endpoint, inspect
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
import type { Context } from "koa";
|
||||
import type { BackendDbInterface } from "../db/types/DbInterface.js";
|
||||
import type { ClientUser } from "../db/types/User.js";
|
||||
import type { ExportFilter } from "../db/types/ExportImport.js";
|
||||
|
||||
/**
|
||||
* Compute the export scope that must be *forced* on a caller, independent of
|
||||
* anything they requested. Returns `null` when the caller is unrestricted (a
|
||||
* global admin whose credential imposes no narrowing) — their own filter, if
|
||||
* any, is then honored verbatim as a voluntary narrowing.
|
||||
*
|
||||
* Otherwise returns `{ abodes, users }`: the abodes the caller resides in, and
|
||||
* the users needed to keep that data referentially whole (the caller plus every
|
||||
* co-resident of those abodes). This is the maximum a non-admin may export; the
|
||||
* route intersects it with any caller-supplied filter (never a union).
|
||||
*/
|
||||
export async function computeForcedExportFilter(
|
||||
db: BackendDbInterface,
|
||||
ctx: { user: ClientUser; session: NonNullable<Context["session"]> },
|
||||
): Promise<ExportFilter | null> {
|
||||
const { user, session } = ctx;
|
||||
|
||||
if (user.flags.admin) {
|
||||
if (session.source !== "apikey") return null;
|
||||
const p = session.key.permissions;
|
||||
const unrestricted =
|
||||
!!p.admin &&
|
||||
!!p.all &&
|
||||
!p.restrict_users?.length &&
|
||||
!p.restrict_abodes?.length;
|
||||
if (unrestricted) return null;
|
||||
}
|
||||
|
||||
const residencies = await db.listResidentsByUserId(user.uid);
|
||||
const abodeSet = new Set(residencies.map((r) => r.aid));
|
||||
const userSet = new Set<string>([user.uid]);
|
||||
for (const aid of abodeSet) {
|
||||
for (const u of await db.listUsersByAbodeId(aid)) userSet.add(u.uid);
|
||||
}
|
||||
|
||||
let abodes = [...abodeSet];
|
||||
let users = [...userSet];
|
||||
|
||||
// An apikey can only narrow what its owning user could otherwise export.
|
||||
if (session.source === "apikey") {
|
||||
const p = session.key.permissions;
|
||||
if (p.restrict_abodes?.length) {
|
||||
const allow = new Set(p.restrict_abodes);
|
||||
abodes = abodes.filter((a) => allow.has(a));
|
||||
}
|
||||
if (p.restrict_users?.length) {
|
||||
const allow = new Set(p.restrict_users);
|
||||
users = users.filter((u) => allow.has(u));
|
||||
}
|
||||
}
|
||||
|
||||
return { abodes, users };
|
||||
}
|
||||
Reference in New Issue
Block a user