Compare commits

...
8 Commits
Author SHA1 Message Date
codingetandCodex 0355bd0b2e fix: restrict user email visibility
Co-Authored-By: gpt-5.6-sol <noreply@openai.com>
2026-07-23 23:18:59 +02:00
codingetandCodex 1e10391e20 style: format postgres notes implementation
CI / lint (pull_request) Successful in 26s
CI / format (pull_request) Successful in 27s
CI / install-and-build (pull_request) Successful in 50s
CI / typecheck-tests (pull_request) Successful in 33s
CI / typecheck-source (pull_request) Successful in 35s
CI / test (pull_request) Successful in 44s
Co-Authored-By: gpt-5.6-luna <noreply@openai.com>
2026-07-23 20:55:45 +00:00
codingetandCodex a0a436c4b8 feat(postgres): implement notes CRUD and export import
CI / lint (pull_request) Successful in 32s
CI / format (pull_request) Failing after 32s
CI / install-and-build (pull_request) Successful in 1m0s
CI / typecheck-tests (pull_request) Successful in 36s
CI / typecheck-source (pull_request) Successful in 37s
CI / test (pull_request) Successful in 41s
Co-Authored-By: gpt-5.6-luna <noreply@openai.com>
2026-07-23 18:49:23 +00:00
codingetandClaude d7e31dfce9 docs: codify the FK-safe stream ordering as a wire-format invariant
CI / lint (pull_request) Successful in 31s
CI / format (pull_request) Successful in 31s
CI / install-and-build (pull_request) Successful in 55s
CI / typecheck-tests (pull_request) Successful in 31s
CI / typecheck-source (pull_request) Successful in 31s
CI / test (pull_request) Successful in 42s
The postgres importer inserts records sequentially with FK enforcement live,
so it depends on records arriving in dependency order. That requirement was
implicit in each backend's export table list; make it explicit and enforced.

- Add EXPORT_KIND_ORDER (user, abode, resident, apikey, note) as a documented
  single source of truth, with the FK dependency chain spelled out on its doc
  comment, and note the ordering guarantee on the ExportEnvelope wire-format
  doc. Derive the isExportKind set from it.
- Both backends' export() now iterate EXPORT_KIND_ORDER via a loader map
  (postgres omits note by leaving it out of the map), so emission order is
  tied to the constant and can't drift.
- Tests: a change-detector on EXPORT_KIND_ORDER, plus assertions that both the
  sqlite and postgres (fake) exports emit record kinds grouped in FK-safe
  order (kind rank non-decreasing down the stream, after the leading meta).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 00:14:15 +00:00
codingetandClaude 3a56dcd9e5 feat: add export/import to the postgres backend; import auto-detects backend
CI / format (pull_request) Successful in 23s
CI / lint (pull_request) Successful in 23s
CI / install-and-build (pull_request) Successful in 45s
CI / typecheck-source (pull_request) Successful in 25s
CI / typecheck-tests (pull_request) Successful in 30s
CI / test (pull_request) Successful in 40s
The export/import plan predated the postgres backend. Bring it up to parity:

- PostgresInterface implements Exportable + Importable, mirroring the sqlite
  backend. Export is a signal-checked async generator (one query per table);
  import drives a transaction via `WrappedPool.multi`, which rolls back on any
  error/abort and commits only after the whole stream is consumed cleanly.
- The `note` kind is skipped on postgres (its note CRUD is still unimplemented,
  so a pg database holds none) — a full dump from sqlite imports its
  user/abode/resident/apikey records and drops notes.
- Unlike sqlite (PRAGMA foreign_keys=off), postgres keeps FK enforcement; the
  FK-safe insertion order keeps a full dump valid, and truly-dangling partial
  dumps will (correctly) fail.
- abode-import now resolves the backend via getDbInterface instead of
  constructing SqliteInterface directly; isImportable keeps it from ever
  running over the remote (api) interface, which has no import.
- Add isImportable(); make postgres selectClientApikeys' where optional.
- Tests: exercise the real PostgresInterface export/import paths against an
  in-memory fake WrappedPgClient (no pg service in CI) — NDJSON shape,
  meta.source, filtering, note-skip, insert dispatch, and error/abort rollback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 23:36:21 +00:00
codingetandClaude aadc950e24 fix: scope apikey export to the caller for non-admins
CI / format (pull_request) Successful in 23s
CI / lint (pull_request) Successful in 23s
CI / install-and-build (pull_request) Successful in 46s
CI / typecheck-tests (pull_request) Successful in 30s
CI / typecheck-source (pull_request) Successful in 31s
CI / test (pull_request) Successful in 40s
A non-admin's forced export filter includes co-resident *user* records so
abode/resident data isn't left with dangling references, but the same `users`
allowlist was also governing `apikey` records — leaking co-residents' apikey
metadata (name/permissions/expiry, though not the secret token).

Add a dedicated `apikeys` uid-allowlist to ExportFilter that scopes apikey
records specifically, falling back to `users` when absent (so existing
unfiltered/voluntary-narrowing behaviour and the round-trip are unchanged).
computeForcedExportFilter now sets it to the caller alone (intersected with an
apikey credential's restrict_users), so a non-admin can only ever export their
own keys. Global admins (forced filter null) are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 23:16:31 +00:00
codingetandClaude ccb970f200 feat: add export/import streaming to the pluggable backends
CI / format (pull_request) Successful in 28s
CI / lint (pull_request) Successful in 28s
CI / install-and-build (pull_request) Successful in 53s
CI / typecheck-tests (pull_request) Successful in 28s
CI / typecheck-source (pull_request) Successful in 32s
CI / test (pull_request) Successful in 43s
Add backend-agnostic data export/import over an NDJSON wire format, plus an
inspect utility, exposed via three new CLIs and an HTTP export endpoint.

- ExportImport types + filter helpers (kind/record scoping, hard-intersection
  of filters) in src/db/{types/ExportImport,export/filter}.ts
- SqliteInterface implements Exportable + Importable: signal-checked async
  generator export (one query per table, per-record yield, trailing error
  sentinel on mid-stream failure) and a manually-driven import transaction
  that rolls back on any error/abort and never commits partial data
- ApiInterface implements Exportable via its own fetch({signal})
- computeForcedExportFilter enforces non-global-admin scope (resided-in abodes
  + co-resident users, intersected with apikey restrict_*); GET /export
  intersects it with the caller's filter and wires an AbortController to the
  response socket
- inspectExportStream reports kinds/counts from any stream without a db
- abode-export / abode-import / abode-inspect CLIs (import is sqlite-only)
- Secrets are not exported: imported users default to '#unset' passwords and
  apikeys are re-minted a token (ClientApikey view round-trips exactly)
- test/tools/export-import.test.ts: round-trip, filter narrowing, forced-scope,
  export/import cancellation, in-process Koa endpoint, inspect

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 22:29:03 +00:00
codingetandCodex 31d4636dde ci: add pull request quality gates
CI / install-and-build (pull_request) Successful in 1m29s
CI / format (pull_request) Successful in 51s
CI / typecheck-source (pull_request) Successful in 41s
CI / typecheck-tests (pull_request) Successful in 39s
CI / test (pull_request) Successful in 51s
CI / lint (pull_request) Successful in 21s
Co-Authored-By: gpt-5.6-terra <noreply@openai.com>
2026-07-22 21:50:29 +00:00
93 changed files with 4691 additions and 420 deletions
+9
View File
@@ -0,0 +1,9 @@
name: Setup
description: Set up the Node version required by this repository. Requires actions/checkout to have already run.
runs:
using: composite
steps:
- uses: actions/setup-node@v4
with:
node-version-file: .node-version
+124
View File
@@ -0,0 +1,124 @@
name: CI
on:
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
install-and-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- name: Clean install dependencies
run: npm ci
- name: Production build
run: npm run build
- name: Archive dependencies
run: tar -czf node_modules.tar.gz node_modules
- uses: actions/upload-artifact@v3
with:
name: node_modules-${{ github.run_id }}
path: node_modules.tar.gz
- name: Archive build artifacts
run: tar -czf build-artifacts.tar.gz dist
- uses: actions/upload-artifact@v3
with:
name: build-artifacts-${{ github.run_id }}
path: build-artifacts.tar.gz
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- name: Clean install dependencies
run: npm ci
- name: Lint
run: npm run lint
format:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- name: Clean install dependencies
run: npm ci
- name: Check formatting
run: npm run format:check
typecheck-source:
runs-on: ubuntu-latest
needs: install-and-build
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- uses: actions/download-artifact@v3
with:
name: node_modules-${{ github.run_id }}
- uses: actions/download-artifact@v3
with:
name: build-artifacts-${{ github.run_id }}
- name: Restore dependencies and build artifacts
run: |
tar -xzf node_modules.tar.gz
tar -xzf build-artifacts.tar.gz
- name: Typecheck source
run: npm run typecheck
typecheck-tests:
runs-on: ubuntu-latest
needs: install-and-build
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- uses: actions/download-artifact@v3
with:
name: node_modules-${{ github.run_id }}
- name: Restore dependencies
run: tar -xzf node_modules.tar.gz
- name: Typecheck tests
run: npm run typecheck:test
test:
runs-on: ubuntu-latest
needs: install-and-build
steps:
- uses: actions/checkout@v4
- uses: ./.gitea/actions/setup
- uses: actions/download-artifact@v3
with:
name: node_modules-${{ github.run_id }}
- name: Restore dependencies
run: tar -xzf node_modules.tar.gz
- name: Run full test suite
run: npm test
+1
View File
@@ -0,0 +1 @@
22
+2
View File
@@ -0,0 +1,2 @@
dist
node_modules
+22
View File
@@ -0,0 +1,22 @@
import eslint from "@eslint/js";
import tseslint from "typescript-eslint";
export default tseslint.config(
{
ignores: ["dist/**", "node_modules/**"],
},
eslint.configs.recommended,
tseslint.configs.recommended,
{
rules: {
"no-control-regex": "off",
"no-empty": "off",
"no-fallthrough": "off",
"prefer-const": "off",
"@typescript-eslint/no-empty-object-type": "off",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-non-null-asserted-optional-chain": "off",
"@typescript-eslint/no-unused-vars": "off",
},
},
);
+1306 -20
View File
File diff suppressed because it is too large Load Diff
+15 -3
View File
@@ -9,7 +9,10 @@
"abode-migrate": "dist/bin/abode-migrate.cjs",
"abode-repl": "dist/bin/abode-repl.cjs",
"abode-web": "dist/bin/abode-web.cjs",
"abode-tui": "dist/bin/abode-tui.cjs"
"abode-tui": "dist/bin/abode-tui.cjs",
"abode-export": "dist/bin/abode-export.cjs",
"abode-import": "dist/bin/abode-import.cjs",
"abode-inspect": "dist/bin/abode-inspect.cjs"
},
"scripts": {
"repl": "tsx --import ./src/meta/dev/register.ts",
@@ -18,6 +21,9 @@
"abode-web": "tsx --import ./src/meta/dev/register.ts --import ./src/meta/dev/webhot.ts src/bin/abode-web.ts",
"abode-tui": "tsx --import ./src/meta/dev/register.ts --import ./src/meta/dev/silenthot.ts src/bin/abode-tui.ts",
"abode-sources": "tsx --import ./src/meta/dev/register.ts src/bin/abode-sources.ts",
"abode-export": "tsx --import ./src/meta/dev/register.ts src/bin/abode-export.ts",
"abode-import": "tsx --import ./src/meta/dev/register.ts src/bin/abode-import.ts",
"abode-inspect": "tsx --import ./src/meta/dev/register.ts src/bin/abode-inspect.ts",
"build": "NODE_ENV=production npm run build:impl",
"build:impl": "rm -rf dist && tsx node_modules/.bin/webpack && chmod +x dist/bin/* && chmod -x dist/bin/*.*",
"test": "node --import tsx/esm --import ./src/meta/dev/register.ts --test $(find test -name '*.test.ts' | sort)",
@@ -25,7 +31,9 @@
"test:shared": "node --import tsx/esm --import ./src/meta/dev/register.ts --test $(find test/backends -name 'index.test.ts' | sort)",
"test:tools": "node --import tsx/esm --import ./src/meta/dev/register.ts --test $(find test/tools -name '*.test.ts' | sort)",
"typecheck": "tsc --noEmit",
"typecheck:test": "tsc --noEmit -p tsconfig.test.json"
"typecheck:test": "tsc --noEmit -p tsconfig.test.json",
"lint": "eslint .",
"format:check": "prettier --check ."
},
"dependencies": {
"@koa/bodyparser": "^6.0.0",
@@ -44,21 +52,25 @@
"react-redux": "^9.2.0"
},
"devDependencies": {
"@eslint/js": "^9.39.5",
"@types/better-sqlite3": "^7.6.13",
"@types/pg": "^8.20.0",
"@types/koa": "^3.0.0",
"@types/koa__router": "^12.0.4",
"@types/pg": "^8.20.0",
"@types/react": "^19.1.12",
"@types/webpack-bundle-analyzer": "^4.7.0",
"copy-webpack-plugin": "^13.0.1",
"css-loader": "^7.1.2",
"dynohot": "^2.1.1",
"eslint": "^9.39.5",
"mini-css-extract-plugin": "^2.9.4",
"prettier": "^3.6.2",
"raw-loader": "^4.0.2",
"scss-loader": "^0.0.1",
"ts-loader": "^9.5.4",
"tsx": "^4.20.5",
"typescript": "^5.9.2",
"typescript-eslint": "^8.65.0",
"val-loader": "^6.0.0",
"webpack-bundle-analyzer": "^4.10.2",
"webpack-cli": "^6.0.1"
+97
View File
@@ -0,0 +1,97 @@
import { createWriteStream } from "node:fs";
import { pipeline } from "node:stream/promises";
import { getDbInterface } from "../db/index.js";
import { isExportable, type ExportFilter } from "../db/types/ExportImport.js";
import { isExportKind } from "../db/export/filter.js";
const args = process.argv.slice(2);
function printUsage(err: boolean | string = false): never {
const log = (err ? console.error : console.log).bind(console);
if (typeof err === "string") {
log(`Error: ${err}`);
log("");
}
log("Usage:");
log("\tabode-export --help");
log(
"\tabode-export <database-url> [--kinds=user,abode,...] [--exclude-kinds=...] \\",
);
log("\t [--abodes=aid,...] [--users=uid,...] [--out=file|-]");
process.exit(err ? 1 : 0);
}
if (["-h", "--help", "help"].some((x) => args.includes(x))) printUsage();
let url: string | undefined;
const flags = new Map<string, string>();
for (const arg of args) {
if (arg.startsWith("--")) {
const eq = arg.indexOf("=");
if (eq === -1) printUsage(`invalid flag: ${arg}`);
flags.set(arg.slice(2, eq), arg.slice(eq + 1));
} else if (url === undefined) {
url = arg;
} else {
printUsage("too many arguments");
}
}
if (!url) printUsage("missing <database-url>");
const knownFlags = ["kinds", "exclude-kinds", "abodes", "users", "out"];
for (const key of flags.keys()) {
if (!knownFlags.includes(key)) printUsage(`unknown flag: --${key}`);
}
function parseList(value: string | undefined): string[] | undefined {
if (value === undefined) return undefined;
return value.split(",").filter(Boolean);
}
function parseKinds(value: string | undefined) {
const list = parseList(value);
if (!list) return undefined;
const bad = list.filter((k) => !isExportKind(k));
if (bad.length) printUsage(`invalid kind(s): ${bad.join(", ")}`);
return list.filter(isExportKind);
}
const filter: ExportFilter = {};
const kinds = parseKinds(flags.get("kinds"));
if (kinds) filter.kinds = kinds;
const excludeKinds = parseKinds(flags.get("exclude-kinds"));
if (excludeKinds) filter.excludeKinds = excludeKinds;
const abodes = parseList(flags.get("abodes"));
if (abodes) filter.abodes = abodes;
const users = parseList(flags.get("users"));
if (users) filter.users = users;
const db = await getDbInterface(url);
if (!isExportable(db)) {
console.error(`Error: backend '${db.name}' does not support export`);
process.exit(1);
}
const out = flags.get("out") ?? "-";
const dest =
out === "-" ? process.stdout : createWriteStream(out, { encoding: "utf8" });
const ac = new AbortController();
const onSignal = () => ac.abort();
process.on("SIGINT", onSignal);
process.on("SIGTERM", onSignal);
try {
await pipeline(db.export({ filter, signal: ac.signal }), dest);
} catch (e) {
if (ac.signal.aborted) {
console.error("Export aborted");
process.exit(130);
}
throw e;
} finally {
process.off("SIGINT", onSignal);
process.off("SIGTERM", onSignal);
await db.close().catch(() => {});
}
process.exit(0);
+111
View File
@@ -0,0 +1,111 @@
import { createReadStream } from "node:fs";
import { getDbInterface } from "../db/index.js";
import { isExportKind } from "../db/export/filter.js";
import { isImportable, type ExportFilter } from "../db/types/ExportImport.js";
const args = process.argv.slice(2);
function printUsage(err: boolean | string = false): never {
const log = (err ? console.error : console.log).bind(console);
if (typeof err === "string") {
log(`Error: ${err}`);
log("");
}
log("Usage:");
log("\tabode-import --help");
log("\tabode-import <database-url> <input-file|-> [--kinds=...] \\");
log(
"\t [--exclude-kinds=...] [--abodes=aid,...] [--users=uid,...]",
);
log("");
log(
"The target must be a local backend (sqlite or postgres), already migrated",
);
log("(run abode-migrate first). Remote (api) targets are not importable.");
process.exit(err ? 1 : 0);
}
if (["-h", "--help", "help"].some((x) => args.includes(x))) printUsage();
const positional: string[] = [];
const flags = new Map<string, string>();
for (const arg of args) {
if (arg.startsWith("--")) {
const eq = arg.indexOf("=");
if (eq === -1) printUsage(`invalid flag: ${arg}`);
flags.set(arg.slice(2, eq), arg.slice(eq + 1));
} else {
positional.push(arg);
}
}
const url = positional[0];
const input = positional[1];
if (!url) printUsage("missing <database-url>");
if (!input) printUsage("missing <input-file|->");
if (positional.length > 2) printUsage("too many arguments");
const knownFlags = ["kinds", "exclude-kinds", "abodes", "users"];
for (const key of flags.keys()) {
if (!knownFlags.includes(key)) printUsage(`unknown flag: --${key}`);
}
function parseList(value: string | undefined): string[] | undefined {
if (value === undefined) return undefined;
return value.split(",").filter(Boolean);
}
function parseKinds(value: string | undefined) {
const list = parseList(value);
if (!list) return undefined;
const bad = list.filter((k) => !isExportKind(k));
if (bad.length) printUsage(`invalid kind(s): ${bad.join(", ")}`);
return list.filter(isExportKind);
}
const filter: ExportFilter = {};
const kinds = parseKinds(flags.get("kinds"));
if (kinds) filter.kinds = kinds;
const excludeKinds = parseKinds(flags.get("exclude-kinds"));
if (excludeKinds) filter.excludeKinds = excludeKinds;
const abodes = parseList(flags.get("abodes"));
if (abodes) filter.abodes = abodes;
const users = parseList(flags.get("users"));
if (users) filter.users = users;
// Resolve the backend generically. Import lives on the local backends (sqlite,
// postgres); the remote (api) interface has no `import`, so `isImportable`
// keeps it from ever running over HTTP.
const db = await getDbInterface(url);
if (!isImportable(db)) {
console.error(`Error: backend '${db.name}' does not support import`);
await db.close().catch(() => {});
process.exit(1);
}
const source = input === "-" ? process.stdin : createReadStream(input);
const ac = new AbortController();
const onSignal = () => ac.abort();
process.on("SIGINT", onSignal);
process.on("SIGTERM", onSignal);
try {
const result = await db.import(source, { filter, signal: ac.signal });
const total = Object.values(result.counts).reduce((a, b) => a + b, 0);
console.log(`Imported ${total} record(s):`);
for (const [kind, count] of Object.entries(result.counts)) {
console.log(`- ${kind}: ${count}`);
}
} catch (e) {
if (ac.signal.aborted) {
console.error("Import aborted; no changes committed");
process.exit(130);
}
throw e;
} finally {
process.off("SIGINT", onSignal);
process.off("SIGTERM", onSignal);
await db.close().catch(() => {});
}
process.exit(0);
+68
View File
@@ -0,0 +1,68 @@
import { createReadStream } from "node:fs";
import { inspectExportStream } from "../db/export/inspect.js";
import { isExportKind } from "../db/export/filter.js";
import type { ExportKind } from "../db/types/ExportImport.js";
const args = process.argv.slice(2);
function printUsage(err: boolean | string = false): never {
const log = (err ? console.error : console.log).bind(console);
if (typeof err === "string") {
log(`Error: ${err}`);
log("");
}
log("Usage:");
log("\tabode-inspect --help");
log("\tabode-inspect <input-file|-> [--stop-after=note,...]");
process.exit(err ? 1 : 0);
}
if (["-h", "--help", "help"].some((x) => args.includes(x))) printUsage();
let input: string | undefined;
const flags = new Map<string, string>();
for (const arg of args) {
if (arg.startsWith("--")) {
const eq = arg.indexOf("=");
if (eq === -1) printUsage(`invalid flag: ${arg}`);
flags.set(arg.slice(2, eq), arg.slice(eq + 1));
} else if (input === undefined) {
input = arg;
} else {
printUsage("too many arguments");
}
}
if (!input) printUsage("missing <input-file|->");
for (const key of flags.keys()) {
if (key !== "stop-after") printUsage(`unknown flag: --${key}`);
}
let stopAfterKinds: ExportKind[] | undefined;
const rawStop = flags.get("stop-after");
if (rawStop !== undefined) {
const list = rawStop.split(",").filter(Boolean);
const bad = list.filter((k) => !isExportKind(k));
if (bad.length) printUsage(`invalid kind(s): ${bad.join(", ")}`);
stopAfterKinds = list.filter(isExportKind);
}
const source = input === "-" ? process.stdin : createReadStream(input);
const { counts, meta } = await inspectExportStream(source, { stopAfterKinds });
if (meta) {
console.log("Meta:");
console.log(`- format version: ${meta.v ?? "?"}`);
console.log(`- exported at: ${meta.exportedAt ?? "?"}`);
console.log(`- source backend: ${meta.source ?? "?"}`);
if (meta.filter && Object.keys(meta.filter).length) {
console.log(`- effective filter: ${JSON.stringify(meta.filter)}`);
}
}
console.log("Counts:");
const entries = Object.entries(counts);
if (!entries.length) console.log("(none)");
for (const [kind, count] of entries) console.log(`- ${kind}: ${count}`);
process.exit(0);
+3 -3
View File
@@ -40,7 +40,7 @@ switch (cmd) {
if (!current.length) console.log("(none)");
for (const migration of current) {
console.log(
`- ${migration.id} (${migration.name}) applied at ${migration.applied_at}`
`- ${migration.id} (${migration.name}) applied at ${migration.applied_at}`,
);
}
break;
@@ -73,7 +73,7 @@ switch (cmd) {
password: await hashPassword("changeme"),
});
console.log(
`Created user 'admin@codi.moe' (${uid}) with password 'changeme' and admin flag`
`Created user 'admin@codi.moe' (${uid}) with password 'changeme' and admin flag`,
);
}
users = await db.listUsers();
@@ -88,7 +88,7 @@ switch (cmd) {
expires_at: null,
});
console.log(
`Created apikey '${token}' (${apikey.kid}) with permissions admin, all and no expiry`
`Created apikey '${token}' (${apikey.kid}) with permissions admin, all and no expiry`,
);
}
}
+4 -4
View File
@@ -21,7 +21,7 @@ if (args.length > 1) printUsage("too many arguments");
console.log(
`Compiled with ${compiledSources.length} sources:`,
compiledSources.join(", ")
compiledSources.join(", "),
);
const url = args[0] ?? "abode://";
@@ -31,7 +31,7 @@ for (const source of sources) {
console.log(`- ${source.name}`);
console.log(
" - protocols:",
source.protocols.map((x) => `'${x}'`).join(" ")
source.protocols.map((x) => `'${x}'`).join(" "),
);
const match = source.checkUrl(url);
console.log(` - matches url: ${match}`);
@@ -41,7 +41,7 @@ for (const source of sources) {
console.log(
` - generates an interface named ${db.name} ${
db.backend ? "with" : "without"
} backend`
} backend`,
);
await db.close().catch(console.error);
} catch (e) {
@@ -53,7 +53,7 @@ for (const source of sources) {
console.log(
` - generates a migrator knowing ${
db.listAvailableMigrations().length
} migrations`
} migrations`,
);
} catch (e) {
console.error(e);
+2 -2
View File
@@ -30,7 +30,7 @@ const bgColor = await new Promise<string>((ok, ko) => {
process.stdin.once("data", (chunk) => {
const result = chunk.toString("utf8");
const match = result.match(
/^\u001b]11;rgb:([0-9a-fA-F]+)\/([0-9a-fA-F]+)\/([0-9a-fA-F]+)$/
/^\u001b]11;rgb:([0-9a-fA-F]+)\/([0-9a-fA-F]+)\/([0-9a-fA-F]+)$/,
);
if (!match) return ko("Didn't recognize terminal bg color");
const [r, g, b] = match
@@ -78,7 +78,7 @@ if (import.meta.hot) {
import.meta.hot.accept("../tui/App.js", (mod) => {
fullscreenApp.instance.rerender(
(mod.app as typeof app)({ db, bgColor, store })
(mod.app as typeof app)({ db, bgColor, store }),
);
});
}
+48 -4
View File
@@ -25,8 +25,11 @@ import type {
CreateUser,
UpdateUser,
} from "../types/User.js";
import { Readable } from "node:stream";
import type { ReadableStream as WebReadableStream } from "node:stream/web";
import type { Exportable, ExportOptions } from "../types/ExportImport.js";
export class ApiInterface implements DbInterface {
export class ApiInterface implements DbInterface, Exportable {
#root: string;
#headers: Record<string, string>;
#readonly: boolean;
@@ -39,7 +42,7 @@ export class ApiInterface implements DbInterface {
}: {
headers?: Record<string, string>;
readonly?: boolean;
} = {}
} = {},
) {
if (root.endsWith("/")) root = root.slice(0, -1);
this.#root = root;
@@ -82,7 +85,7 @@ export class ApiInterface implements DbInterface {
params?: Record<string, string>;
body?: unknown;
headers?: Record<string, string>;
} = {}
} = {},
): Promise<T> {
const resolvedHeaders = { ...this.#headers, ...headers };
if (body !== undefined) {
@@ -247,7 +250,7 @@ export class ApiInterface implements DbInterface {
});
}
async createApikey(
apikey: CreateApikey
apikey: CreateApikey,
): Promise<[ClientApikey, `at_${string}`]> {
this.#checkReadonly();
const { apikey: key, token } = await this.#call<{
@@ -296,4 +299,45 @@ export class ApiInterface implements DbInterface {
async listNotesByUserId(uid: string): Promise<PartialNote[]> {
return this.#call("GET", "/users/:uid/notes", { params: { uid } });
}
export(options: ExportOptions = {}): NodeJS.ReadableStream {
// Streaming NDJSON bypasses the JSON-only `#call` helper: the raw
// `fetch` gets the caller's `signal` directly, so aborting cancels the
// underlying HTTP request itself.
const { filter, signal } = options;
const sp = new URLSearchParams();
if (filter?.kinds) sp.set("kinds", filter.kinds.join(","));
if (filter?.excludeKinds)
sp.set("excludeKinds", filter.excludeKinds.join(","));
if (filter?.abodes) sp.set("abodes", filter.abodes.join(","));
if (filter?.users) sp.set("users", filter.users.join(","));
const query = sp.toString();
const url = this.#root + "/export" + (query ? `?${query}` : "");
const headers = { ...this.#headers };
async function* generate(): AsyncGenerator<Buffer | Uint8Array> {
const res = await fetch(url, { method: "GET", headers, signal });
if (!res.ok) {
const text = await res.text();
switch (res.status) {
case 400:
throw new InvalidAbodeError();
case 401:
throw new NotAuthorizedAbodeError();
case 403:
throw new ReadonlyAbodeError();
case 404:
throw new NotFoundAbodeError();
case 409:
throw new ConflictAbodeError();
default:
throw new Error(`${res.status} ${res.statusText} ${text}`);
}
}
if (!res.body) return;
yield* Readable.fromWeb(res.body as WebReadableStream<Uint8Array>);
}
return Readable.from(generate());
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ import { apiProtocols } from "./url.js";
const getApi = () =>
import(/* webpackChunkName: 'dbsource-api' */ "./getdb.static.js").then(
(x) => x.default
(x) => x.default,
);
const getApiDynamic: GetDbDynamic = {
+4 -2
View File
@@ -17,7 +17,9 @@ export function parseApiUrl(url: string) {
urlObj.href = urlObj.href.replace(/^abode\+/, "");
const readonly = (urlObj.searchParams.get("readonly") ?? "0") !== "0";
const headers = Object.fromEntries(
[...urlObj.searchParams.entries()].filter(([param]) => param !== "readonly")
[...urlObj.searchParams.entries()].filter(
([param]) => param !== "readonly",
),
);
if (urlObj.username) {
headers["Authorization"] =
@@ -26,7 +28,7 @@ export function parseApiUrl(url: string) {
[
decodeURIComponent(urlObj.username),
decodeURIComponent(urlObj.password),
].join(":")
].join(":"),
);
urlObj.username = "";
urlObj.password = "";
+1 -1
View File
@@ -15,7 +15,7 @@ export async function getDbSources(url: string): Promise<GetDbStatic[]> {
.catch(() => null)
.then((dbSource) => {
if (dbSource) dbSources.push(dbSource);
})
}),
);
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ import type { GetDbStatic } from "./types/GetDb.js";
let rawGetDbSources: typeof getDbSources | undefined = undefined;
const getGetDbSources = () =>
import(/* webpackChunkName: 'dbsources' */ "./dbSources.static.js").then(
(x) => x.getDbSources
(x) => x.getDbSources,
);
export async function getDbSources(url: string): Promise<GetDbStatic[]> {
+82
View File
@@ -0,0 +1,82 @@
import {
EXPORT_KIND_ORDER,
type ExportFilter,
type ExportKind,
} from "../types/ExportImport.js";
const EXPORT_KINDS = new Set<ExportKind>(EXPORT_KIND_ORDER);
export function isExportKind(x: unknown): x is ExportKind {
return typeof x === "string" && EXPORT_KINDS.has(x as ExportKind);
}
/** Whether a `kind` survives a filter's `kinds`/`excludeKinds` rules. */
export function kindAllowed(
filter: ExportFilter | undefined,
kind: ExportKind,
): boolean {
if (!filter) return true;
if (filter.kinds && !filter.kinds.includes(kind)) return false;
if (filter.excludeKinds && filter.excludeKinds.includes(kind)) return false;
return true;
}
/**
* Whether an individual record passes a filter's `abodes`/`users`/`apikeys`
* allowlists. `abodes` scopes abode/resident/note (by aid); `users` scopes user
* (by uid); `apikey` records are scoped by `apikeys` when present, else by
* `users`. An absent allowlist means "unrestricted".
*/
export function recordAllowed(
filter: ExportFilter | undefined,
kind: ExportKind,
record: { uid?: string; aid?: string },
): boolean {
if (!filter) return true;
switch (kind) {
case "user":
return !filter.users || filter.users.includes(record.uid as string);
case "apikey": {
const allow = filter.apikeys ?? filter.users;
return !allow || allow.includes(record.uid as string);
}
case "abode":
case "resident":
case "note":
return !filter.abodes || filter.abodes.includes(record.aid as string);
}
}
function intersectList<T extends string>(a?: T[], b?: T[]): T[] | undefined {
if (!a) return b;
if (!b) return a;
const bs = new Set(b);
return a.filter((x) => bs.has(x));
}
function unionList<T extends string>(a?: T[], b?: T[]): T[] | undefined {
if (!a) return b;
if (!b) return a;
return [...new Set([...a, ...b])];
}
/**
* Combine two filters as a hard intersection: the result can never permit more
* than either input. An absent allowlist is treated as "unrestricted", so
* intersecting it with a present one yields the present one. `excludeKinds`
* are unioned (either exclusion still excludes).
*/
export function intersectExportFilters(
a: ExportFilter | null | undefined,
b: ExportFilter | null | undefined,
): ExportFilter {
if (!a) return b ?? {};
if (!b) return a;
return {
kinds: intersectList(a.kinds, b.kinds),
excludeKinds: unionList(a.excludeKinds, b.excludeKinds),
abodes: intersectList(a.abodes, b.abodes),
users: intersectList(a.users, b.users),
apikeys: intersectList(a.apikeys, b.apikeys),
};
}
+60
View File
@@ -0,0 +1,60 @@
import readline from "node:readline";
import type {
ExportFilter,
ExportKind,
ExportMeta,
} from "../types/ExportImport.js";
import { isExportKind } from "./filter.js";
export type InspectResult = {
counts: Partial<Record<ExportKind, number>>;
meta?: Partial<ExportMeta> & { filter?: ExportFilter };
};
/**
* Tally the kinds/counts contained in an NDJSON export stream without ever
* touching a database. Works on any stream — a file, {@link ApiInterface}'s
* export, or a pipe straight from {@link SqliteInterface}'s export.
*
* If `stopAfterKinds` is given, reading stops as soon as at least one record of
* every requested kind has been seen, rather than draining to EOF — useful for
* probing large dumps ("does this contain notes at all?").
*/
export async function inspectExportStream(
source: NodeJS.ReadableStream,
options: { signal?: AbortSignal; stopAfterKinds?: ExportKind[] } = {},
): Promise<InspectResult> {
const { signal, stopAfterKinds } = options;
const counts: Partial<Record<ExportKind, number>> = {};
let meta: InspectResult["meta"];
const rl = readline.createInterface({
input: source,
crlfDelay: Infinity,
signal,
});
try {
for await (const raw of rl) {
signal?.throwIfAborted();
const line = raw.trim();
if (!line) continue;
let parsed: { kind?: unknown; data?: unknown };
try {
parsed = JSON.parse(line);
} catch {
continue;
}
if (parsed.kind === "meta") {
meta = parsed.data as InspectResult["meta"];
continue;
}
if (!isExportKind(parsed.kind)) continue;
counts[parsed.kind] = (counts[parsed.kind] ?? 0) + 1;
if (stopAfterKinds && stopAfterKinds.every((k) => counts[k])) break;
}
} finally {
rl.close();
}
return { counts, meta };
}
+309 -44
View File
@@ -35,6 +35,9 @@ import {
selectClientUsers,
selectResident,
selectResidents,
selectNote,
selectNotes,
selectPartialNotes,
} from "./query.js";
import type {
CreateNote,
@@ -43,8 +46,22 @@ import type {
UpdateNote,
} from "../types/Note.js";
import type { WrappedPgClient } from "./pool.js";
import { Readable } from "node:stream";
import readline from "node:readline";
import {
EXPORT_KIND_ORDER,
type Exportable,
type ExportKind,
type ExportOptions,
type Importable,
type ImportOptions,
type ImportResult,
} from "../types/ExportImport.js";
import { isExportKind, kindAllowed, recordAllowed } from "../export/filter.js";
export class PostgresInterface implements BackendDbInterface {
export class PostgresInterface
implements BackendDbInterface, Exportable, Importable
{
#db: WrappedPgClient;
constructor(db: WrappedPgClient) {
@@ -92,7 +109,7 @@ export class PostgresInterface implements BackendDbInterface {
async getUserByEmail(email: string): Promise<ClientUser> {
const user = await selectClientUser(
this.#db,
sql`u."email" = ${{ text: email }}`
sql`u."email" = ${{ text: email }}`,
);
if (!user) throw new NotFoundAbodeError();
return user;
@@ -111,7 +128,7 @@ export class PostgresInterface implements BackendDbInterface {
SELECT "uid", "email", "name", "flags", "created_at", "updated_at", "password"
FROM "users"
WHERE "email" = ${{ text: email }}
`
`,
);
if (!rawUser) throw new NotFoundAbodeError();
if (rawUser.password.startsWith("#")) throw new ConflictAbodeError();
@@ -125,7 +142,7 @@ export class PostgresInterface implements BackendDbInterface {
sql`
DELETE FROM "users"
WHERE "uid" = ${{ uuid: id }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
@@ -145,10 +162,10 @@ export class PostgresInterface implements BackendDbInterface {
${{ text: user.password }},
${{ jsonb: user.flags }}
)
`
`,
);
return this.#getUserById(uid, tx);
})
}),
);
}
async updateUser(user: UpdateUser): Promise<ClientUser> {
@@ -161,7 +178,8 @@ export class PostgresInterface implements BackendDbInterface {
if ("name" in user && user.name !== undefined)
updates.push(sql`"name" = ${{ text: user.name }}`);
if ("password" in user && user.password !== undefined) {
if (!isValidUserPassword(user.password)) throw new InvalidAbodeError();
if (!isValidUserPassword(user.password))
throw new InvalidAbodeError();
if (user.password.startsWith("#")) {
await tx.run(sql`
DELETE FROM "apikeys"
@@ -185,11 +203,11 @@ export class PostgresInterface implements BackendDbInterface {
"updated_at" = NOW(),
${joinSql(updates, sql`, `)}
WHERE "uid" = ${{ uuid: user.uid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getUserById(user.uid, tx);
})
}),
);
}
@@ -210,7 +228,7 @@ export class PostgresInterface implements BackendDbInterface {
sql`
DELETE FROM "abodes"
WHERE "aid" = ${{ uuid: id }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
@@ -228,10 +246,10 @@ export class PostgresInterface implements BackendDbInterface {
${{ uuid: ctx.uid }},
${{ uuid: ctx.uid }}
)
`
`,
);
return this.#getAbodeById(aid, tx);
})
}),
);
}
async updateAbode(abode: UpdateAbode, ctx: { uid: string }): Promise<Abode> {
@@ -250,11 +268,11 @@ export class PostgresInterface implements BackendDbInterface {
"updated_by" = ${{ uuid: ctx.uid }},
${joinSql(updates, sql`, `)}
WHERE "aid" = ${{ uuid: abode.aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getAbodeById(abode.aid, tx);
})
}),
);
}
@@ -270,11 +288,11 @@ export class PostgresInterface implements BackendDbInterface {
async #getResidentById(
uid: string,
aid: string,
db: WrappedPgClient
db: WrappedPgClient,
): Promise<Resident> {
const resident = await selectResident(
db,
sql`"uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}`
sql`"uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}`,
);
if (!resident) throw new NotFoundAbodeError();
return resident;
@@ -288,13 +306,13 @@ export class PostgresInterface implements BackendDbInterface {
sql`
DELETE FROM "residents"
WHERE "uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
async createResident(
resident: CreateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident> {
this.#checkReadonly();
return this.#db.rethrow(() =>
@@ -309,15 +327,15 @@ export class PostgresInterface implements BackendDbInterface {
${{ uuid: ctx.uid }},
${{ uuid: ctx.uid }}
)
`
`,
);
return this.#getResidentById(resident.uid, resident.aid, tx);
})
}),
);
}
async updateResident(
resident: updateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident> {
this.#checkReadonly();
const updates = calcUpdates({
@@ -336,11 +354,11 @@ export class PostgresInterface implements BackendDbInterface {
WHERE
"uid" = ${{ uuid: resident.uid }}
AND "aid" = ${{ uuid: resident.aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getResidentById(resident.uid, resident.aid, tx);
})
}),
);
}
@@ -350,7 +368,7 @@ export class PostgresInterface implements BackendDbInterface {
sql`
JOIN "residents" r ON u."uid" = r."uid"
WHERE r."aid" = ${{ uuid: id }}
`
`,
);
}
async listAbodesByUserId(id: string): Promise<Abode[]> {
@@ -359,7 +377,7 @@ export class PostgresInterface implements BackendDbInterface {
sql`
JOIN "residents" r ON a."aid" = r."aid"
WHERE r."uid" = ${{ uuid: id }}
`
`,
);
}
@@ -418,17 +436,17 @@ export class PostgresInterface implements BackendDbInterface {
async #getApikeyByToken(
token: `at_${string}`,
db: WrappedPgClient
db: WrappedPgClient,
): Promise<ClientApikey> {
const apikey = await selectClientApikey(
db,
sql`k."token" = ${{ text: token }}`
sql`k."token" = ${{ text: token }}`,
);
if (!apikey) throw new NotFoundAbodeError();
return apikey;
}
async getUserByApikey(
token: `at_${string}`
token: `at_${string}`,
): Promise<[ClientUser, ClientApikey]> {
const apikey = await this.#getApikeyByToken(token, this.#db);
if (
@@ -445,13 +463,13 @@ export class PostgresInterface implements BackendDbInterface {
async getApikeyById(kid: string): Promise<ClientApikey> {
const apikey = await selectClientApikey(
this.#db,
sql`k."kid" = ${{ uuid: kid }}`
sql`k."kid" = ${{ uuid: kid }}`,
);
if (!apikey) throw new NotFoundAbodeError();
return apikey;
}
async createApikey(
apikey: CreateApikey
apikey: CreateApikey,
): Promise<[ClientApikey, `at_${string}`]> {
this.#checkReadonly();
const token = createApikeyToken();
@@ -459,7 +477,7 @@ export class PostgresInterface implements BackendDbInterface {
let expires = apikey.expires_at;
if (expires === undefined)
expires = new Date(
new Date().getTime() + 1000 * 60 * 60 * 24 * 365
new Date().getTime() + 1000 * 60 * 60 * 24 * 365,
).toISOString();
if (expires && new Date(expires).getTime() < Date.now())
throw new InvalidAbodeError();
@@ -487,24 +505,271 @@ export class PostgresInterface implements BackendDbInterface {
}
async listNotes(): Promise<PartialNote[]> {
throw new Error("Unimplemented");
return selectPartialNotes(this.#db);
}
async getNoteById(_nid: string): Promise<Note> {
throw new Error("Unimplemented");
async #getNoteById(nid: string, db: WrappedPgClient): Promise<Note> {
const note = await selectNote(db, sql`n."nid" = ${{ uuid: nid }}`);
if (!note) throw new NotFoundAbodeError();
return note;
}
async deleteNoteById(_nid: string): Promise<void> {
throw new Error("Unimplemented");
async getNoteById(nid: string): Promise<Note> {
return this.#getNoteById(nid, this.#db);
}
async createNote(_note: CreateNote, _ctx: { uid: string }): Promise<Note> {
throw new Error("Unimplemented");
async deleteNoteById(nid: string): Promise<void> {
this.#checkReadonly();
const { changes } = await this.#db.run(sql`
DELETE FROM "notes" WHERE "nid" = ${{ uuid: nid }}
`);
if (!changes) throw new NotFoundAbodeError();
}
async updateNote(_note: UpdateNote, _ctx: { uid: string }): Promise<Note> {
throw new Error("Unimplemented");
async createNote(note: CreateNote, ctx: { uid: string }): Promise<Note> {
this.#checkReadonly();
const nid = crypto.randomUUID();
return this.#db.rethrow(() =>
this.#db.multi(async (tx) => {
await tx.run(sql`
INSERT INTO "notes"("nid", "aid", "name", "content", "properties", "created_by", "updated_by")
VALUES(
${{ uuid: nid }}, ${{ uuid: note.aid }}, ${{ text: note.name }},
${{ text: note.content ?? "" }}, ${{ jsonb: note.properties }},
${{ uuid: ctx.uid }}, ${{ uuid: ctx.uid }}
)
`);
return this.#getNoteById(nid, tx);
}),
);
}
async listNotesByAbodeId(_aid: string): Promise<PartialNote[]> {
throw new Error("Unimplemented");
async updateNote(note: UpdateNote, ctx: { uid: string }): Promise<Note> {
this.#checkReadonly();
const updates = calcUpdates({
name: (value: string) => sql`"name" = ${{ text: value }}`,
content: (value: string) => sql`"content" = ${{ text: value }}`,
properties: (value: object) => sql`"properties" = ${{ jsonb: value }}`,
})(note);
if (!updates.length) throw new InvalidAbodeError();
return this.#db.rethrow(() =>
this.#db.multi(async (tx) => {
const { changes } = await tx.run(sql`
UPDATE "notes"
SET "updated_at" = NOW(), "updated_by" = ${{ uuid: ctx.uid }},
${joinSql(updates, sql`, `)}
WHERE "nid" = ${{ uuid: note.nid }}
`);
if (!changes) throw new NotFoundAbodeError();
return this.#getNoteById(note.nid, tx);
}),
);
}
async listNotesByUserId(_uid: string): Promise<PartialNote[]> {
throw new Error("Unimplemented");
async listNotesByAbodeId(aid: string): Promise<PartialNote[]> {
return selectPartialNotes(this.#db, sql`n."aid" = ${{ uuid: aid }}`);
}
async listNotesByUserId(uid: string): Promise<PartialNote[]> {
return selectPartialNotes(this.#db, sql`n."created_by" = ${{ uuid: uid }}`);
}
export(options: ExportOptions = {}): NodeJS.ReadableStream {
const { filter, signal } = options;
const db = this.#db;
const source = this.name;
// Emission follows the FK-safe EXPORT_KIND_ORDER (part of the wire
// contract; each `load()` is a single query, run lazily and skipped once
// the destination aborts.
const loaders: Partial<
Record<ExportKind, () => Promise<{ uid?: string; aid?: string }[]>>
> = {
user: () => selectClientUsers(db),
abode: () => selectAbodes(db),
resident: () => selectResidents(db),
apikey: () => selectClientApikeys(db),
note: () => selectNotes(db),
};
async function* generate(): AsyncGenerator<string> {
if (signal?.aborted) return;
yield JSON.stringify({
kind: "meta",
data: {
v: 1,
exportedAt: new Date().toISOString(),
source,
filter: filter ?? {},
},
}) + "\n";
try {
for (const kind of EXPORT_KIND_ORDER) {
if (signal?.aborted) return;
const load = loaders[kind];
if (!load) continue;
if (!kindAllowed(filter, kind)) continue;
for (const row of await load()) {
if (signal?.aborted) return;
if (recordAllowed(filter, kind, row)) {
yield JSON.stringify({ kind, data: row }) + "\n";
}
}
}
} catch (e) {
if (signal?.aborted) return;
yield JSON.stringify({
kind: "error",
data: {
message: e instanceof Error ? e.message : String(e),
code: e instanceof Error ? e.name : undefined,
},
}) + "\n";
}
}
return Readable.from(generate());
}
async import(
source: NodeJS.ReadableStream,
options: ImportOptions = {},
): Promise<ImportResult> {
this.#checkReadonly();
const { filter, signal } = options;
const counts: Partial<Record<ExportKind, number>> = {};
const rl = readline.createInterface({
input: source,
crlfDelay: Infinity,
signal,
});
// Postgres holds FK enforcement (unlike sqlite, which we toggle off): the
// FK-safe insertion order keeps a full dump valid.
// The transaction commits only if the whole stream is consumed cleanly; an
// error/abort rolls it back via `multi`.
try {
await this.#db.rethrow(() =>
this.#db.multi(async (tx) => {
for await (const raw of rl) {
signal?.throwIfAborted();
const line = raw.trim();
if (!line) continue;
let parsed: { kind?: unknown; data?: unknown };
try {
parsed = JSON.parse(line);
} catch {
throw new InvalidAbodeError();
}
if (parsed.kind === "meta") continue;
if (parsed.kind === "error") {
throw new Error(
`export stream reported an error: ${
(parsed.data as { message?: string })?.message ?? "unknown"
}`,
);
}
if (!isExportKind(parsed.kind)) continue;
if (!kindAllowed(filter, parsed.kind)) continue;
const data = parsed.data as { uid?: string; aid?: string };
if (!recordAllowed(filter, parsed.kind, data)) continue;
await this.#importRecord(tx, parsed.kind, parsed.data);
counts[parsed.kind] = (counts[parsed.kind] ?? 0) + 1;
}
// An abort while blocked on the source closes readline without
// throwing, so re-check before the transaction commits.
signal?.throwIfAborted();
}),
);
} finally {
rl.close();
}
return { counts };
}
async #importRecord(
tx: WrappedPgClient,
kind: ExportKind,
data: unknown,
): Promise<void> {
switch (kind) {
case "user": {
const u = data as ClientUser;
// `password` is never exported; imported users land on the schema
// default ('#unset') and must reset before they can log in.
await tx.run(sql`
INSERT INTO "users"("uid", "email", "name", "flags", "created_at", "updated_at")
VALUES(
${{ uuid: u.uid }},
${{ text: u.email }},
${{ text: u.name }},
${{ jsonb: u.flags }},
${{ date: u.created_at }},
${{ date: u.updated_at }}
)
`);
break;
}
case "abode": {
const a = data as Abode;
await tx.run(sql`
INSERT INTO "abodes"("aid", "name", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: a.aid }},
${{ text: a.name }},
${{ date: a.created_at }},
${a.created_by ? { uuid: a.created_by } : { null: true }},
${{ date: a.updated_at }},
${a.updated_by ? { uuid: a.updated_by } : { null: true }}
)
`);
break;
}
case "resident": {
const r = data as Resident;
await tx.run(sql`
INSERT INTO "residents"("uid", "aid", "flags", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: r.uid }},
${{ uuid: r.aid }},
${{ jsonb: r.flags }},
${{ date: r.created_at }},
${r.created_by ? { uuid: r.created_by } : { null: true }},
${{ date: r.updated_at }},
${r.updated_by ? { uuid: r.updated_by } : { null: true }}
)
`);
break;
}
case "apikey": {
const k = data as ClientApikey;
// `token` is never exported; mint a fresh unique one so the record's
// metadata (kid/permissions/expiry) survives even though the original
// secret cannot.
await tx.run(sql`
INSERT INTO "apikeys"("uid", "kid", "token", "name", "permissions", "created_at", "expires_at")
VALUES(
${{ uuid: k.uid }},
${{ uuid: k.kid }},
${{ text: createApikeyToken() }},
${{ text: k.name }},
${{ jsonb: k.permissions }},
${{ date: k.created_at }},
${k.expires_at ? { date: k.expires_at } : { null: true }}
)
`);
break;
}
case "note": {
const n = data as Note;
await tx.run(sql`
INSERT INTO "notes"("nid", "aid", "name", "content", "properties", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: n.nid }}, ${{ uuid: n.aid }}, ${{ text: n.name }},
${{ text: n.content ?? "" }}, ${{ jsonb: n.properties }},
${{ date: n.created_at }},
${n.created_by ? { uuid: n.created_by } : { null: true }},
${{ date: n.updated_at }},
${n.updated_by ? { uuid: n.updated_by } : { null: true }}
)
`);
break;
}
}
}
}
+5 -6
View File
@@ -24,7 +24,7 @@ export class PostgresMigrator implements Migrator {
`SELECT EXISTS (
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = '_migrations'
) AS "exists"`
) AS "exists"`,
);
if (!existsResult.rows[0]?.exists) return null;
@@ -33,7 +33,7 @@ export class PostgresMigrator implements Migrator {
name: string;
applied_at: Date;
}>(
`SELECT "id", "name", "applied_at" FROM "_migrations" ORDER BY "id" ASC`
`SELECT "id", "name", "applied_at" FROM "_migrations" ORDER BY "id" ASC`,
);
return result.rows.map((x) => ({
...x,
@@ -73,17 +73,16 @@ export class PostgresMigrator implements Migrator {
throw new Error(`Applied migration ${id} (${name}) not known`);
if (migration.name !== name)
throw new Error(
`Applied migration ${id} (${name}) has a different name from expected (${migration.name})`
`Applied migration ${id} (${name}) has a different name from expected (${migration.name})`,
);
}
const start =
migrations.findIndex((x) => x.id === current.at(-1)?.id) + 1;
const start = migrations.findIndex((x) => x.id === current.at(-1)?.id) + 1;
const end = migrations.indexOf(target) + 1;
if (end < start) {
throw new Error(
`Cannot migrate backward, at ${current.at(-1)?.id ?? 0}, going to ${target.id}`
`Cannot migrate backward, at ${current.at(-1)?.id ?? 0}, going to ${target.id}`,
);
}
+71
View File
@@ -1,6 +1,12 @@
import type { Abode } from "../types/Abode.js";
import type { ApikeyPermissions, ClientApikey } from "../types/Apikey.js";
import type { Resident, ResidentFlags } from "../types/Resident.js";
import type {
Note,
NoteProperties,
PartialNote,
PartialNoteProperties,
} from "../types/Note.js";
import type { ClientUser, PartialUser, UserFlags } from "../types/User.js";
export function pgToDate(d: Date | string): string {
@@ -135,3 +141,68 @@ export function pgToClientApikey(apikey: {
expires_at: apikey.expires_at ? pgToDate(apikey.expires_at) : null,
};
}
const validNoteTypes = new Set(["note"]);
function pgToNoteProperties(properties: unknown): NoteProperties {
if (
typeof properties !== "object" ||
!properties ||
Array.isArray(properties)
)
return {};
const value = properties as Record<string, unknown>;
return validNoteTypes.has(value.type as string)
? { type: value.type as "note" }
: {};
}
function pgToPartialNoteProperties(properties: unknown): PartialNoteProperties {
return { type: pgToNoteProperties(properties).type ?? "note" };
}
export function pgToNote(note: {
nid: string;
aid: string;
name: string;
content: string;
properties: unknown;
created_at: Date | string;
created_by: string | null;
updated_at: Date | string;
updated_by: string | null;
}): Note {
return {
nid: note.nid,
aid: note.aid,
name: note.name,
content: note.content,
properties: pgToNoteProperties(note.properties),
created_at: pgToDate(note.created_at),
created_by: note.created_by,
updated_at: pgToDate(note.updated_at),
updated_by: note.updated_by,
};
}
export function pgToPartialNote(note: {
nid: string;
aid: string;
name: string;
properties: unknown;
created_at: Date | string;
created_by: string | null;
updated_at: Date | string;
updated_by: string | null;
}): PartialNote {
return {
nid: note.nid,
aid: note.aid,
name: note.name,
properties: pgToPartialNoteProperties(note.properties),
created_at: pgToDate(note.created_at),
created_by: note.created_by,
updated_at: pgToDate(note.updated_at),
updated_by: note.updated_by,
};
}
+3 -5
View File
@@ -31,9 +31,7 @@ async function rethrow<R>(fn: () => Promise<R>): Promise<R> {
// Rollback on a broken connection can itself throw; the original error is
// the one worth surfacing.
export async function rollbackQuietly(
client: pg.PoolClient
): Promise<void> {
export async function rollbackQuietly(client: pg.PoolClient): Promise<void> {
try {
await client.query("ROLLBACK");
} catch {}
@@ -58,7 +56,7 @@ abstract class WrappedPgBase implements WrappedPgClient {
async all<R>(stmt: SqlCode): Promise<R[]> {
const result = await this.#queryable.query(
toPositional(stmt._sql),
stmt._vars
stmt._vars,
);
return result.rows as R[];
}
@@ -72,7 +70,7 @@ abstract class WrappedPgBase implements WrappedPgClient {
async run(stmt: SqlCode): Promise<{ changes: number }> {
const result = await this.#queryable.query(
toPositional(stmt._sql),
stmt._vars
stmt._vars,
);
return { changes: result.rowCount ?? 0 };
}
+67 -16
View File
@@ -2,11 +2,14 @@ import type { Abode } from "../types/Abode.js";
import type { ClientApikey } from "../types/Apikey.js";
import type { Resident } from "../types/Resident.js";
import type { ClientUser } from "../types/User.js";
import type { Note, PartialNote } from "../types/Note.js";
import {
pgToAbode,
pgToClientApikey,
pgToClientUser,
pgToResident,
pgToNote,
pgToPartialNote,
} from "./cast.js";
import type { WrappedPgClient } from "./pool.js";
import { sql, type SqlCode } from "./sql.js";
@@ -26,20 +29,18 @@ const sqlClientUser = sql`
export async function selectClientUser(
db: WrappedPgClient,
where: SqlCode
where: SqlCode,
): Promise<ClientUser | null> {
const raw = await db.get<RawClientUser>(
sql`${sqlClientUser} WHERE ${where}`
);
const raw = await db.get<RawClientUser>(sql`${sqlClientUser} WHERE ${where}`);
if (raw) return pgToClientUser(raw);
return null;
}
export async function selectClientUsers(
db: WrappedPgClient,
rest?: SqlCode
rest?: SqlCode,
): Promise<ClientUser[]> {
const rows = await db.all<RawClientUser>(
rest ? sql`${sqlClientUser} ${rest}` : sqlClientUser
rest ? sql`${sqlClientUser} ${rest}` : sqlClientUser,
);
return rows.map(pgToClientUser);
}
@@ -59,7 +60,7 @@ const sqlAbode = sql`
export async function selectAbode(
db: WrappedPgClient,
where: SqlCode
where: SqlCode,
): Promise<Abode | null> {
const raw = await db.get<RawAbode>(sql`${sqlAbode} WHERE ${where}`);
if (raw) return pgToAbode(raw);
@@ -67,10 +68,10 @@ export async function selectAbode(
}
export async function selectAbodes(
db: WrappedPgClient,
rest?: SqlCode
rest?: SqlCode,
): Promise<Abode[]> {
const rows = await db.all<RawAbode>(
rest ? sql`${sqlAbode} ${rest}` : sqlAbode
rest ? sql`${sqlAbode} ${rest}` : sqlAbode,
);
return rows.map(pgToAbode);
}
@@ -91,7 +92,7 @@ const sqlResident = sql`
export async function selectResident(
db: WrappedPgClient,
where: SqlCode
where: SqlCode,
): Promise<Resident | null> {
const raw = await db.get<RawResident>(sql`${sqlResident} WHERE ${where}`);
if (raw) return pgToResident(raw);
@@ -99,10 +100,10 @@ export async function selectResident(
}
export async function selectResidents(
db: WrappedPgClient,
where?: SqlCode
where?: SqlCode,
): Promise<Resident[]> {
const rows = await db.all<RawResident>(
where ? sql`${sqlResident} WHERE ${where}` : sqlResident
where ? sql`${sqlResident} WHERE ${where}` : sqlResident,
);
return rows.map(pgToResident);
}
@@ -122,20 +123,70 @@ const sqlClientApikey = sql`
export async function selectClientApikey(
db: WrappedPgClient,
where: SqlCode
where: SqlCode,
): Promise<ClientApikey | null> {
const raw = await db.get<RawClientApikey>(
sql`${sqlClientApikey} WHERE ${where}`
sql`${sqlClientApikey} WHERE ${where}`,
);
if (raw) return pgToClientApikey(raw);
return null;
}
export async function selectClientApikeys(
db: WrappedPgClient,
where: SqlCode
where?: SqlCode,
): Promise<ClientApikey[]> {
const rows = await db.all<RawClientApikey>(
sql`${sqlClientApikey} WHERE ${where}`
where ? sql`${sqlClientApikey} WHERE ${where}` : sqlClientApikey,
);
return rows.map(pgToClientApikey);
}
type RawNote = {
nid: string;
aid: string;
name: string;
content: string;
properties: unknown;
created_at: Date;
created_by: string | null;
updated_at: Date;
updated_by: string | null;
};
const sqlNote = sql`
SELECT n."nid", n."aid", n."name", n."content", n."properties",
n."created_at", n."created_by", n."updated_at", n."updated_by"
FROM "notes" n
`;
type RawPartialNote = Omit<RawNote, "content">;
const sqlPartialNote = sql`
SELECT n."nid", n."aid", n."name", n."properties",
n."created_at", n."created_by", n."updated_at", n."updated_by"
FROM "notes" n
`;
export async function selectNote(
db: WrappedPgClient,
where: SqlCode,
): Promise<Note | null> {
const raw = await db.get<RawNote>(sql`${sqlNote} WHERE ${where}`);
return raw ? pgToNote(raw) : null;
}
export async function selectNotes(
db: WrappedPgClient,
where?: SqlCode,
): Promise<Note[]> {
const rows = await db.all<RawNote>(
where ? sql`${sqlNote} WHERE ${where}` : sqlNote,
);
return rows.map(pgToNote);
}
export async function selectPartialNotes(
db: WrappedPgClient,
where?: SqlCode,
): Promise<PartialNote[]> {
const rows = await db.all<RawPartialNote>(
where ? sql`${sqlPartialNote} WHERE ${where}` : sqlPartialNote,
);
return rows.map(pgToPartialNote);
}
+1 -1
View File
@@ -65,7 +65,7 @@ export function calcUpdates<T extends object>(updater: {
for (const [prop, update] of Object.entries(updater)) {
if (prop in obj) {
updates.push(
(update as (value: unknown) => SqlCode)(obj[prop as keyof T]!)
(update as (value: unknown) => SqlCode)(obj[prop as keyof T]!),
);
}
}
+4 -1
View File
@@ -9,7 +9,10 @@ export function isPgUrl(url: string): boolean {
}
}
export function parsePgUrl(url: string): { connectionString: string; readonly: boolean } {
export function parsePgUrl(url: string): {
connectionString: string;
readonly: boolean;
} {
if (!isPgUrl(url)) throw new Error("Not a postgres: URL");
const urlObj = new URL(url);
const readonly = (urlObj.searchParams.get("readonly") ?? "0") !== "0";
+261 -29
View File
@@ -34,6 +34,7 @@ import {
selectClientUser,
selectClientUsers,
selectNote,
selectNotes,
selectPartialNotes,
selectResident,
selectResidents,
@@ -45,8 +46,22 @@ import type {
UpdateNote,
} from "../types/Note.js";
import type { WrappedDb } from "./impl/types.js";
import { Readable } from "node:stream";
import readline from "node:readline";
import {
EXPORT_KIND_ORDER,
type Exportable,
type ExportKind,
type ExportOptions,
type Importable,
type ImportOptions,
type ImportResult,
} from "../types/ExportImport.js";
import { isExportKind, kindAllowed, recordAllowed } from "../export/filter.js";
export class SqliteInterface implements BackendDbInterface {
export class SqliteInterface
implements BackendDbInterface, Exportable, Importable
{
#db: WrappedDb;
constructor(db: WrappedDb) {
@@ -110,7 +125,7 @@ export class SqliteInterface implements BackendDbInterface {
SELECT "uid", "email", "name", json("flags") AS "flags", "created_at", "updated_at", "password"
FROM "users"
WHERE "email" = ${{ text: email }}
`
`,
);
if (!rawUser) throw new NotFoundAbodeError();
if (rawUser.password.startsWith("#")) throw new ConflictAbodeError();
@@ -124,7 +139,7 @@ export class SqliteInterface implements BackendDbInterface {
sql`
DELETE FROM "users"
WHERE "uid" = ${{ uuid: id }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
@@ -142,10 +157,10 @@ export class SqliteInterface implements BackendDbInterface {
${{ text: user.email }},
${{ text: user.name }},
${{ text: user.password }},${{ jsonb: user.flags }})
`
`,
);
return this.#getUserById(uid);
})
}),
);
}
async updateUser(user: UpdateUser): Promise<ClientUser> {
@@ -179,11 +194,11 @@ export class SqliteInterface implements BackendDbInterface {
"updated_at" = datetime('now', 'localtime', 'subsec'),
${joinSql(updates, sql`, `)}
WHERE "uid" = ${{ uuid: user.uid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getUserById(user.uid);
})
}),
);
}
@@ -204,7 +219,7 @@ export class SqliteInterface implements BackendDbInterface {
sql`
DELETE FROM "abodes"
WHERE "aid" = ${{ uuid: id }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
@@ -222,10 +237,10 @@ export class SqliteInterface implements BackendDbInterface {
${{ uuid: ctx.uid }},
${{ uuid: ctx.uid }}
)
`
`,
);
return this.#getAbodeById(aid);
})
}),
);
}
async updateAbode(abode: UpdateAbode, ctx: { uid: string }): Promise<Abode> {
@@ -244,11 +259,11 @@ export class SqliteInterface implements BackendDbInterface {
"updated_by" = ${{ uuid: ctx.uid }},
${joinSql(updates, sql`, `)}
WHERE "aid" = ${{ uuid: abode.aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getAbodeById(abode.aid);
})
}),
);
}
@@ -264,7 +279,7 @@ export class SqliteInterface implements BackendDbInterface {
#getResidentById(uid: string, aid: string): Resident {
const resident = selectResident(
this.#db,
sql`"uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}`
sql`"uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}`,
);
if (!resident) throw new NotFoundAbodeError();
return resident;
@@ -278,13 +293,13 @@ export class SqliteInterface implements BackendDbInterface {
sql`
DELETE FROM "residents"
WHERE "uid" = ${{ uuid: uid }} AND "aid" = ${{ uuid: aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
}
async createResident(
resident: CreateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident> {
this.#checkReadonly();
return this.#db.rethrow(() =>
@@ -299,15 +314,15 @@ export class SqliteInterface implements BackendDbInterface {
${{ uuid: ctx.uid }},
${{ uuid: ctx.uid }}
)
`
`,
);
return this.#getResidentById(resident.uid, resident.aid);
})
}),
);
}
async updateResident(
resident: updateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident> {
this.#checkReadonly();
const updates = calcUpdates({
@@ -326,11 +341,11 @@ export class SqliteInterface implements BackendDbInterface {
WHERE
"uid" = ${{ uuid: resident.uid }}
AND "aid" = ${{ uuid: resident.aid }}
`
`,
);
if (!changes) throw new NotFoundAbodeError();
return this.#getResidentById(resident.uid, resident.aid);
})
}),
);
}
@@ -340,7 +355,7 @@ export class SqliteInterface implements BackendDbInterface {
sql`
JOIN "residents" r ON u."uid" = r."uid"
WHERE r."aid" = ${{ uuid: id }}
`
`,
);
}
async listAbodesByUserId(id: string): Promise<Abode[]> {
@@ -349,7 +364,7 @@ export class SqliteInterface implements BackendDbInterface {
sql`
JOIN "residents" r ON a."aid" = r."aid"
WHERE r."uid" = ${{ uuid: id }}
`
`,
);
}
@@ -405,13 +420,13 @@ export class SqliteInterface implements BackendDbInterface {
#getApikeyByToken(token: `at_${string}`): ClientApikey {
const apikey = selectClientApikey(
this.#db,
sql`"token" = ${{ text: token }}`
sql`"token" = ${{ text: token }}`,
);
if (!apikey) throw new NotFoundAbodeError();
return apikey;
}
async getUserByApikey(
token: `at_${string}`
token: `at_${string}`,
): Promise<[ClientUser, ClientApikey]> {
const apikey = this.#getApikeyByToken(token);
if (
@@ -431,7 +446,7 @@ export class SqliteInterface implements BackendDbInterface {
return apikey;
}
async createApikey(
apikey: CreateApikey
apikey: CreateApikey,
): Promise<[ClientApikey, `at_${string}`]> {
this.#checkReadonly();
const token = createApikeyToken();
@@ -439,7 +454,7 @@ export class SqliteInterface implements BackendDbInterface {
let expires = apikey.expires_at;
if (expires === undefined)
expires = new Date(
new Date().getTime() + 1000 * 60 * 60 * 24 * 365
new Date().getTime() + 1000 * 60 * 60 * 24 * 365,
).toISOString();
if (expires && new Date(expires).getTime() < Date.now())
throw new InvalidAbodeError();
@@ -480,7 +495,7 @@ export class SqliteInterface implements BackendDbInterface {
async deleteNoteById(nid: string): Promise<void> {
this.#checkReadonly();
const { changes } = this.#db.run(
sql`DELETE FROM "notes" WHERE "nid" = ${{ uuid: nid }}`
sql`DELETE FROM "notes" WHERE "nid" = ${{ uuid: nid }}`,
);
if (!changes) throw new NotFoundAbodeError();
}
@@ -502,7 +517,7 @@ export class SqliteInterface implements BackendDbInterface {
)
`);
return this.#getNoteById(nid);
})
}),
);
}
async updateNote(note: UpdateNote, ctx: { uid: string }): Promise<Note> {
@@ -525,7 +540,7 @@ export class SqliteInterface implements BackendDbInterface {
`);
if (!changes) throw new NotFoundAbodeError();
return this.#getNoteById(note.nid);
})
}),
);
}
async listNotesByAbodeId(aid: string): Promise<PartialNote[]> {
@@ -534,4 +549,221 @@ export class SqliteInterface implements BackendDbInterface {
async listNotesByUserId(uid: string): Promise<PartialNote[]> {
return selectPartialNotes(this.#db, sql`n."created_by" = ${{ uuid: uid }}`);
}
export(options: ExportOptions = {}): NodeJS.ReadableStream {
const { filter, signal } = options;
const db = this.#db;
const source = this.name;
// Per-table full materialization + JS-side per-record yielding (each
// `load()` is exactly one `WrappedDb.all()`). Kept lazy so the first query
// only fires once the destination starts pulling, and skipped entirely
// once the signal is aborted — no further reads after the destination
// goes away. Emission follows the FK-safe EXPORT_KIND_ORDER (part of the
// wire contract; see ExportImport.ts).
const loaders: Record<ExportKind, () => { uid?: string; aid?: string }[]> =
{
user: () => selectClientUsers(db),
abode: () => selectAbodes(db),
resident: () => selectResidents(db),
apikey: () => selectClientApikeys(db),
note: () => selectNotes(db),
};
async function* generate(): AsyncGenerator<string> {
if (signal?.aborted) return;
yield JSON.stringify({
kind: "meta",
data: {
v: 1,
exportedAt: new Date().toISOString(),
source,
filter: filter ?? {},
},
}) + "\n";
try {
for (const kind of EXPORT_KIND_ORDER) {
if (signal?.aborted) return;
if (!kindAllowed(filter, kind)) continue;
for (const row of loaders[kind]()) {
if (signal?.aborted) return;
if (recordAllowed(filter, kind, row)) {
yield JSON.stringify({ kind, data: row }) + "\n";
}
}
}
} catch (e) {
// Aborts unwind via early `return`, never here; a genuine mid-stream
// failure is surfaced as a trailing sentinel line (HTTP 200 headers
// are already flushed, so `convertError` can no longer apply).
if (signal?.aborted) return;
yield JSON.stringify({
kind: "error",
data: {
message: e instanceof Error ? e.message : String(e),
code: e instanceof Error ? e.name : undefined,
},
}) + "\n";
}
}
return Readable.from(generate());
}
async import(
source: NodeJS.ReadableStream,
options: ImportOptions = {},
): Promise<ImportResult> {
this.#checkReadonly();
const { filter, signal } = options;
const db = this.#db;
const counts: Partial<Record<ExportKind, number>> = {};
const rl = readline.createInterface({
input: source,
crlfDelay: Infinity,
signal,
});
// Bulk restore trusts the export's referential integrity, and a filtered
// dump may legitimately reference `created_by`/`updated_by` users outside
// its scope. Suppress FK enforcement for the duration (can only be toggled
// outside a transaction) and restore it in `finally`.
db.run(sql`PRAGMA foreign_keys = OFF`);
db.run(sql`BEGIN`);
try {
for await (const raw of rl) {
signal?.throwIfAborted();
const line = raw.trim();
if (!line) continue;
let parsed: { kind?: unknown; data?: unknown };
try {
parsed = JSON.parse(line);
} catch {
throw new InvalidAbodeError();
}
if (parsed.kind === "meta") continue;
if (parsed.kind === "error") {
throw new Error(
`export stream reported an error: ${
(parsed.data as { message?: string })?.message ?? "unknown"
}`,
);
}
if (!isExportKind(parsed.kind)) continue;
if (!kindAllowed(filter, parsed.kind)) continue;
const data = parsed.data as { uid?: string; aid?: string };
if (!recordAllowed(filter, parsed.kind, data)) continue;
this.#importRecord(parsed.kind, parsed.data);
counts[parsed.kind] = (counts[parsed.kind] ?? 0) + 1;
}
// An abort while blocked on the source closes readline without throwing,
// so re-check before committing to guarantee no partial commit.
signal?.throwIfAborted();
db.run(sql`COMMIT`);
} catch (e) {
try {
db.run(sql`ROLLBACK`);
} catch {
/* already rolled back */
}
throw e;
} finally {
rl.close();
db.run(sql`PRAGMA foreign_keys = ON`);
}
return { counts };
}
#importRecord(kind: ExportKind, data: unknown): void {
const db = this.#db;
switch (kind) {
case "user": {
const u = data as ClientUser;
// `password` is never exported; imported users land on the schema
// default ('#unset') and must reset before they can log in.
db.run(sql`
INSERT INTO "users"("uid", "email", "name", "flags", "created_at", "updated_at")
VALUES(
${{ uuid: u.uid }},
${{ text: u.email }},
${{ text: u.name }},
${{ jsonb: u.flags }},
${{ date: u.created_at }},
${{ date: u.updated_at }}
)
`);
break;
}
case "abode": {
const a = data as Abode;
db.run(sql`
INSERT INTO "abodes"("aid", "name", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: a.aid }},
${{ text: a.name }},
${{ date: a.created_at }},
${a.created_by ? { uuid: a.created_by } : { null: true }},
${{ date: a.updated_at }},
${a.updated_by ? { uuid: a.updated_by } : { null: true }}
)
`);
break;
}
case "resident": {
const r = data as Resident;
db.run(sql`
INSERT INTO "residents"("uid", "aid", "flags", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: r.uid }},
${{ uuid: r.aid }},
${{ jsonb: r.flags }},
${{ date: r.created_at }},
${r.created_by ? { uuid: r.created_by } : { null: true }},
${{ date: r.updated_at }},
${r.updated_by ? { uuid: r.updated_by } : { null: true }}
)
`);
break;
}
case "apikey": {
const k = data as ClientApikey;
// `token` is never exported; mint a fresh unique one so the record's
// metadata (kid/permissions/expiry) survives even though the original
// secret cannot.
db.run(sql`
INSERT INTO "apikeys"("uid", "kid", "token", "name", "permissions", "created_at", "expires_at")
VALUES(
${{ uuid: k.uid }},
${{ uuid: k.kid }},
${{ text: createApikeyToken() }},
${{ text: k.name }},
${{ jsonb: k.permissions }},
${{ date: k.created_at }},
${k.expires_at ? { date: k.expires_at } : { null: true }}
)
`);
break;
}
case "note": {
const n = data as Note;
db.run(sql`
INSERT INTO "notes"("nid", "aid", "name", "content", "properties", "created_at", "created_by", "updated_at", "updated_by")
VALUES(
${{ uuid: n.nid }},
${{ uuid: n.aid }},
${{ text: n.name }},
${{ text: n.content ?? "" }},
${{ jsonb: n.properties }},
${{ date: n.created_at }},
${n.created_by ? { uuid: n.created_by } : { null: true }},
${{ date: n.updated_at }},
${n.updated_by ? { uuid: n.updated_by } : { null: true }}
)
`);
break;
}
}
}
}
+5 -6
View File
@@ -16,12 +16,11 @@ export class SqliteMigrator implements Migrator {
}
#listAppliedMigrations():
| { id: number; name: string; applied_at: string }[]
| null {
{ id: number; name: string; applied_at: string }[] | null {
try {
return this.#db
.all<{ id: number; name: string; applied_at: string }>(
sql`SELECT "id", "name", "applied_at" FROM "_migrations" ORDER BY "id" ASC`
sql`SELECT "id", "name", "applied_at" FROM "_migrations" ORDER BY "id" ASC`,
)
.map((x) => ({ ...x, applied_at: sqliteToDate(x.applied_at) }));
} catch (e) {
@@ -59,7 +58,7 @@ export class SqliteMigrator implements Migrator {
throw new Error(`Applied migration ${id} (${name}) not known`);
if (migration.name !== name)
throw new Error(
`Applied migration ${id} (${name}) has a different name from expected (${migration.name})`
`Applied migration ${id} (${name}) has a different name from expected (${migration.name})`,
);
}
@@ -70,7 +69,7 @@ export class SqliteMigrator implements Migrator {
throw new Error(
`Cannot migrate backward, at ${current.at(-1)?.id ?? 0}, going to ${
target.id
}`
}`,
);
}
@@ -94,7 +93,7 @@ export class SqliteMigrator implements Migrator {
sql`
INSERT INTO "_migrations"("id", "name")
VALUES (${{ int: migration.id }}, ${{ text: migration.name }})
`
`,
);
this.#db.run(sql`COMMIT`);
} catch (e) {
+3 -3
View File
@@ -22,7 +22,7 @@ export function uuidToSqlite(uuid: string) {
}
export function sqliteToUuid(uuid: Buffer | Uint8Array) {
const hex = (uuid instanceof Buffer ? uuid : Buffer.from(uuid)).toString(
"hex"
"hex",
);
return [
hex.slice(0, 8),
@@ -126,7 +126,7 @@ export function sqliteToResident(resident: {
const defaultApikeyPermissions: ApikeyPermissions = {};
export function sqliteToApikeyPermissions(
permissions: string
permissions: string,
): ApikeyPermissions {
const parsed = JSON.parse(permissions);
const out = { ...defaultApikeyPermissions };
@@ -184,7 +184,7 @@ export function sqliteToNoteProperties(props: string): NoteProperties {
}
export function sqliteToPartialNoteProperties(
props: string
props: string,
): PartialNoteProperties {
const base = sqliteToNoteProperties(props);
return { type: base.type ?? "note" };
+1 -1
View File
@@ -3,7 +3,7 @@ import { sqliteProtocols } from "./url.js";
const getSqlite = () =>
import(/* webpackChunkName: 'dbsource-sqlite' */ "./getdb.static.js").then(
(x) => x.default
(x) => x.default,
);
const getSqliteDynamic: GetDbDynamic = {
+1 -1
View File
@@ -6,7 +6,7 @@ import type { WrappedDb, WrappedDbOptions } from "./types.js";
function getDatabase(
path: string,
options?: Omit<sqlite.Options, "nativeBinding">
options?: Omit<sqlite.Options, "nativeBinding">,
): sqlite.Database {
if (!natives.sqlite) throw new Error("No natives found for better-sqlite3");
options = { ...options };
+1 -1
View File
@@ -4,7 +4,7 @@ import { node, bs3 } from "./implementations.js";
export function getWrappedDb(
kind: "any" | "node" | "bs3",
path: string,
options: WrappedDbOptions
options: WrappedDbOptions,
): WrappedDb {
if (kind === "node") {
if (!node) throw new Error("Requesting unavailable node backend");
+13 -11
View File
@@ -29,7 +29,7 @@ const sqlClientUser = sql`
export function selectClientUser(
db: WrappedDb,
where: SqlCode
where: SqlCode,
): ClientUser | null {
const rawUser = db.get<RawClientUser>(sql`${sqlClientUser} WHERE ${where}`);
if (rawUser) return sqliteToClientUser(rawUser);
@@ -37,7 +37,7 @@ export function selectClientUser(
}
export function selectClientUsers(db: WrappedDb, rest?: SqlCode): ClientUser[] {
const rawUsers = db.all<RawClientUser>(
rest ? sql`${sqlClientUser} ${rest}` : sqlClientUser
rest ? sql`${sqlClientUser} ${rest}` : sqlClientUser,
);
return rawUsers.map(sqliteToClientUser);
}
@@ -62,7 +62,7 @@ export function selectAbode(db: WrappedDb, where: SqlCode): Abode | null {
}
export function selectAbodes(db: WrappedDb, rest?: SqlCode): Abode[] {
const rawAbodes = db.all<RawAbode>(
rest ? sql`${sqlAbode} ${rest}` : sqlAbode
rest ? sql`${sqlAbode} ${rest}` : sqlAbode,
);
return rawAbodes.map(sqliteToAbode);
}
@@ -88,7 +88,7 @@ export function selectResident(db: WrappedDb, where: SqlCode): Resident | null {
}
export function selectResidents(db: WrappedDb, where?: SqlCode): Resident[] {
const rawResidents = db.all<RawResident>(
where ? sql`${sqlResident} WHERE ${where}` : sqlResident
where ? sql`${sqlResident} WHERE ${where}` : sqlResident,
);
return rawResidents.map(sqliteToResident);
}
@@ -108,20 +108,20 @@ const sqlClientApikey = sql`
export function selectClientApikey(
db: WrappedDb,
where: SqlCode
where: SqlCode,
): ClientApikey | null {
const rawApikey = db.get<RawClientApikey>(
sql`${sqlClientApikey} WHERE ${where}`
sql`${sqlClientApikey} WHERE ${where}`,
);
if (rawApikey) return sqliteToClientApikey(rawApikey);
return null;
}
export function selectClientApikeys(
db: WrappedDb,
where: SqlCode
where?: SqlCode,
): ClientApikey[] {
const rawApikeys = db.all<RawClientApikey>(
sql`${sqlClientApikey} WHERE ${where}`
where ? sql`${sqlClientApikey} WHERE ${where}` : sqlClientApikey,
);
return rawApikeys.map(sqliteToClientApikey);
}
@@ -156,15 +156,17 @@ export function selectNote(db: WrappedDb, where: SqlCode): Note | null {
return null;
}
export function selectNotes(db: WrappedDb, where?: SqlCode): Note[] {
const raws = db.all<RawNote>(where ? sql`${sqlNote} WHERE ${where}` : sqlNote);
const raws = db.all<RawNote>(
where ? sql`${sqlNote} WHERE ${where}` : sqlNote,
);
return raws.map(sqliteToNote);
}
export function selectPartialNotes(
db: WrappedDb,
where?: SqlCode
where?: SqlCode,
): PartialNote[] {
const raws = db.all<RawPartialNote>(
where ? sql`${sqlPartialNote} WHERE ${where}` : sqlPartialNote
where ? sql`${sqlPartialNote} WHERE ${where}` : sqlPartialNote,
);
return raws.map(sqliteToPartialNote);
}
+1 -1
View File
@@ -65,7 +65,7 @@ export function calcUpdates<T extends object>(updater: {
for (const [prop, update] of Object.entries(updater)) {
if (prop in obj) {
updates.push(
(update as (value: unknown) => SqlCode)(obj[prop as keyof T]!)
(update as (value: unknown) => SqlCode)(obj[prop as keyof T]!),
);
}
}
+3 -3
View File
@@ -27,7 +27,7 @@ export function isSqliteUrl(url: string) {
}
export function parseSqliteUrl(
url: string
url: string,
): ["any" | "node" | "bs3", string, WrappedDbOptions] {
if (!isSqliteUrl(url)) throw new Error("Not sqlite: protocol");
const urlObj = new URL(url);
@@ -39,7 +39,7 @@ export function parseSqliteUrl(
urlObj.protocol === "node+sqlite:"
? "node"
: urlObj.protocol === "bs3+sqlite:"
? "bs3"
: "any";
? "bs3"
: "any";
return [kind, urlObj.pathname, options];
}
+3 -3
View File
@@ -40,11 +40,11 @@ export interface DbInterface {
deleteResidentById(uid: string, aid: string): Promise<void>;
createResident(
resident: CreateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident>;
updateResident(
resident: updateResident,
ctx: { uid: string }
ctx: { uid: string },
): Promise<Resident>;
// list residents by member
@@ -104,7 +104,7 @@ export function isBackendInterface(db: DbInterface): db is BackendDbInterface {
] as const
).every(
(x) =>
x in db && typeof (db as Partial<BackendDbInterface>)[x] === "function"
x in db && typeof (db as Partial<BackendDbInterface>)[x] === "function",
)
);
}
+110
View File
@@ -0,0 +1,110 @@
import type { DbInterface } from "./DbInterface.js";
/**
* The kinds of records that can travel through an export/import stream.
*
* `Session` is intentionally excluded: it has no CRUD/list surface in
* {@link DbInterface} and is ephemeral/non-portable between instances.
*/
export type ExportKind = "user" | "abode" | "resident" | "apikey" | "note";
/**
* Canonical order in which record kinds are emitted into an export stream, and
* the order in which an importer may safely apply them.
*
* This ordering is **part of the wire contract**, not an implementation
* detail. It is FK-safe: every foreign key points only at a kind that appears
* earlier (or at the same kind, earlier in the stream), so an importer that
* inserts records one-by-one with referential integrity enforced never
* forward-references a row it hasn't inserted yet:
*
* - `abode.created_by`/`updated_by` → `user`
* - `resident.uid` → `user`, `resident.aid` → `abode`
* - `apikey.uid` → `user`
* - `note.aid` → `abode`, `note.created_by`/`updated_by` → `user`
*
* The sqlite backend can afford to relax this (it suspends FK enforcement for
* the load), but the postgres backend relies on it: it inserts sequentially
* with constraints live. Every {@link Exportable} MUST emit in this order, and
* reordering it is a breaking change to the format.
*/
export const EXPORT_KIND_ORDER = [
"user",
"abode",
"resident",
"apikey",
"note",
] as const satisfies readonly ExportKind[];
export type ExportFilter = {
/** Include only these kinds; omit = all kinds. */
kinds?: ExportKind[];
/** Excluded after `kinds` is applied. */
excludeKinds?: ExportKind[];
/** aid allowlist — scopes abode/resident/note. */
abodes?: string[];
/** uid allowlist — scopes user (and apikey, unless `apikeys` is set). */
users?: string[];
/**
* uid allowlist scoping apikey records specifically. When set it takes
* precedence over `users` for the `apikey` kind — used to export a
* non-admin's own keys while still exporting co-residents' *user* records
* for referential integrity, without leaking their apikey metadata. Absent =
* fall back to `users`.
*/
apikeys?: string[];
};
export interface ExportOptions {
filter?: ExportFilter;
signal?: AbortSignal;
}
export interface Exportable {
/** Produce a stream of NDJSON lines (one JSON envelope per line). */
export(options?: ExportOptions): NodeJS.ReadableStream;
}
export interface ImportOptions {
filter?: ExportFilter;
signal?: AbortSignal;
}
export interface ImportResult {
counts: Partial<Record<ExportKind, number>>;
}
export interface Importable {
import(
source: NodeJS.ReadableStream,
options?: ImportOptions,
): Promise<ImportResult>;
}
/**
* The NDJSON envelope written/read for every line. The leading line is a
* `meta` record; the record lines that follow are grouped by kind in
* {@link EXPORT_KIND_ORDER} (an FK-safe order importers may rely on); a
* trailing `error` record may appear if the source failed after streaming had
* already begun.
*/
export type ExportEnvelope =
| { kind: "meta"; data: ExportMeta }
| { kind: ExportKind; data: unknown }
| { kind: "error"; data: { message: string; code?: string } };
export type ExportMeta = {
v: number;
exportedAt: string;
source: string;
/** The *effective* filter actually applied (may be narrower than requested). */
filter: ExportFilter;
};
export function isExportable(db: DbInterface): db is DbInterface & Exportable {
return typeof (db as Partial<Exportable>).export === "function";
}
export function isImportable(db: DbInterface): db is DbInterface & Importable {
return typeof (db as Partial<Importable>).import === "function";
}
+1 -1
View File
@@ -27,7 +27,7 @@ export type LoginUser = {
};
export function isValidUserPassword(
password: string
password: string,
): password is User["password"] {
if (password.startsWith("#")) {
return ["unset"].includes(password.slice(1));
+1 -1
View File
@@ -2,7 +2,7 @@ if (import.meta.hot) {
import.meta.hot.on("message", (msg) => {
if (
msg.includes(
"A pending update was not accepted, and reached the root module:"
"A pending update was not accepted, and reached the root module:",
)
) {
throw new Error("[hot] Restarting due to unaccepted pending update");
+4 -4
View File
@@ -32,23 +32,23 @@ async function getPackageJsonDir(path: string): Promise<string | null> {
export async function findNative(
module: string,
native: string
native: string,
): Promise<string> {
const path = await getPackageJsonDir(
createRequire(import.meta.url).resolve(module)
createRequire(import.meta.url).resolve(module),
);
if (!path) throw new Error(`Cannot find module directory for ${module}`);
const file = await find(path, native);
if (!file)
throw new Error(
`Cannot find native ${native} of package ${module} in ${path}`
`Cannot find native ${native} of package ${module} in ${path}`,
);
return file;
}
export async function tryFindNative(
module: string,
native: string
native: string,
): Promise<string | null> {
try {
return await findNative(module, native);
+2 -2
View File
@@ -22,8 +22,8 @@ export async function webpack(): Promise<{ code: string }> {
let code: string = standaloneCode(
validator,
Object.fromEntries(
Object.entries(schemas).map(([id, schema]) => [id, schema.$id])
)
Object.entries(schemas).map(([id, schema]) => [id, schema.$id]),
),
);
// assign the .schema ourselves to the validation functions
+4 -4
View File
@@ -12,7 +12,7 @@ export interface PopupManagerContextData {
openPopup(popup: ComponentType<{ id: string; onClose: () => void }>): string;
openPopup<T>(
popup: ComponentType<{ id: string; onClose: () => void } & T>,
props: T
props: T,
): string;
closePopup(id: string): void;
@@ -34,7 +34,7 @@ export function PopupManager({ children }: { children: ReactNode }) {
const openPopup = useCallback<PopupManagerContextData["openPopup"]>(
(
Component: ComponentType<{ id: string; onClose: () => void }>,
props = {}
props = {},
) => {
const id = crypto.randomUUID();
Object.assign(props, {
@@ -45,14 +45,14 @@ export function PopupManager({ children }: { children: ReactNode }) {
setPopups((prev) => [...prev, { id, Component, props }]);
return id;
},
[]
[],
);
const closePopup = useCallback((id: string) => {
setPopups((prev) => prev.filter((x) => x.id !== id));
}, []);
const ctx = useMemo<PopupManagerContextData>(
() => ({ openPopup, closePopup }),
[]
[],
);
return (
+2 -2
View File
@@ -19,7 +19,7 @@ export function useDataResidentsByAbodeId(aid: string) {
const status = useLoad(loadResidentsByAbodeId, { aid });
const residents = useMemo(
() => Object.values(allResidents).filter((x) => x.aid === aid),
[allResidents, aid]
[allResidents, aid],
);
return { ...status, residents };
}
@@ -29,7 +29,7 @@ export function useDataResidentsByUserId(uid: string) {
const status = useLoad(loadResidentsByUserId, { uid });
const residents = useMemo(
() => Object.values(allResidents).filter((x) => x.uid === uid),
[allResidents, uid]
[allResidents, uid],
);
return { ...status, residents };
}
+2 -2
View File
@@ -5,7 +5,7 @@ import type { Store } from "../store/store.js";
import { useStore } from "../store/react.js";
export function useAction<P extends any[], R>(
action: (...params: [...P, { db: DbInterface; store: Store }]) => Promise<R>
action: (...params: [...P, { db: DbInterface; store: Store }]) => Promise<R>,
): (...args: P) => Promise<R> {
const db = use(DbContext);
const store = useStore();
@@ -15,6 +15,6 @@ export function useAction<P extends any[], R>(
if (!db) throw new Error("DB not present");
return action(...params, { db, store });
},
[action, db]
[action, db],
);
}
+1 -1
View File
@@ -32,7 +32,7 @@ export function useLoad<P>(loader: Loader<P>, params?: P): UseLoadResult {
const refresh = useCallback(
() => load({ loader, params: params!, store, db, refresh: true }),
[loader, params, db]
[loader, params, db],
);
return {
+3 -3
View File
@@ -8,7 +8,7 @@ import type { Store } from "../store.js";
export async function deleteAbodeById(
aid: string,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<void> {
await Promise.all([
waitForLoadIfLoading(store, "loadAllAbodes"),
@@ -22,7 +22,7 @@ export async function deleteAbodeById(
export async function updateAbode(
abode: UpdateAbode,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<void> {
await Promise.all([
waitForLoadIfLoading(store, "loadAllAbodes"),
@@ -38,7 +38,7 @@ export async function updateAbode(
export async function createAbode(
abode: CreateAbode,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<string> {
await waitForLoadIfLoading(store, "loadAllAbodes");
+3 -3
View File
@@ -7,7 +7,7 @@ import type { Store } from "../store.js";
export async function deleteUserById(
uid: string,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<void> {
await Promise.all([
waitForLoadIfLoading(store, "loadAllUsers"),
@@ -24,7 +24,7 @@ export async function deleteUserById(
export async function updateUser(
user: UpdateUser,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<void> {
await Promise.all([
waitForLoadIfLoading(store, "loadAllUsers"),
@@ -41,7 +41,7 @@ export async function updateUser(
export async function createUser(
user: CreateUser,
{ store, db }: { store: Store; db: DbInterface }
{ store, db }: { store: Store; db: DbInterface },
): Promise<string> {
await waitForLoadIfLoading(store, "loadAllUsers");
+7 -4
View File
@@ -48,7 +48,7 @@ async function loadImpl<P>({
setLoading([
id,
{ status: refresh ? "refreshing" : "loading", type, params },
])
]),
);
const controller = new AbortController();
try {
@@ -69,7 +69,10 @@ async function loadImpl<P>({
store.dispatch(setLoading([id, { status: "loaded", type, params }]));
} catch (e) {
store.dispatch(
setLoading([id, { status: "error", type, params, error: objectError(e) }])
setLoading([
id,
{ status: "error", type, params, error: objectError(e) },
]),
);
throw e;
}
@@ -114,7 +117,7 @@ export function loader<P>(loader: Loader<P>): Loader<P> {
export async function waitForLoadIfLoading(
store: Store,
id: string,
{ signal }: { signal?: AbortSignal } = {}
{ signal }: { signal?: AbortSignal } = {},
) {
if (!getLoadingStatus(store.getState(), id)) return;
return waitFor(
@@ -123,6 +126,6 @@ export async function waitForLoadIfLoading(
const status = getLoadingStatus(state, id);
return status === "loaded" || status === "error";
},
{ signal }
{ signal },
);
}
+1 -1
View File
@@ -16,7 +16,7 @@ export function Provider(
props: Omit<ProviderProps, "context" | "store" | "serverState"> & {
store: Store;
serverState?: State;
}
},
) {
return <RawProvider context={AbodeStoreContext} {...props} />;
}
+1 -1
View File
@@ -40,7 +40,7 @@ const loadingSlice = createSlice({
reducers: {
setLoading: (
state,
action: PayloadAction<[id: string, state: LoadingState]>
action: PayloadAction<[id: string, state: LoadingState]>,
) => {
state[action.payload[0]] = action.payload[1];
},
+1 -1
View File
@@ -19,7 +19,7 @@ const usersSlice = createSlice({
getUser: usersSelectors.selectById,
getUserByEmail: (state, email: string) =>
Object.values(state.entities).find(
(x): x is ClientUser => "email" in x && x.email === email
(x): x is ClientUser => "email" in x && x.email === email,
),
getUsers: usersSelectors.selectEntities,
getUserIds: usersSelectors.selectIds,
+2 -2
View File
@@ -3,7 +3,7 @@ import type { State, Store } from "./store.js";
export async function waitFor(
store: Store,
cond: (state: State) => boolean,
{ signal }: { signal?: AbortSignal } = {}
{ signal }: { signal?: AbortSignal } = {},
): Promise<void> {
return new Promise<void>((ok, ko) => {
signal?.throwIfAborted();
@@ -28,7 +28,7 @@ export async function waitFor(
() => {
controller.abort();
},
{ signal: controller.signal }
{ signal: controller.signal },
);
});
}
+19 -19
View File
@@ -1,23 +1,23 @@
export { default as user } from "./user/user.schema.json" with {type: 'json'};
export { default as createuser } from "./user/createuser.schema.json" with {type: 'json'};
export { default as updateuser } from "./user/updateuser.schema.json" with {type: 'json'};
export { default as partialuser } from "./user/partialuser.schema.json" with {type: 'json'};
export { default as clientuser } from "./user/clientuser.schema.json" with {type: 'json'};
export { default as userflags } from "./user/userflags.schema.json" with {type: 'json'};
export { default as loginuser } from "./user/loginuser.schema.json" with {type: 'json'};
export { default as user } from "./user/user.schema.json" with { type: "json" };
export { default as createuser } from "./user/createuser.schema.json" with { type: "json" };
export { default as updateuser } from "./user/updateuser.schema.json" with { type: "json" };
export { default as partialuser } from "./user/partialuser.schema.json" with { type: "json" };
export { default as clientuser } from "./user/clientuser.schema.json" with { type: "json" };
export { default as userflags } from "./user/userflags.schema.json" with { type: "json" };
export { default as loginuser } from "./user/loginuser.schema.json" with { type: "json" };
export { default as abode } from './abode/abode.schema.json' with {type: 'json'};
export { default as createabode } from './abode/createabode.schema.json' with {type: 'json'};
export { default as updateabode } from './abode/updateabode.schema.json' with {type: 'json'};
export { default as abode } from "./abode/abode.schema.json" with { type: "json" };
export { default as createabode } from "./abode/createabode.schema.json" with { type: "json" };
export { default as updateabode } from "./abode/updateabode.schema.json" with { type: "json" };
export { default as resident } from './resident/resident.schema.json' with {type: 'json'};
export { default as createresident } from './resident/createresident.schema.json' with {type: 'json'};
export { default as updateresident } from './resident/updateresident.schema.json' with {type: 'json'};
export { default as residentflags } from './resident/residentflags.schema.json' with {type: 'json'};
export { default as resident } from "./resident/resident.schema.json" with { type: "json" };
export { default as createresident } from "./resident/createresident.schema.json" with { type: "json" };
export { default as updateresident } from "./resident/updateresident.schema.json" with { type: "json" };
export { default as residentflags } from "./resident/residentflags.schema.json" with { type: "json" };
export { default as createapikey } from './apikey/createapikey.schema.json' with {type: 'json'};
export { default as apikeypermissions } from './apikey/apikeypermissions.schema.json' with {type: 'json'};
export { default as createapikey } from "./apikey/createapikey.schema.json" with { type: "json" };
export { default as apikeypermissions } from "./apikey/apikeypermissions.schema.json" with { type: "json" };
export { default as createnote } from './note/createnote.schema.json' with {type: 'json'};
export { default as updatenote } from './note/updatenote.schema.json' with {type: 'json'};
export { default as partialnoteproperties } from './note/partialnoteproperties.schema.json' with {type: 'json'};
export { default as createnote } from "./note/createnote.schema.json" with { type: "json" };
export { default as updatenote } from "./note/updatenote.schema.json" with { type: "json" };
export { default as partialnoteproperties } from "./note/partialnoteproperties.schema.json" with { type: "json" };
+1 -1
View File
@@ -39,7 +39,7 @@ function checkSchema(name: string, schema: AnySchema) {
if (!schema.$id) throw new Error(`Missing $id for schema ${name}`);
if (
!schema.$id.match(
/^https:\/\/abode\.codi\.moe\/schema\/[a-zA-Z0_9_-]+\.schema\.json$/
/^https:\/\/abode\.codi\.moe\/schema\/[a-zA-Z0_9_-]+\.schema\.json$/,
)
)
throw new Error(`Unexpected $id for schema ${name}`);
+1 -1
View File
@@ -18,7 +18,7 @@ const validators = Object.fromEntries(
Object.entries(schemas).map(([name, schema]) => [
name,
validator.compile(schema),
])
]),
) as unknown as {
[T in keyof Types]: {
(obj: unknown): obj is Types[T];
+1 -1
View File
@@ -69,7 +69,7 @@ function App() {
(input, key) => {
if (input === "q" || key.escape) app.exit();
},
{ isActive }
{ isActive },
);
const [activeCollection, setActiveCollection] =
+2 -2
View File
@@ -41,11 +41,11 @@ export function AbodesPanel() {
const onSelect = useCallback(
(abode: Abode) => openPopup(AbodePopup, { aid: abode.aid }),
[openPopup]
[openPopup],
);
const buttons = useMemo<ButtonListItem[]>(
() => [{ children: "New", onClick: () => openPopup(CreateAbodePopup) }],
[openPopup]
[openPopup],
);
return (
+2 -2
View File
@@ -59,11 +59,11 @@ export function UsersPanel() {
const onSelect = useCallback(
(user: ClientUser | PartialUser) => openPopup(UserPopup, { uid: user.uid }),
[openPopup]
[openPopup],
);
const buttons = useMemo<ButtonListItem[]>(
() => [{ children: "New", onClick: () => openPopup(CreateUserPopup) }],
[openPopup]
[openPopup],
);
return (
+2 -2
View File
@@ -18,7 +18,7 @@ export function Button({
(input, key) => {
if (input === " " || key.return) onClick();
},
{ isActive: isFocused }
{ isActive: isFocused },
);
return <Text inverse={isFocused}>[{children}]</Text>;
@@ -66,7 +66,7 @@ export function ButtonList({
setSelected((prev) => (prev - 1 + buttons.length) % buttons.length);
}
},
{ isActive: isFocused || forceFocus || false }
{ isActive: isFocused || forceFocus || false },
);
return (
+2 -2
View File
@@ -49,7 +49,7 @@ export function ListBox<T extends string>({
setSelected(items[(items.indexOf(selected) + 1) % items.length]);
} else if (key.upArrow) {
setSelected(
items[(items.indexOf(selected) - 1 + items.length) % items.length]
items[(items.indexOf(selected) - 1 + items.length) % items.length],
);
} else if (key.pageUp) {
setSelected(items[0]);
@@ -57,7 +57,7 @@ export function ListBox<T extends string>({
setSelected(items[items.length - 1]);
}
},
{ isActive: isFocused }
{ isActive: isFocused },
);
return (
+3 -3
View File
@@ -36,14 +36,14 @@ export function ListDisplay<T>({
onSelect?.(items[selected]);
}
},
{ isActive: isFocused }
{ isActive: isFocused },
);
useEffect(() => {
if (selected < start) setStart(selected);
else if (selected >= slice)
setStart(
Math.max(Math.min(selected - slice + start + 1, items.length - 1), 0)
Math.max(Math.min(selected - slice + start + 1, items.length - 1), 0),
);
}, [selected, start, slice, items.length]);
useEffect(() => {
@@ -56,7 +56,7 @@ export function ListDisplay<T>({
const indexed = useMemo(
() => items.map((item, index) => ({ item, index })),
[items]
[items],
);
return (
+1 -1
View File
@@ -16,7 +16,7 @@ export function Popup({
(_, key) => {
if (key.escape) onClose?.();
},
{ isActive: active && !!onClose }
{ isActive: active && !!onClose },
);
return (
+1 -1
View File
@@ -52,7 +52,7 @@ export function SearchPanel<T>({
refresh?.();
}
},
{ isActive: isFocused && !!refresh }
{ isActive: isFocused && !!refresh },
);
const topbar = !!match || !!buttons?.length;
+2 -2
View File
@@ -2,13 +2,13 @@ import { argon2id, argon2Verify } from "hash-wasm";
export async function validatePassword(
password: string,
hash: string
hash: string,
): Promise<boolean> {
return await argon2Verify({ password, hash });
}
export async function hashPassword(
password: string
password: string,
): Promise<`$${string}$${string}`> {
const salt = new Uint8Array(16);
crypto.getRandomValues(salt);
+12 -14
View File
@@ -7,11 +7,11 @@ const escapes = {
};
function parseAdd<T>(
rest: (Record<string, string> | string | ((writer: XmlWriter) => T))[]
rest: (Record<string, string> | string | ((writer: XmlWriter) => T))[],
): [
props?: Record<string, string>,
content?: string,
children?: (writer: XmlWriter) => T
children?: (writer: XmlWriter) => T,
] {
let props: Record<string, string> | undefined;
let children: ((writer: XmlWriter) => T) | undefined;
@@ -55,7 +55,7 @@ export class XmlWriter {
tag: string,
props?: Record<string, string>,
content?: string,
children?: NonNullable<unknown>
children?: NonNullable<unknown>,
) {
const top = this.#stack.at(-1);
if (top && !top.children) {
@@ -104,7 +104,7 @@ export class XmlWriter {
add(
tag: string,
props: Record<string, string>,
children: (writer: XmlWriter) => void
children: (writer: XmlWriter) => void,
): XmlWriter;
add(
tag: string,
@@ -125,23 +125,21 @@ export class XmlWriter {
addAsync(
tag: string,
props: Record<string, string>,
content: string
content: string,
): Promise<void>;
addAsync(
tag: string,
children: (writer: XmlWriter) => Promise<void>
children: (writer: XmlWriter) => Promise<void>,
): Promise<void>;
addAsync(
tag: string,
props: Record<string, string>,
children: (writer: XmlWriter) => Promise<void>
children: (writer: XmlWriter) => Promise<void>,
): Promise<void>;
async addAsync(
tag: string,
...rest: (
| Record<string, string>
| string
| ((writer: XmlWriter) => Promise<void>)
Record<string, string> | string | ((writer: XmlWriter) => Promise<void>)
)[]
): Promise<void> {
const [props, content, children] = parseAdd(rest);
@@ -164,7 +162,7 @@ export class XmlWriter {
| Parameters<InstanceType<typeof XmlWriter>["add"]>
| [
NonNullable<ConstructorParameters<typeof XmlWriter>[0]>,
...Parameters<InstanceType<typeof XmlWriter>["add"]>
...Parameters<InstanceType<typeof XmlWriter>["add"]>,
]
): string {
let options: ConstructorParameters<typeof XmlWriter>[0];
@@ -172,7 +170,7 @@ export class XmlWriter {
options = rest.shift()! as ConstructorParameters<typeof XmlWriter>[0];
}
return new XmlWriter(options).add(
...(rest as Parameters<InstanceType<typeof XmlWriter>["add"]>)
...(rest as Parameters<InstanceType<typeof XmlWriter>["add"]>),
).content;
}
@@ -188,7 +186,7 @@ export class XmlWriter {
| Parameters<InstanceType<typeof XmlWriter>["addAsync"]>
| [
NonNullable<ConstructorParameters<typeof XmlWriter>[0]>,
...Parameters<InstanceType<typeof XmlWriter>["addAsync"]>
...Parameters<InstanceType<typeof XmlWriter>["addAsync"]>,
]
): Promise<string> {
let options: ConstructorParameters<typeof XmlWriter>[0];
@@ -197,7 +195,7 @@ export class XmlWriter {
}
const writer = new XmlWriter(options);
await writer.addAsync(
...(rest as Parameters<InstanceType<typeof XmlWriter>["addAsync"]>)
...(rest as Parameters<InstanceType<typeof XmlWriter>["addAsync"]>),
);
return writer.content;
}
+100 -11
View File
@@ -15,7 +15,39 @@ import {
updateuser,
} from "../schema/validators.js";
import { authenticate } from "./middleware/authenticate.js";
import { InvalidAbodeError, NotFoundAbodeError } from "../db/types/errors.js";
import {
InvalidAbodeError,
NotAuthorizedAbodeError,
NotFoundAbodeError,
} from "../db/types/errors.js";
import { isExportable } from "../db/types/ExportImport.js";
import type { ExportFilter, ExportKind } from "../db/types/ExportImport.js";
import { intersectExportFilters, isExportKind } from "../db/export/filter.js";
import { computeForcedExportFilter } from "./exportScope.js";
import {
hasGlobalUserVisibility,
hideUserEmail,
userForCaller,
} from "./userVisibility.js";
function parseExportFilter(query: Record<string, unknown>): ExportFilter {
const list = (v: unknown): string[] | undefined => {
if (typeof v !== "string" || !v) return undefined;
return v.split(",").filter(Boolean);
};
const kinds = (v: unknown): ExportKind[] | undefined =>
list(v)?.filter(isExportKind);
const filter: ExportFilter = {};
const k = kinds(query.kinds);
if (k) filter.kinds = k;
const ek = kinds(query.excludeKinds);
if (ek) filter.excludeKinds = ek;
const abodes = list(query.abodes);
if (abodes) filter.abodes = abodes;
const users = list(query.users);
if (users) filter.users = users;
return filter;
}
export function apirouter(db: BackendDbInterface): KoaRouter {
const router = new KoaRouter();
@@ -42,26 +74,66 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
ctx.status = 204;
});
router.get("/export", authenticate(db), async (ctx) => {
// Everything that can throw a domain error runs *before* any byte is
// written, so `convertError` still applies. Once `ctx.body` is a stream,
// a mid-stream failure surfaces as a trailing `error` NDJSON line instead.
const forced = await computeForcedExportFilter(db, {
user: ctx.user!,
session: ctx.session!,
});
const effective = intersectExportFilters(
parseExportFilter(ctx.query),
forced,
);
if (!isExportable(db)) {
ctx.status = 501;
ctx.body = { ok: false, error: "export_unsupported" };
return;
}
const ac = new AbortController();
ctx.res.on("close", () => {
if (!ctx.res.writableEnded) ac.abort();
});
ctx.type = "application/x-ndjson";
ctx.body = db.export({ filter: effective, signal: ac.signal });
});
router.use("/users", authenticate(db));
router.get("/users", async (ctx) => {
ctx.body = await db.listUsers();
const users = await db.listUsers();
ctx.body = users.map((user) =>
userForCaller(user, { user: ctx.user!, session: ctx.session! }),
);
});
router.post("/users", jsonBody({ validate: createuser }), async (ctx) => {
ctx.body = await db.createUser(ctx.request.body);
});
router.get("/users/by-email", async (ctx) => {
if (typeof ctx.query.email !== "string") throw new InvalidAbodeError();
if (
!hasGlobalUserVisibility({
user: ctx.user!,
session: ctx.session!,
})
) {
throw new NotAuthorizedAbodeError();
}
ctx.body = await db.getUserByEmail(ctx.query.email);
});
router.get("/users/:uid", async (ctx) => {
ctx.body = await db.getUserById(ctx.params.uid);
const user = await db.getUserById(ctx.params.uid);
ctx.body = userForCaller(user, {
user: ctx.user!,
session: ctx.session!,
});
});
router.patch(
"/users/:uid",
jsonBody({ validate: updateuser, includeParams: ["uid"] }),
async (ctx) => {
ctx.body = await db.updateUser(ctx.request.body);
}
},
);
router.delete("/users/:uid", async (ctx) => {
await db.deleteUserById(ctx.params.uid);
@@ -82,7 +154,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
async (ctx) => {
const [apikey, token] = await db.createApikey(ctx.request.body);
ctx.body = { apikey, token };
}
},
);
router.get("/users/:uid/apikeys/:kid", async (ctx) => {
const apikey = await db.getApikeyById(ctx.params.kid);
@@ -118,7 +190,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
jsonBody({ validate: updateabode, includeParams: ["aid"] }),
async (ctx) => {
ctx.body = await db.updateAbode(ctx.request.body, { uid: ctx.user!.uid });
}
},
);
router.delete("/abodes/:aid", async (ctx) => {
await db.deleteAbodeById(ctx.params.aid);
@@ -128,7 +200,24 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
ctx.body = await db.listResidentsByAbodeId(ctx.params.aid);
});
router.get("/abodes/:aid/users", async (ctx) => {
ctx.body = await db.listUsersByAbodeId(ctx.params.aid);
const users = await db.listUsersByAbodeId(ctx.params.aid);
const globalVisibility = hasGlobalUserVisibility({
user: ctx.user!,
session: ctx.session!,
});
const residents = globalVisibility
? []
: await db.listResidentsByAbodeId(ctx.params.aid);
const abodeAdmin = residents.some(
(resident) =>
resident.uid === ctx.user!.uid && resident.flags.admin === true,
);
ctx.body =
globalVisibility || abodeAdmin
? users
: users.map((user) =>
user.uid === ctx.user!.uid ? user : hideUserEmail(user),
);
});
router.get("/abodes/:aid/notes", async (ctx) => {
ctx.body = await db.listNotesByAbodeId(ctx.params.aid);
@@ -138,7 +227,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
jsonBody({ validate: createnote, includeParams: ["aid"] }),
async (ctx) => {
ctx.body = await db.createNote(ctx.request.body, { uid: ctx.user!.uid });
}
},
);
router.use("/residents", authenticate(db));
@@ -152,7 +241,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
ctx.body = await db.createResident(ctx.request.body, {
uid: ctx.user!.uid,
});
}
},
);
router.get("/residents/:uid/:aid", async (ctx) => {
ctx.body = await db.getResidentById(ctx.params.uid, ctx.params.aid);
@@ -164,7 +253,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
ctx.body = await db.updateResident(ctx.request.body, {
uid: ctx.user!.uid,
});
}
},
);
router.delete("/residents/:uid/:aid", async (ctx) => {
await db.deleteResidentById(ctx.params.uid, ctx.params.aid);
@@ -192,7 +281,7 @@ export function apirouter(db: BackendDbInterface): KoaRouter {
jsonBody({ validate: updatenote, includeParams: ["nid"] }),
async (ctx) => {
ctx.body = await db.updateNote(ctx.request.body, { uid: ctx.user!.uid });
}
},
);
router.delete("/notes/:nid", async (ctx) => {
await db.deleteNoteById(ctx.params.nid);
+64
View File
@@ -0,0 +1,64 @@
import type { Context } from "koa";
import type { BackendDbInterface } from "../db/types/DbInterface.js";
import type { ClientUser } from "../db/types/User.js";
import type { ExportFilter } from "../db/types/ExportImport.js";
/**
* Compute the export scope that must be *forced* on a caller, independent of
* anything they requested. Returns `null` when the caller is unrestricted (a
* global admin whose credential imposes no narrowing) — their own filter, if
* any, is then honored verbatim as a voluntary narrowing.
*
* Otherwise returns `{ abodes, users, apikeys }`: the abodes the caller resides
* in, the users needed to keep that data referentially whole (the caller plus
* every co-resident of those abodes), and — scoped tighter than `users` —
* apikeys limited to the caller alone, so a non-admin never exports another
* user's apikey metadata even though that user's record is included. This is
* the maximum a non-admin may export; the route intersects it with any
* caller-supplied filter (never a union).
*/
export async function computeForcedExportFilter(
db: BackendDbInterface,
ctx: { user: ClientUser; session: NonNullable<Context["session"]> },
): Promise<ExportFilter | null> {
const { user, session } = ctx;
if (user.flags.admin) {
if (session.source !== "apikey") return null;
const p = session.key.permissions;
const unrestricted =
!!p.admin &&
!!p.all &&
!p.restrict_users?.length &&
!p.restrict_abodes?.length;
if (unrestricted) return null;
}
const residencies = await db.listResidentsByUserId(user.uid);
const abodeSet = new Set(residencies.map((r) => r.aid));
const userSet = new Set<string>([user.uid]);
for (const aid of abodeSet) {
for (const u of await db.listUsersByAbodeId(aid)) userSet.add(u.uid);
}
let abodes = [...abodeSet];
let users = [...userSet];
// apikeys are self-only for non-admins, regardless of co-residency.
let apikeys = [user.uid];
// An apikey can only narrow what its owning user could otherwise export.
if (session.source === "apikey") {
const p = session.key.permissions;
if (p.restrict_abodes?.length) {
const allow = new Set(p.restrict_abodes);
abodes = abodes.filter((a) => allow.has(a));
}
if (p.restrict_users?.length) {
const allow = new Set(p.restrict_users);
users = users.filter((u) => allow.has(u));
apikeys = apikeys.filter((u) => allow.has(u));
}
}
return { abodes, users, apikeys };
}
+1 -1
View File
@@ -16,7 +16,7 @@ export function schemarouter(): KoaRouter {
url: `${ctx.URL}/${name}.schema.json`,
},
];
})
}),
);
});
+36
View File
@@ -0,0 +1,36 @@
import type { Context } from "koa";
import type { ClientUser, PartialUser } from "../db/types/User.js";
type AuthContext = {
user: ClientUser;
session: NonNullable<Context["session"]>;
};
export function hasGlobalUserVisibility(ctx: AuthContext): boolean {
if (!ctx.user.flags.admin) return false;
if (ctx.session.source !== "apikey") return true;
const permissions = ctx.session.key.permissions;
return (
!!permissions.admin &&
!!permissions.all &&
!permissions.restrict_users?.length &&
!permissions.restrict_abodes?.length
);
}
export function hideUserEmail(user: PartialUser | ClientUser): PartialUser {
if (!("email" in user)) return user;
const { email: _email, ...partial } = user;
return partial;
}
export function userForCaller(
user: PartialUser | ClientUser,
ctx: AuthContext,
): PartialUser | ClientUser {
if (user.uid === ctx.user.uid || hasGlobalUserVisibility(ctx)) {
return user;
}
return hideUserEmail(user);
}
+19 -11
View File
@@ -3,7 +3,11 @@ import assert from "node:assert/strict";
import { createServer } from "node:http";
import type { AddressInfo } from "node:net";
import { ApiInterface } from "../../../src/db/api/ApiInterface.js";
import { apiProtocols, isApiUrl, parseApiUrl } from "../../../src/db/api/url.js";
import {
apiProtocols,
isApiUrl,
parseApiUrl,
} from "../../../src/db/api/url.js";
import {
NotFoundAbodeError,
NotAuthorizedAbodeError,
@@ -86,7 +90,7 @@ describe("parseApiUrl", () => {
it("extra query params become headers", () => {
const [, { headers }] = parseApiUrl(
"http://example.com?X-Custom-Header=value"
"http://example.com?X-Custom-Header=value",
);
assert.equal(headers["X-Custom-Header"], "value");
});
@@ -94,7 +98,7 @@ describe("parseApiUrl", () => {
it("throws for non-api protocol", () => {
assert.throws(
() => parseApiUrl("sqlite:///db.sqlite"),
/Not an \{abode\+,\}http\{s,\}: protocol/
/Not an \{abode\+,\}http\{s,\}: protocol/,
);
});
});
@@ -106,18 +110,22 @@ describe("ApiInterface HTTP error mapping", () => {
before(async () => {
let nextStatus = 500;
respondWith = (s) => { nextStatus = s; };
respondWith = (s) => {
nextStatus = s;
};
const server = createServer((req, res) => {
res.writeHead(nextStatus, { "Content-Type": "application/json" });
res.end(JSON.stringify({ ok: false, error: "test" }));
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
await new Promise<void>((resolve) =>
server.listen(0, "127.0.0.1", resolve),
);
const { port } = server.address() as AddressInfo;
serverUrl = `http://127.0.0.1:${port}`;
closeServer = () =>
new Promise<void>((resolve, reject) =>
server.close((err) => (err ? reject(err) : resolve()))
server.close((err) => (err ? reject(err) : resolve())),
);
});
@@ -131,7 +139,7 @@ describe("ApiInterface HTTP error mapping", () => {
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -143,7 +151,7 @@ describe("ApiInterface HTTP error mapping", () => {
(err) => {
assert.ok(err instanceof NotAuthorizedAbodeError);
return true;
}
},
);
});
@@ -155,7 +163,7 @@ describe("ApiInterface HTTP error mapping", () => {
(err) => {
assert.ok(err instanceof ReadonlyAbodeError);
return true;
}
},
);
});
@@ -167,7 +175,7 @@ describe("ApiInterface HTTP error mapping", () => {
(err) => {
assert.ok(err instanceof InvalidAbodeError);
return true;
}
},
);
});
@@ -179,7 +187,7 @@ describe("ApiInterface HTTP error mapping", () => {
(err) => {
assert.ok(err instanceof ConflictAbodeError);
return true;
}
},
);
});
});
+5 -1
View File
@@ -92,7 +92,11 @@ describe("api backend: auth over HTTP", async () => {
const self = await fetch(`${server.url}/auth/self`, {
headers: { Cookie: `abode_session=${cookie}` },
});
assert.equal(self.status, 401, "session was invalidated server-side, not just the cookie cleared");
assert.equal(
self.status,
401,
"session was invalidated server-side, not just the cookie cleared",
);
});
it("POST /auth/clear-sessions invalidates outstanding session cookies", async () => {
+3 -1
View File
@@ -17,7 +17,9 @@ async function getApiDb() {
email: AUTH_EMAIL,
name: "API Auth User",
password: pw,
flags: {},
// The shared backend contract suite exercises unrestricted user lookup,
// which the HTTP API now reserves for global administrators.
flags: { admin: true },
});
const server = await createTestServer(sqliteDb);
const authHeader = "Basic " + btoa(`${AUTH_EMAIL}:${AUTH_PASSWORD}`);
@@ -0,0 +1,328 @@
import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { Readable } from "node:stream";
import { PostgresInterface } from "../../../src/db/postgres/PostgresInterface.js";
import type { WrappedPgClient } from "../../../src/db/postgres/pool.js";
import type { SqlCode } from "../../../src/db/postgres/sql.js";
import {
EXPORT_KIND_ORDER,
type ExportKind,
} from "../../../src/db/types/ExportImport.js";
// The postgres backend has no CI database, so these tests drive the real
// PostgresInterface.export/import code paths against an in-memory fake client.
// They verify control flow (NDJSON shape, meta.source, filtering, note
// skipping, counts, insert dispatch, abort -> rollback); the SQL-arg forms are
// the same {uuid}/{text}/{jsonb}/{date} patterns the pg backend's own CRUD
// already exercises against real Postgres.
interface Rows {
users?: Record<string, unknown>[];
abodes?: Record<string, unknown>[];
residents?: Record<string, unknown>[];
apikeys?: Record<string, unknown>[];
notes?: Record<string, unknown>[];
}
class FakePg implements WrappedPgClient {
readonly = false;
inserts: { table: string; vars: unknown[] }[] = [];
committed = false;
rolledBack = false;
#rows: Rows;
constructor(rows: Rows = {}) {
this.#rows = rows;
}
async destroy(): Promise<void> {}
async all<R>(stmt: SqlCode): Promise<R[]> {
const s = stmt._sql;
if (s.includes('FROM "users"')) return (this.#rows.users ?? []) as R[];
if (s.includes('FROM "abodes"')) return (this.#rows.abodes ?? []) as R[];
if (s.includes('FROM "residents"'))
return (this.#rows.residents ?? []) as R[];
if (s.includes('FROM "apikeys"')) return (this.#rows.apikeys ?? []) as R[];
if (s.includes('FROM "notes"')) return (this.#rows.notes ?? []) as R[];
return [] as R[];
}
async get<R>(stmt: SqlCode): Promise<R | null> {
const rows = await this.all<R>(stmt);
return rows[0] ?? null;
}
async run(stmt: SqlCode): Promise<{ changes: number }> {
const table = stmt._sql.match(/INSERT INTO "(\w+)"/)?.[1] ?? "?";
this.inserts.push({ table, vars: stmt._vars });
return { changes: 1 };
}
async multi<R>(fn: (tx: WrappedPgClient) => Promise<R>): Promise<R> {
try {
const r = await fn(this);
this.committed = true;
return r;
} catch (e) {
this.rolledBack = true;
throw e;
}
}
async rethrow<R>(fn: () => Promise<R>): Promise<R> {
return fn();
}
}
const iso = "2026-01-02T03:04:05.000Z";
function seededRows(): Rows {
return {
users: [
{
uid: "11111111-1111-1111-1111-111111111111",
email: "u1@test.example",
name: "User One",
flags: {},
created_at: new Date(iso),
updated_at: new Date(iso),
},
],
abodes: [
{
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1",
name: "Abode One",
created_at: new Date(iso),
created_by: "11111111-1111-1111-1111-111111111111",
updated_at: new Date(iso),
updated_by: null,
},
{
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa2",
name: "Abode Two",
created_at: new Date(iso),
created_by: null,
updated_at: new Date(iso),
updated_by: null,
},
],
residents: [
{
uid: "11111111-1111-1111-1111-111111111111",
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1",
flags: {},
created_at: new Date(iso),
created_by: null,
updated_at: new Date(iso),
updated_by: null,
},
],
apikeys: [
{
uid: "11111111-1111-1111-1111-111111111111",
kid: "kkkkkkkk-kkkk-kkkk-kkkk-kkkkkkkkkkk1",
name: "key one",
permissions: {},
created_at: new Date(iso),
expires_at: null,
},
],
notes: [
{
nid: "nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnn1",
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1",
name: "Note one",
content: "# Content",
properties: { type: "note" },
created_at: new Date(iso),
created_by: "11111111-1111-1111-1111-111111111111",
updated_at: new Date(iso),
updated_by: null,
},
],
};
}
async function streamToString(s: NodeJS.ReadableStream): Promise<string> {
let out = "";
for await (const chunk of s) out += chunk;
return out;
}
function parseLines(ndjson: string): { kind: string; data: any }[] {
return ndjson
.split("\n")
.filter(Boolean)
.map((l) => JSON.parse(l));
}
function line(kind: string, data: unknown): string {
return JSON.stringify({ kind, data }) + "\n";
}
describe("postgres export", () => {
it("streams meta and all supported kinds, including notes", async () => {
const db = new PostgresInterface(new FakePg(seededRows()));
const lines = parseLines(await streamToString(db.export()));
const meta = lines.find((l) => l.kind === "meta");
assert.ok(meta);
assert.equal(meta!.data.source, "postgres");
assert.equal(meta!.data.v, 1);
const kinds = new Set(lines.map((l) => l.kind));
assert.ok(kinds.has("user"));
assert.ok(kinds.has("abode"));
assert.ok(kinds.has("resident"));
assert.ok(kinds.has("apikey"));
assert.ok(kinds.has("note"));
});
it("emits record kinds grouped in FK-safe EXPORT_KIND_ORDER", async () => {
const db = new PostgresInterface(new FakePg(seededRows()));
const lines = parseLines(await streamToString(db.export()));
assert.equal(lines[0]?.kind, "meta", "first line is meta");
const rank = (k: string) => EXPORT_KIND_ORDER.indexOf(k as ExportKind);
let last = -1;
for (const { kind } of lines.slice(1)) {
const r = rank(kind);
assert.notEqual(r, -1, `unexpected kind ${kind}`);
assert.ok(r >= last, `kind ${kind} out of FK-safe order`);
last = r;
}
});
it("applies the kinds filter", async () => {
const db = new PostgresInterface(new FakePg(seededRows()));
const lines = parseLines(
await streamToString(db.export({ filter: { kinds: ["abode"] } })),
);
const kinds = new Set(
lines.filter((l) => l.kind !== "meta").map((l) => l.kind),
);
assert.deepEqual(kinds, new Set(["abode"]));
});
it("applies the abodes allowlist to abode/resident records", async () => {
const db = new PostgresInterface(new FakePg(seededRows()));
const lines = parseLines(
await streamToString(
db.export({
filter: { abodes: ["aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"] },
}),
),
);
const abodeAids = lines
.filter((l) => l.kind === "abode")
.map((l) => l.data.aid);
assert.deepEqual(abodeAids, ["aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"]);
});
});
describe("postgres import", () => {
it("dispatches inserts per kind, including note, and commits", async () => {
const fake = new FakePg();
const db = new PostgresInterface(fake);
const source = Readable.from([
line("meta", { v: 1 }),
line("user", {
uid: "11111111-1111-1111-1111-111111111111",
email: "u1@test.example",
name: "User One",
flags: {},
created_at: iso,
updated_at: iso,
}),
line("abode", {
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1",
name: "Abode One",
created_at: iso,
created_by: "11111111-1111-1111-1111-111111111111",
updated_at: iso,
updated_by: null,
}),
line("apikey", {
uid: "11111111-1111-1111-1111-111111111111",
kid: "kkkkkkkk-kkkk-kkkk-kkkk-kkkkkkkkkkk1",
name: "key one",
permissions: {},
created_at: iso,
expires_at: null,
}),
line("note", {
nid: "nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnn1",
aid: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1",
name: "Note",
content: "x",
properties: {},
created_at: iso,
created_by: null,
updated_at: iso,
updated_by: null,
}),
]);
const result = await db.import(source);
assert.deepEqual(result.counts, { user: 1, abode: 1, apikey: 1, note: 1 });
assert.ok(fake.committed);
assert.deepEqual(fake.inserts.map((i) => i.table).sort(), [
"abodes",
"apikeys",
"notes",
"users",
]);
// apikey gets a freshly-minted token (never exported)
const apikeyInsert = fake.inserts.find((i) => i.table === "apikeys")!;
assert.ok(
apikeyInsert.vars.some(
(v) => typeof v === "string" && v.startsWith("at_"),
),
"apikey insert carries a fresh token",
);
});
it("rejects and rolls back when the stream carries an error sentinel", async () => {
const fake = new FakePg();
const db = new PostgresInterface(fake);
const source = Readable.from([
line("meta", { v: 1 }),
line("user", {
uid: "11111111-1111-1111-1111-111111111111",
email: "u1@test.example",
name: "User One",
flags: {},
created_at: iso,
updated_at: iso,
}),
line("error", { message: "boom" }),
]);
await assert.rejects(() => db.import(source), /boom/);
assert.ok(fake.rolledBack);
assert.ok(!fake.committed);
});
it("rejects and rolls back on signal abort", async () => {
const fake = new FakePg();
const db = new PostgresInterface(fake);
const ac = new AbortController();
const source = Readable.from(
(async function* () {
yield line("meta", { v: 1 });
yield line("user", {
uid: "11111111-1111-1111-1111-111111111111",
email: "u1@test.example",
name: "User One",
flags: {},
created_at: iso,
updated_at: iso,
});
ac.abort();
await new Promise((r) => setTimeout(r, 1000));
})(),
);
await assert.rejects(() => db.import(source, { signal: ac.signal }));
assert.ok(fake.rolledBack);
assert.ok(!fake.committed);
});
});
+2 -2
View File
@@ -10,10 +10,10 @@ import { runAuthTests } from "../../shared/auth.js";
async function createExpiredApikey(
db: BackendDbInterface,
uid: string
uid: string,
): Promise<`at_${string}`> {
const si = db as SqliteInterface;
const token = (`at_${"e".repeat(32)}`) as `at_${string}`;
const token = `at_${"e".repeat(32)}` as `at_${string}`;
const kid = crypto.randomUUID();
const { sql } = si._;
si._.db.run(sql`
+2 -2
View File
@@ -46,7 +46,7 @@ describe("SqliteMigrator", () => {
assert.equal(applied.length, 3);
assert.deepEqual(
applied.map((m) => m.id),
[1, 2, 3]
[1, 2, 3],
);
db.destroy();
});
@@ -66,7 +66,7 @@ describe("SqliteMigrator", () => {
const migrator = new SqliteMigrator(db);
await assert.rejects(
() => migrator.migrateTo(9999),
/No known migration with id 9999/
/No known migration with id 9999/,
);
db.destroy();
});
+7 -1
View File
@@ -1,6 +1,12 @@
import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { sql, catSql, joinSql, calcUpdates, unsafeSql } from "../../../src/db/sqlite/sql.js";
import {
sql,
catSql,
joinSql,
calcUpdates,
unsafeSql,
} from "../../../src/db/sqlite/sql.js";
describe("sql template tag", () => {
it("produces correct sql and empty vars for plain text", () => {
+22 -10
View File
@@ -10,13 +10,19 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
before(() => {
db = makeDb();
db.run(unsafeSql("CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, val TEXT NOT NULL)"));
db.run(
unsafeSql(
"CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, val TEXT NOT NULL)",
),
);
});
after(() => db.destroy());
it("run INSERT returns changes count", () => {
const { changes } = db.run(sql`INSERT INTO test(val) VALUES(${{ text: "hello" }})`);
const { changes } = db.run(
sql`INSERT INTO test(val) VALUES(${{ text: "hello" }})`,
);
assert.equal(changes, 1);
});
@@ -24,7 +30,9 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
db.run(unsafeSql("DELETE FROM test"));
db.run(sql`INSERT INTO test(val) VALUES(${{ text: "a" }})`);
db.run(sql`INSERT INTO test(val) VALUES(${{ text: "b" }})`);
const rows = db.all<{ val: string }>(unsafeSql("SELECT val FROM test ORDER BY val"));
const rows = db.all<{ val: string }>(
unsafeSql("SELECT val FROM test ORDER BY val"),
);
assert.equal(rows.length, 2);
assert.equal(rows[0].val, "a");
assert.equal(rows[1].val, "b");
@@ -38,7 +46,7 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
assert.equal(row.val, "one");
const none = db.get<{ val: string }>(
sql`SELECT val FROM test WHERE val = ${{ text: "none" }}`
sql`SELECT val FROM test WHERE val = ${{ text: "none" }}`,
);
assert.equal(none, null);
});
@@ -49,7 +57,7 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
db.run(sql`INSERT INTO test(val) VALUES(${{ text: "dup2" }})`);
assert.throws(
() => db.get<{ val: string }>(unsafeSql("SELECT val FROM test")),
/Multiple results/
/Multiple results/,
);
});
@@ -69,7 +77,7 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
db.multi(() => {
db.run(sql`INSERT INTO test(val) VALUES(${{ text: "rollback" }})`);
throw new Error("abort!");
})
}),
);
const rows = db.all<{ val: string }>(unsafeSql("SELECT val FROM test"));
assert.equal(rows.length, 0);
@@ -82,7 +90,13 @@ function runWrappedDbSuite(name: string, makeDb: () => WrappedDb) {
it("rethrow propagates non-SQLite errors unchanged", () => {
const err = new Error("custom error");
assert.throws(() => db.rethrow(() => { throw err; }), (e) => e === err);
assert.throws(
() =>
db.rethrow(() => {
throw err;
}),
(e) => e === err,
);
});
});
}
@@ -93,9 +107,7 @@ describe("better-sqlite3 WrappedDb", async () => {
let bs3Ctor: (new (path: string) => WrappedDb) | null = null;
try {
const mod = await import(
"../../../src/db/sqlite/impl/better-sqlite3.js"
);
const mod = await import("../../../src/db/sqlite/impl/better-sqlite3.js");
bs3Ctor = mod.WrappedBetterSqlite3Db;
} catch {
// better-sqlite3 not available, skip
+2 -2
View File
@@ -10,7 +10,7 @@ export interface TestServer {
}
export async function createTestServer(
db: BackendDbInterface
db: BackendDbInterface,
): Promise<TestServer> {
const app = new Koa();
const router = apirouter(db);
@@ -23,7 +23,7 @@ export async function createTestServer(
url: `http://127.0.0.1:${port}`,
close: () =>
new Promise<void>((resolve, reject) =>
server.close((err) => (err ? reject(err) : resolve()))
server.close((err) => (err ? reject(err) : resolve())),
),
};
}
+27 -12
View File
@@ -1,12 +1,15 @@
import { describe, it, before, after } from "node:test";
import assert from "node:assert/strict";
import type { DbInterface } from "../../src/db/types/DbInterface.js";
import { NotFoundAbodeError, InvalidAbodeError } from "../../src/db/types/errors.js";
import {
NotFoundAbodeError,
InvalidAbodeError,
} from "../../src/db/types/errors.js";
import { hashPassword } from "../../src/util/hash.js";
export function runAbodeTests(
name: string,
getDb: () => Promise<{ db: DbInterface; close(): void }>
getDb: () => Promise<{ db: DbInterface; close(): void }>,
): void {
describe(`${name}: abodes`, async () => {
let db: DbInterface;
@@ -28,7 +31,10 @@ export function runAbodeTests(
after(() => close());
it("createAbode returns an Abode with expected fields", async () => {
const abode = await db.createAbode({ name: "Test Abode" }, { uid: ctxUid });
const abode = await db.createAbode(
{ name: "Test Abode" },
{ uid: ctxUid },
);
assert.ok(abode.aid, "has aid");
assert.equal(abode.name, "Test Abode");
assert.ok(abode.created_at);
@@ -36,7 +42,10 @@ export function runAbodeTests(
});
it("getAbodeById returns the created abode", async () => {
const created = await db.createAbode({ name: "ById Abode" }, { uid: ctxUid });
const created = await db.createAbode(
{ name: "ById Abode" },
{ uid: ctxUid },
);
const found = await db.getAbodeById(created.aid);
assert.equal(found.aid, created.aid);
assert.equal(found.name, "ById Abode");
@@ -48,14 +57,14 @@ export function runAbodeTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
it("listAbodes includes the created abode", async () => {
const created = await db.createAbode(
{ name: `Listed Abode ${Date.now()}` },
{ uid: ctxUid }
{ uid: ctxUid },
);
const abodes = await db.listAbodes();
assert.ok(Array.isArray(abodes));
@@ -67,32 +76,38 @@ export function runAbodeTests(
const created = await db.createAbode({ name: "Before" }, { uid: ctxUid });
const updated = await db.updateAbode(
{ aid: created.aid, name: "After" },
{ uid: ctxUid }
{ uid: ctxUid },
);
assert.equal(updated.aid, created.aid);
assert.equal(updated.name, "After");
});
it("updateAbode with no fields throws InvalidAbodeError", async () => {
const created = await db.createAbode({ name: "No Update" }, { uid: ctxUid });
const created = await db.createAbode(
{ name: "No Update" },
{ uid: ctxUid },
);
await assert.rejects(
() => db.updateAbode({ aid: created.aid }, { uid: ctxUid }),
(err) => {
assert.ok(err instanceof InvalidAbodeError);
return true;
}
},
);
});
it("deleteAbodeById removes the abode", async () => {
const created = await db.createAbode({ name: "To Delete" }, { uid: ctxUid });
const created = await db.createAbode(
{ name: "To Delete" },
{ uid: ctxUid },
);
await db.deleteAbodeById(created.aid);
await assert.rejects(
() => db.getAbodeById(created.aid),
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -102,7 +117,7 @@ export function runAbodeTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
+4 -4
View File
@@ -6,7 +6,7 @@ import { hashPassword } from "../../src/util/hash.js";
export function runApikeyTests(
name: string,
getDb: () => Promise<{ db: DbInterface; close(): void }>
getDb: () => Promise<{ db: DbInterface; close(): void }>,
): void {
describe(`${name}: apikeys`, async () => {
let db: DbInterface;
@@ -69,7 +69,7 @@ export function runApikeyTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -85,7 +85,7 @@ export function runApikeyTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -95,7 +95,7 @@ export function runApikeyTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
+15 -7
View File
@@ -11,7 +11,10 @@ import { hashPassword } from "../../src/util/hash.js";
export function runAuthTests(
name: string,
getDb: () => Promise<{ db: BackendDbInterface; close(): void }>,
createExpiredApikey?: (db: BackendDbInterface, uid: string) => Promise<`at_${string}`>
createExpiredApikey?: (
db: BackendDbInterface,
uid: string,
) => Promise<`at_${string}`>,
): void {
describe(`${name}: auth`, async () => {
let db: BackendDbInterface;
@@ -23,7 +26,12 @@ export function runAuthTests(
({ db, close } = await getDb());
email = `auth-user-${Date.now()}@test.example`;
const pw = await hashPassword(password);
await db.createUser({ email, name: "Auth User", password: pw, flags: {} });
await db.createUser({
email,
name: "Auth User",
password: pw,
flags: {},
});
});
after(() => close());
@@ -40,7 +48,7 @@ export function runAuthTests(
(err) => {
assert.ok(err instanceof NotAuthorizedAbodeError);
return true;
}
},
);
});
@@ -54,7 +62,7 @@ export function runAuthTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -71,7 +79,7 @@ export function runAuthTests(
(err) => {
assert.ok(err instanceof ConflictAbodeError);
return true;
}
},
);
});
});
@@ -118,7 +126,7 @@ export function runAuthTests(
(err) => {
assert.ok(err instanceof NotAuthorizedAbodeError);
return true;
}
},
);
});
@@ -129,7 +137,7 @@ export function runAuthTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
+21 -12
View File
@@ -1,12 +1,15 @@
import { describe, it, before, after } from "node:test";
import assert from "node:assert/strict";
import type { DbInterface } from "../../src/db/types/DbInterface.js";
import { NotFoundAbodeError, InvalidAbodeError } from "../../src/db/types/errors.js";
import {
NotFoundAbodeError,
InvalidAbodeError,
} from "../../src/db/types/errors.js";
import { hashPassword } from "../../src/util/hash.js";
export function runResidentTests(
name: string,
getDb: () => Promise<{ db: DbInterface; close(): void }>
getDb: () => Promise<{ db: DbInterface; close(): void }>,
): void {
describe(`${name}: residents`, async () => {
let db: DbInterface;
@@ -36,7 +39,7 @@ export function runResidentTests(
uid = resUser.uid;
const abode = await db.createAbode(
{ name: `Resident Abode ${Date.now()}` },
{ uid: ctxUid }
{ uid: ctxUid },
);
aid = abode.aid;
await db.createResident({ uid, aid, flags: {} }, { uid: ctxUid });
@@ -56,12 +59,12 @@ export function runResidentTests(
() =>
db.getResidentById(
"00000000-0000-0000-0000-000000000000",
"00000000-0000-0000-0000-000000000001"
"00000000-0000-0000-0000-000000000001",
),
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -101,7 +104,7 @@ export function runResidentTests(
it("updateResident updates flags", async () => {
const updated = await db.updateResident(
{ uid, aid, flags: { admin: true } },
{ uid: ctxUid }
{ uid: ctxUid },
);
assert.equal(updated.uid, uid);
assert.deepEqual(updated.flags, { admin: true });
@@ -113,7 +116,7 @@ export function runResidentTests(
(err) => {
assert.ok(err instanceof InvalidAbodeError);
return true;
}
},
);
});
@@ -125,15 +128,21 @@ export function runResidentTests(
password: pw,
flags: {},
});
const abode2 = await db.createAbode({ name: "Del Abode" }, { uid: ctxUid });
await db.createResident({ uid: user2.uid, aid: abode2.aid, flags: {} }, { uid: ctxUid });
const abode2 = await db.createAbode(
{ name: "Del Abode" },
{ uid: ctxUid },
);
await db.createResident(
{ uid: user2.uid, aid: abode2.aid, flags: {} },
{ uid: ctxUid },
);
await db.deleteResidentById(user2.uid, abode2.aid);
await assert.rejects(
() => db.getResidentById(user2.uid, abode2.aid),
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -142,12 +151,12 @@ export function runResidentTests(
() =>
db.deleteResidentById(
"00000000-0000-0000-0000-000000000002",
"00000000-0000-0000-0000-000000000003"
"00000000-0000-0000-0000-000000000003",
),
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
+3 -3
View File
@@ -6,7 +6,7 @@ import { hashPassword } from "../../src/util/hash.js";
export function runSessionTests(
name: string,
getDb: () => Promise<{ db: BackendDbInterface; close(): void }>
getDb: () => Promise<{ db: BackendDbInterface; close(): void }>,
): void {
describe(`${name}: sessions`, async () => {
let db: BackendDbInterface;
@@ -46,7 +46,7 @@ export function runSessionTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -58,7 +58,7 @@ export function runSessionTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
+17 -9
View File
@@ -11,7 +11,7 @@ import { hashPassword } from "../../src/util/hash.js";
export function runUserTests(
name: string,
getDb: () => Promise<{ db: DbInterface; close(): void }>,
getReadonlyDb?: () => Promise<{ db: DbInterface; close(): void }>
getReadonlyDb?: () => Promise<{ db: DbInterface; close(): void }>,
): void {
describe(`${name}: users`, async () => {
let db: DbInterface;
@@ -57,13 +57,18 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
it("getUserByEmail returns the created user", async () => {
const email = `user-byemail-${Date.now()}@test.example`;
await db.createUser({ email, name: "ByEmail User", password: hashedPw, flags: {} });
await db.createUser({
email,
name: "ByEmail User",
password: hashedPw,
flags: {},
});
const found = await db.getUserByEmail(email);
assert.ok("email" in found, "result includes email");
assert.equal(found.email, email);
@@ -75,7 +80,7 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -100,7 +105,10 @@ export function runUserTests(
password: hashedPw,
flags: {},
});
const updated = await db.updateUser({ uid: created.uid, name: "After Update" });
const updated = await db.updateUser({
uid: created.uid,
name: "After Update",
});
assert.equal(updated.uid, created.uid);
assert.equal(updated.name, "After Update");
});
@@ -117,7 +125,7 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof InvalidAbodeError);
return true;
}
},
);
});
@@ -134,7 +142,7 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
@@ -144,7 +152,7 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof NotFoundAbodeError);
return true;
}
},
);
});
});
@@ -178,7 +186,7 @@ export function runUserTests(
(err) => {
assert.ok(err instanceof ReadonlyAbodeError);
return true;
}
},
);
});
});
+110 -29
View File
@@ -29,7 +29,7 @@ const MOCK_APIKEY: ClientApikey = {
};
function makeMockDb(
overrides: Partial<BackendDbInterface> = {}
overrides: Partial<BackendDbInterface> = {},
): BackendDbInterface {
return {
readonly: false,
@@ -37,42 +37,97 @@ function makeMockDb(
name: "mock",
close: async () => {},
listUsers: async () => [],
getUserById: async () => { throw new NotFoundAbodeError(); },
getUserById: async () => {
throw new NotFoundAbodeError();
},
deleteUserById: async () => {},
createUser: async () => MOCK_USER,
updateUser: async () => MOCK_USER,
getUserByEmail: async () => { throw new NotFoundAbodeError(); },
getUserByEmail: async () => {
throw new NotFoundAbodeError();
},
listAbodes: async () => [],
getAbodeById: async () => { throw new NotFoundAbodeError(); },
getAbodeById: async () => {
throw new NotFoundAbodeError();
},
deleteAbodeById: async () => {},
createAbode: async () => ({ aid: "a", name: "A", created_at: "", created_by: null, updated_at: "", updated_by: null }),
updateAbode: async () => ({ aid: "a", name: "A", created_at: "", created_by: null, updated_at: "", updated_by: null }),
createAbode: async () => ({
aid: "a",
name: "A",
created_at: "",
created_by: null,
updated_at: "",
updated_by: null,
}),
updateAbode: async () => ({
aid: "a",
name: "A",
created_at: "",
created_by: null,
updated_at: "",
updated_by: null,
}),
listResidents: async () => [],
getResidentById: async () => { throw new NotFoundAbodeError(); },
getResidentById: async () => {
throw new NotFoundAbodeError();
},
deleteResidentById: async () => {},
createResident: async () => ({ uid: "", aid: "", flags: {}, created_at: "", created_by: null, updated_at: "", updated_by: null }),
updateResident: async () => ({ uid: "", aid: "", flags: {}, created_at: "", created_by: null, updated_at: "", updated_by: null }),
createResident: async () => ({
uid: "",
aid: "",
flags: {},
created_at: "",
created_by: null,
updated_at: "",
updated_by: null,
}),
updateResident: async () => ({
uid: "",
aid: "",
flags: {},
created_at: "",
created_by: null,
updated_at: "",
updated_by: null,
}),
listResidentsByUserId: async () => [],
listResidentsByAbodeId: async () => [],
listUsersByAbodeId: async () => [],
listAbodesByUserId: async () => [],
listNotes: async () => [],
getNoteById: async () => { throw new NotFoundAbodeError(); },
getNoteById: async () => {
throw new NotFoundAbodeError();
},
deleteNoteById: async () => {},
createNote: async () => { throw new Error("unimplemented"); },
updateNote: async () => { throw new Error("unimplemented"); },
createNote: async () => {
throw new Error("unimplemented");
},
updateNote: async () => {
throw new Error("unimplemented");
},
listNotesByAbodeId: async () => [],
listNotesByUserId: async () => [],
deleteSessionsByUser: async () => {},
listApikeysByUser: async () => [],
getApikeyById: async () => { throw new NotFoundAbodeError(); },
createApikey: async () => [MOCK_APIKEY, "at_" + "0".repeat(32) as `at_${string}`],
getApikeyById: async () => {
throw new NotFoundAbodeError();
},
createApikey: async () => [
MOCK_APIKEY,
("at_" + "0".repeat(32)) as `at_${string}`,
],
deleteApikeyById: async () => {},
getUserByLogin: async () => { throw new NotFoundAbodeError(); },
getUserBySession: async () => { throw new NotFoundAbodeError(); },
getUserByLogin: async () => {
throw new NotFoundAbodeError();
},
getUserBySession: async () => {
throw new NotFoundAbodeError();
},
createSession: async () => `as_${"0".repeat(32)}`,
deleteSession: async () => {},
getUserByApikey: async () => { throw new NotFoundAbodeError(); },
getUserByApikey: async () => {
throw new NotFoundAbodeError();
},
...overrides,
};
}
@@ -92,7 +147,10 @@ type MockCtx = {
};
};
function makeMockCtx(headerOverrides: Record<string, string> = {}, cookieOverrides: Record<string, string> = {}): MockCtx {
function makeMockCtx(
headerOverrides: Record<string, string> = {},
cookieOverrides: Record<string, string> = {},
): MockCtx {
const clearedCookies = new Set<string>();
const ctx: MockCtx = {
headers: headerOverrides,
@@ -110,7 +168,10 @@ function makeMockCtx(headerOverrides: Record<string, string> = {}, cookieOverrid
return cookieOverrides[name];
},
set(name: string, value: string, opts?: unknown) {
if (value === "" || (opts && (opts as { expires?: Date }).expires?.getFullYear()! < 2000)) {
if (
value === "" ||
(opts && (opts as { expires?: Date }).expires?.getFullYear()! < 2000)
) {
clearedCookies.add(name);
}
},
@@ -121,11 +182,13 @@ function makeMockCtx(headerOverrides: Record<string, string> = {}, cookieOverrid
async function runMiddleware(
db: BackendDbInterface,
ctx: MockCtx
ctx: MockCtx,
): Promise<boolean> {
let nextCalled = false;
const mw = authenticate(db);
await mw(ctx as any, async () => { nextCalled = true; });
await mw(ctx as any, async () => {
nextCalled = true;
});
return nextCalled;
}
@@ -157,7 +220,9 @@ describe("authenticate middleware", () => {
it("wrong password (NotAuthorizedAbodeError) → 401 invalid_password", async () => {
const db = makeMockDb({
getUserByLogin: async () => { throw new NotAuthorizedAbodeError(); },
getUserByLogin: async () => {
throw new NotAuthorizedAbodeError();
},
});
const ctx = makeMockCtx({ Authorization: "Basic " + btoa("user:wrong") });
await runMiddleware(db, ctx);
@@ -167,9 +232,13 @@ describe("authenticate middleware", () => {
it("unknown user (NotFoundAbodeError) → 401 unknown_user", async () => {
const db = makeMockDb({
getUserByLogin: async () => { throw new NotFoundAbodeError(); },
getUserByLogin: async () => {
throw new NotFoundAbodeError();
},
});
const ctx = makeMockCtx({
Authorization: "Basic " + btoa("nobody:pass"),
});
const ctx = makeMockCtx({ Authorization: "Basic " + btoa("nobody:pass") });
await runMiddleware(db, ctx);
assert.equal(ctx.status, 401);
assert.deepEqual((ctx.body as any)?.error, "unknown_user");
@@ -177,7 +246,9 @@ describe("authenticate middleware", () => {
it("ConflictAbodeError (#unset password) → 401 user_not_loggable", async () => {
const db = makeMockDb({
getUserByLogin: async () => { throw new ConflictAbodeError(); },
getUserByLogin: async () => {
throw new ConflictAbodeError();
},
});
const ctx = makeMockCtx({ Authorization: "Basic " + btoa("user:pass") });
await runMiddleware(db, ctx);
@@ -211,7 +282,9 @@ describe("authenticate middleware", () => {
it("invalid/expired at_ token → 401 invalid_apikey", async () => {
const db = makeMockDb({
getUserByApikey: async () => { throw new NotFoundAbodeError(); },
getUserByApikey: async () => {
throw new NotFoundAbodeError();
},
});
const ctx = makeMockCtx({ Authorization: `Bearer ${validToken}` });
await runMiddleware(db, ctx);
@@ -246,17 +319,25 @@ describe("authenticate middleware", () => {
const db = makeMockDb();
const ctx = makeMockCtx({}, { abode_session: "not-a-session-token" });
await runMiddleware(db, ctx);
assert.ok(ctx.clearedCookies.has("abode_session"), "cookie should be cleared");
assert.ok(
ctx.clearedCookies.has("abode_session"),
"cookie should be cleared",
);
assert.equal(ctx.status, 401);
});
it("expired/unknown session token clears cookie and returns 401", async () => {
const db = makeMockDb({
getUserBySession: async () => { throw new NotFoundAbodeError(); },
getUserBySession: async () => {
throw new NotFoundAbodeError();
},
});
const ctx = makeMockCtx({}, { abode_session: validToken });
await runMiddleware(db, ctx);
assert.ok(ctx.clearedCookies.has("abode_session"), "cookie should be cleared");
assert.ok(
ctx.clearedCookies.has("abode_session"),
"cookie should be cleared",
);
assert.equal(ctx.status, 401);
});
});
+3 -1
View File
@@ -25,7 +25,9 @@ describe("convertError middleware", () => {
it("does not interfere when next succeeds", async () => {
const ctx = makeCtx();
let nextCalled = false;
await convertError(ctx as any, async () => { nextCalled = true; });
await convertError(ctx as any, async () => {
nextCalled = true;
});
assert.equal(nextCalled, true);
assert.equal(ctx.status, 200);
});
+794
View File
@@ -0,0 +1,794 @@
import { describe, it, before, after } from "node:test";
import assert from "node:assert/strict";
import readline from "node:readline";
import { Readable } from "node:stream";
import { createServer } from "node:http";
import type { AddressInfo } from "node:net";
import Koa from "koa";
import { createTestDb, type TestDb } from "../helpers/sqlite.js";
import { apirouter } from "../../src/webapi/apirouter.js";
import { computeForcedExportFilter } from "../../src/webapi/exportScope.js";
import {
intersectExportFilters,
recordAllowed,
kindAllowed,
} from "../../src/db/export/filter.js";
import { inspectExportStream } from "../../src/db/export/inspect.js";
import { hashPassword } from "../../src/util/hash.js";
import type { ClientUser } from "../../src/db/types/User.js";
import {
EXPORT_KIND_ORDER,
type ExportKind,
} from "../../src/db/types/ExportImport.js";
/**
* Assert the record lines of a parsed export are grouped in FK-safe
* EXPORT_KIND_ORDER: the leading line is `meta`, and every record kind's
* position in the order is non-decreasing down the stream.
*/
function assertFkSafeOrder(lines: { kind: string }[]): void {
assert.equal(lines[0]?.kind, "meta", "first line is meta");
const rank = (k: string) => EXPORT_KIND_ORDER.indexOf(k as ExportKind);
let last = -1;
for (const { kind } of lines.slice(1)) {
if (kind === "error") continue;
const r = rank(kind);
assert.notEqual(r, -1, `unexpected kind ${kind}`);
assert.ok(
r >= last,
`kind ${kind} (order ${r}) appears after a later kind (order ${last})`,
);
last = r;
}
}
// ---------------------------------------------------------------------------
// helpers
// ---------------------------------------------------------------------------
async function streamToString(s: NodeJS.ReadableStream): Promise<string> {
let out = "";
for await (const chunk of s) out += chunk;
return out;
}
function parseLines(ndjson: string): { kind: string; data: any }[] {
return ndjson
.split("\n")
.filter(Boolean)
.map((l) => JSON.parse(l));
}
/** Normalise `*_at` fields to epoch ms so formatting never breaks equality. */
function norm(obj: Record<string, unknown>): Record<string, unknown> {
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(obj)) {
out[k] = k.endsWith("_at")
? v == null
? null
: new Date(v as string).getTime()
: v;
}
return out;
}
function normSorted(
arr: Record<string, unknown>[],
key: (x: any) => string,
): Record<string, unknown>[] {
return arr.map(norm).sort((a, b) => key(a).localeCompare(key(b)));
}
interface Seed {
admin: ClientUser;
normal: ClientUser;
co: ClientUser;
aid1: string;
aid2: string;
nid1: string;
nid2: string;
}
async function seed(db: TestDb["db"]): Promise<Seed> {
const pw = await hashPassword("password");
const admin = await db.createUser({
email: "admin@test.example",
name: "Admin",
password: pw,
flags: { admin: true },
});
const normal = await db.createUser({
email: "normal@test.example",
name: "Normal",
password: pw,
flags: {},
});
const co = await db.createUser({
email: "co@test.example",
name: "Co Resident",
password: pw,
flags: {},
});
const abode1 = await db.createAbode(
{ name: "Abode One" },
{ uid: admin.uid },
);
const abode2 = await db.createAbode(
{ name: "Abode Two" },
{ uid: admin.uid },
);
await db.createResident(
{ uid: normal.uid, aid: abode1.aid, flags: {} },
{ uid: admin.uid },
);
await db.createResident(
{ uid: co.uid, aid: abode1.aid, flags: {} },
{ uid: admin.uid },
);
await db.createResident(
{ uid: admin.uid, aid: abode2.aid, flags: { admin: true } },
{ uid: admin.uid },
);
await db.createApikey({
uid: normal.uid,
name: "normal key",
permissions: {},
expires_at: null,
});
await db.createApikey({
uid: co.uid,
name: "co key",
permissions: {},
expires_at: null,
});
const note1 = await db.createNote(
{
aid: abode1.aid,
name: "Note One",
content: "hello",
properties: { type: "note" },
},
{ uid: normal.uid },
);
const note2 = await db.createNote(
{
aid: abode2.aid,
name: "Note Two",
content: "world",
properties: { type: "note" },
},
{ uid: admin.uid },
);
return {
admin,
normal,
co,
aid1: abode1.aid,
aid2: abode2.aid,
nid1: note1.nid,
nid2: note2.nid,
};
}
// ---------------------------------------------------------------------------
// 0. wire-format ordering invariant
// ---------------------------------------------------------------------------
describe("export ordering (FK-safe wire contract)", () => {
it("EXPORT_KIND_ORDER is the FK-safe dependency order", () => {
// Change-detector: reordering is a breaking change to the format and must
// keep every kind after the kinds it references (see ExportImport.ts).
assert.deepEqual(EXPORT_KIND_ORDER, [
"user",
"abode",
"resident",
"apikey",
"note",
]);
});
it("sqlite export emits record kinds grouped in EXPORT_KIND_ORDER", async () => {
const src = await createTestDb();
try {
await seed(src.db);
const lines = parseLines(await streamToString(src.db.export()));
// every kind must be present so the ordering is actually exercised
const kinds = new Set(lines.map((l) => l.kind));
for (const k of EXPORT_KIND_ORDER) {
assert.ok(kinds.has(k), `stream contains ${k}`);
}
assertFkSafeOrder(lines);
} finally {
src.close();
}
});
});
// ---------------------------------------------------------------------------
// 1. round-trip
// ---------------------------------------------------------------------------
describe("export/import: sqlite -> sqlite round-trip", () => {
it("reproduces users, abodes, residents, apikeys, notes", async () => {
const src = await createTestDb();
const dst = await createTestDb();
try {
const s = await seed(src.db);
const result = await dst.db.import(src.db.export());
assert.ok(result.counts.user && result.counts.user >= 3);
assert.ok(result.counts.note && result.counts.note >= 2);
assert.deepEqual(
normSorted(await dst.db.listUsers(), (u) => u.uid),
normSorted(await src.db.listUsers(), (u) => u.uid),
);
assert.deepEqual(
normSorted(await dst.db.listAbodes(), (a) => a.aid),
normSorted(await src.db.listAbodes(), (a) => a.aid),
);
assert.deepEqual(
normSorted(await dst.db.listResidents(), (r) => r.uid + r.aid),
normSorted(await src.db.listResidents(), (r) => r.uid + r.aid),
);
// apikeys: token is regenerated on import, so the ClientApikey view
// (which omits token) must still match exactly.
assert.deepEqual(
normSorted(await dst.db.listApikeysByUser(s.normal.uid), (k) => k.kid),
normSorted(await src.db.listApikeysByUser(s.normal.uid), (k) => k.kid),
);
// notes (full, with content)
const srcNote = await src.db.getNoteById(s.nid1);
const dstNote = await dst.db.getNoteById(s.nid1);
assert.deepEqual(norm(dstNote), norm(srcNote));
} finally {
src.close();
dst.close();
}
});
});
// ---------------------------------------------------------------------------
// 2. filter narrowing
// ---------------------------------------------------------------------------
describe("export/import: filter narrowing", () => {
it("scopes abode/resident/note to the requested abodes", async () => {
const src = await createTestDb();
try {
const s = await seed(src.db);
const ndjson = await streamToString(
src.db.export({ filter: { abodes: [s.aid1] } }),
);
const lines = parseLines(ndjson);
const abodeAids = lines
.filter((l) => l.kind === "abode")
.map((l) => l.data.aid);
assert.deepEqual(abodeAids, [s.aid1]);
const noteAids = new Set(
lines.filter((l) => l.kind === "note").map((l) => l.data.aid),
);
assert.ok(noteAids.has(s.aid1));
assert.ok(!noteAids.has(s.aid2));
const residentAids = new Set(
lines.filter((l) => l.kind === "resident").map((l) => l.data.aid),
);
assert.ok(!residentAids.has(s.aid2));
} finally {
src.close();
}
});
it("imports the narrowed dump into a fresh db with the same scope", async () => {
const src = await createTestDb();
const dst = await createTestDb();
try {
const s = await seed(src.db);
await dst.db.import(src.db.export({ filter: { abodes: [s.aid1] } }));
const abodes = await dst.db.listAbodes();
assert.deepEqual(
abodes.map((a) => a.aid),
[s.aid1],
);
const notes = await dst.db.listNotesByAbodeId(s.aid1);
assert.equal(notes.length, 1);
assert.equal((await dst.db.listNotesByAbodeId(s.aid2)).length, 0);
} finally {
src.close();
dst.close();
}
});
it("kinds filter selects only the requested kinds", async () => {
const src = await createTestDb();
try {
await seed(src.db);
const ndjson = await streamToString(
src.db.export({ filter: { kinds: ["abode"] } }),
);
const kinds = new Set(parseLines(ndjson).map((l) => l.kind));
assert.ok(kinds.has("abode"));
assert.ok(!kinds.has("user"));
assert.ok(!kinds.has("note"));
} finally {
src.close();
}
});
});
// ---------------------------------------------------------------------------
// 3. forced-filter enforcement
// ---------------------------------------------------------------------------
describe("exportScope: computeForcedExportFilter", () => {
it("returns null for a global admin on a basic/session credential", async () => {
const t = await createTestDb();
try {
const s = await seed(t.db);
const forced = await computeForcedExportFilter(t.db, {
user: s.admin,
session: { source: "basic" },
});
assert.equal(forced, null);
} finally {
t.close();
}
});
it("returns null for a global admin with an unrestricted apikey", async () => {
const t = await createTestDb();
try {
const s = await seed(t.db);
const forced = await computeForcedExportFilter(t.db, {
user: s.admin,
session: {
source: "apikey",
key: {
kid: "k",
uid: s.admin.uid,
name: "k",
permissions: { admin: true, all: true },
created_at: new Date().toISOString(),
expires_at: null,
},
},
});
assert.equal(forced, null);
} finally {
t.close();
}
});
it("forces a non-admin to their abodes + co-residents", async () => {
const t = await createTestDb();
try {
const s = await seed(t.db);
const forced = await computeForcedExportFilter(t.db, {
user: s.normal,
session: { source: "basic" },
});
assert.ok(forced);
assert.deepEqual(forced!.abodes, [s.aid1]);
assert.deepEqual(
new Set(forced!.users),
new Set([s.normal.uid, s.co.uid]),
);
assert.ok(!forced!.users!.includes(s.admin.uid));
// apikeys are self-only, even though co is a co-resident whose user
// record is exported for referential integrity.
assert.deepEqual(forced!.apikeys, [s.normal.uid]);
// A caller requesting a wider abode never gets it: intersection, not union.
const effective = intersectExportFilters(
{ abodes: [s.aid1, s.aid2] },
forced,
);
assert.deepEqual(effective.abodes, [s.aid1]);
assert.ok(!effective.abodes!.includes(s.aid2));
} finally {
t.close();
}
});
it("intersects a non-admin apikey with restrict_abodes", async () => {
const t = await createTestDb();
try {
const s = await seed(t.db);
const forced = await computeForcedExportFilter(t.db, {
user: s.normal,
session: {
source: "apikey",
key: {
kid: "k",
uid: s.normal.uid,
name: "k",
permissions: { restrict_abodes: [s.aid2] },
created_at: new Date().toISOString(),
expires_at: null,
},
},
});
// normal resides only in aid1; restrict to aid2 => empty intersection.
assert.deepEqual(forced!.abodes, []);
} finally {
t.close();
}
});
});
// ---------------------------------------------------------------------------
// 4. cancellation, export side
// ---------------------------------------------------------------------------
describe("export cancellation", () => {
it("stops querying tables after the destination aborts", async () => {
const t = await createTestDb();
try {
const pw = await hashPassword("password");
// Enough users that the first table can't be buffered in one go, so the
// generator backpressures mid-`user` and never reaches later tables.
for (let i = 0; i < 400; i++) {
await t.db.createUser({
email: `bulk-${i}@test.example`,
name: `Bulk ${i}`,
password: pw,
flags: {},
});
}
let allCalls = 0;
const orig = t.wrapped.all.bind(t.wrapped);
(t.wrapped as { all: unknown }).all = (stmt: never) => {
allCalls++;
return orig(stmt);
};
const ac = new AbortController();
const stream = t.db.export({ signal: ac.signal });
const rl = readline.createInterface({ input: stream });
let lines = 0;
let callsAtAbort = -1;
for await (const _ of rl) {
lines++;
if (lines === 3) {
ac.abort();
callsAtAbort = allCalls;
}
}
assert.ok(callsAtAbort >= 1, "at least the first table was queried");
assert.equal(allCalls, callsAtAbort, "no further queries after abort");
assert.ok(allCalls < 5, "did not materialise all five tables");
} finally {
t.close();
}
});
});
// ---------------------------------------------------------------------------
// 5. cancellation, import side
// ---------------------------------------------------------------------------
describe("import cancellation", () => {
it("rolls back and leaves no dangling transaction when the source errors", async () => {
const dst = await createTestDb();
try {
function line(kind: string, data: unknown): string {
return JSON.stringify({ kind, data }) + "\n";
}
const now = new Date().toISOString();
const mkUser = (i: number) => ({
uid: crypto.randomUUID(),
email: `imp-${i}@test.example`,
name: `Imp ${i}`,
flags: {},
created_at: now,
updated_at: now,
});
// Emits a couple of valid user records, then throws mid-stream.
const source = Readable.from(
(async function* () {
yield line("meta", { v: 1 });
yield line("user", mkUser(1));
yield line("user", mkUser(2));
throw new Error("source exploded");
})(),
);
await assert.rejects(() => dst.db.import(source), /source exploded/);
// ROLLBACK happened: nothing persisted.
assert.equal((await dst.db.listUsers()).length, 0);
// No lingering open transaction: a follow-up write succeeds immediately.
const pw = await hashPassword("password");
const created = await dst.db.createUser({
email: "after@test.example",
name: "After",
password: pw,
flags: {},
});
assert.ok(created.uid);
} finally {
dst.close();
}
});
it("aborts cleanly via signal and commits nothing", async () => {
const dst = await createTestDb();
try {
const ac = new AbortController();
const now = new Date().toISOString();
const source = Readable.from(
(async function* () {
yield JSON.stringify({ kind: "meta", data: { v: 1 } }) + "\n";
yield JSON.stringify({
kind: "user",
data: {
uid: crypto.randomUUID(),
email: "abort@test.example",
name: "Abort",
flags: {},
created_at: now,
updated_at: now,
},
}) + "\n";
ac.abort();
// Keep the stream alive so abort — not EOF — ends the import.
await new Promise((r) => setTimeout(r, 1000));
})(),
);
await assert.rejects(() => dst.db.import(source, { signal: ac.signal }));
assert.equal((await dst.db.listUsers()).length, 0);
} finally {
dst.close();
}
});
});
// ---------------------------------------------------------------------------
// 6. api export endpoint
// ---------------------------------------------------------------------------
describe("GET /export endpoint", () => {
let t: TestDb;
let s: Seed;
let url: string;
let close: () => Promise<void>;
const basic = (email: string) =>
"Basic " + Buffer.from(`${email}:password`).toString("base64");
before(async () => {
t = await createTestDb();
s = await seed(t.db);
const app = new Koa();
const router = apirouter(t.db);
app.use(router.routes());
app.use(router.allowedMethods());
const server = createServer(app.callback());
await new Promise<void>((r) => server.listen(0, "127.0.0.1", r));
const { port } = server.address() as AddressInfo;
url = `http://127.0.0.1:${port}`;
close = () =>
new Promise<void>((resolve, reject) =>
server.close((err) => (err ? reject(err) : resolve())),
);
});
after(async () => {
await close();
t.close();
});
it("global admin exports everything", async () => {
const res = await fetch(`${url}/export`, {
headers: { Authorization: basic(s.admin.email) },
});
assert.equal(res.status, 200);
const lines = parseLines(await res.text());
const abodeAids = new Set(
lines.filter((l) => l.kind === "abode").map((l) => l.data.aid),
);
assert.ok(abodeAids.has(s.aid1));
assert.ok(abodeAids.has(s.aid2));
const userUids = new Set(
lines.filter((l) => l.kind === "user").map((l) => l.data.uid),
);
assert.ok(userUids.has(s.admin.uid));
assert.ok(userUids.has(s.normal.uid));
});
it("non-admin is force-scoped even when requesting wider abodes", async () => {
const res = await fetch(`${url}/export?abodes=${s.aid1},${s.aid2}`, {
headers: { Authorization: basic(s.normal.email) },
});
assert.equal(res.status, 200);
const lines = parseLines(await res.text());
const abodeAids = new Set(
lines.filter((l) => l.kind === "abode").map((l) => l.data.aid),
);
assert.ok(abodeAids.has(s.aid1));
assert.ok(!abodeAids.has(s.aid2), "aid2 forced out of scope");
const userUids = new Set(
lines.filter((l) => l.kind === "user").map((l) => l.data.uid),
);
assert.ok(userUids.has(s.normal.uid));
assert.ok(userUids.has(s.co.uid));
assert.ok(!userUids.has(s.admin.uid), "admin not a co-resident of aid1");
// apikeys are self-only: the caller's own key is exported, but a
// co-resident's key metadata is NOT, even though their user record is.
const apikeyUids = lines
.filter((l) => l.kind === "apikey")
.map((l) => l.data.uid);
assert.deepEqual(new Set(apikeyUids), new Set([s.normal.uid]));
assert.ok(
!apikeyUids.includes(s.co.uid),
"co-resident apikey metadata must not leak",
);
// The meta line records the *effective* (narrowed) filter.
const meta = lines.find((l) => l.kind === "meta");
assert.ok(meta);
assert.deepEqual(meta!.data.filter.abodes, [s.aid1]);
});
it("stops server-side querying shortly after the client aborts", async () => {
// Dedicated db/server. Rows are deliberately large (and inserted in bulk so
// the seed stays fast) so the users table alone can't fit in the socket
// buffer — the export generator backpressures mid-`user` and never reaches
// later tables while the client is still holding the connection open.
const big = await createTestDb();
let server: ReturnType<typeof createServer> | undefined;
try {
const pw = await hashPassword("password");
await big.db.createUser({
email: "a@test.example",
name: "A",
password: pw,
flags: { admin: true },
});
const { sql, db: raw } = big.db._;
// ~32MB of user rows — far more than any socket/kernel buffer can hold,
// so the generator is guaranteed to still be suspended mid-`user` (never
// having queried a later table) when the client aborts.
const bigName = "x".repeat(16000);
raw.multi(() => {
for (let i = 0; i < 2000; i++) {
raw.run(sql`
INSERT INTO "users"("uid", "email", "name", "flags")
VALUES(
${{ uuid: crypto.randomUUID() }},
${{ text: `b-${i}@test.example` }},
${{ text: bigName }},
${{ jsonb: {} }}
)
`);
}
});
let allCalls = 0;
const orig = big.wrapped.all.bind(big.wrapped);
(big.wrapped as { all: unknown }).all = (stmt: never) => {
allCalls++;
return orig(stmt);
};
const app = new Koa();
app.on("error", () => {}); // swallow the expected ECONNRESET on abort
const router = apirouter(big.db);
app.use(router.routes());
app.use(router.allowedMethods());
server = createServer(app.callback());
await new Promise<void>((r) => server!.listen(0, "127.0.0.1", r));
const { port } = server.address() as AddressInfo;
const ac = new AbortController();
const res = await fetch(`http://127.0.0.1:${port}/export`, {
headers: { Authorization: basic("a@test.example") },
signal: ac.signal,
});
const reader = res.body!.getReader();
await reader.read(); // first chunk — server has begun streaming users
const callsWhileStreaming = allCalls;
assert.ok(callsWhileStreaming >= 1, "server queried the first table");
ac.abort();
await reader.cancel().catch(() => {});
await new Promise((r) => setTimeout(r, 250));
const settled = allCalls;
await new Promise((r) => setTimeout(r, 250));
assert.equal(allCalls, settled, "no further queries after abort");
assert.ok(allCalls < 5, "did not materialise all five tables");
} finally {
if (server) await new Promise<void>((r) => server!.close(() => r()));
big.close();
}
});
});
// ---------------------------------------------------------------------------
// 7. inspect utility
// ---------------------------------------------------------------------------
describe("inspectExportStream", () => {
it("tallies counts and reads meta without a db", async () => {
const src = await createTestDb();
try {
await seed(src.db);
const ndjson = await streamToString(src.db.export());
const { counts, meta } = await inspectExportStream(
Readable.from([ndjson]),
);
assert.equal(meta?.v, 1);
assert.equal(meta?.source, "sqlite");
assert.ok((counts.user ?? 0) >= 3);
assert.ok((counts.abode ?? 0) >= 2);
assert.ok((counts.note ?? 0) >= 2);
} finally {
src.close();
}
});
it("stopAfterKinds short-circuits once every requested kind is seen", async () => {
const src = await createTestDb();
try {
await seed(src.db);
const ndjson = await streamToString(src.db.export());
const { counts } = await inspectExportStream(Readable.from([ndjson]), {
stopAfterKinds: ["user"],
});
assert.ok((counts.user ?? 0) >= 1);
// stopped as soon as the first user was seen, before later kinds.
assert.equal(counts.note ?? 0, 0);
} finally {
src.close();
}
});
});
// ---------------------------------------------------------------------------
// filter unit checks
// ---------------------------------------------------------------------------
describe("filter helpers", () => {
it("kindAllowed respects kinds/excludeKinds", () => {
assert.equal(kindAllowed({ kinds: ["abode"] }, "abode"), true);
assert.equal(kindAllowed({ kinds: ["abode"] }, "user"), false);
assert.equal(kindAllowed({ excludeKinds: ["note"] }, "note"), false);
assert.equal(kindAllowed(undefined, "note"), true);
});
it("recordAllowed scopes by aid/uid per kind", () => {
assert.equal(
recordAllowed({ abodes: ["a1"] }, "abode", { aid: "a1" }),
true,
);
assert.equal(
recordAllowed({ abodes: ["a1"] }, "abode", { aid: "a2" }),
false,
);
assert.equal(
recordAllowed({ users: ["u1"] }, "apikey", { uid: "u1" }),
true,
);
assert.equal(
recordAllowed({ users: ["u1"] }, "apikey", { uid: "u2" }),
false,
);
// abodes allowlist does not constrain user records
assert.equal(
recordAllowed({ abodes: ["a1"] }, "user", { uid: "u9" }),
true,
);
});
});
+20 -14
View File
@@ -8,7 +8,7 @@ import { jsonBody } from "../../src/webapi/middleware/jsonBody.js";
async function request(
url: string,
opts: { method?: string; body?: unknown; contentType?: string } = {}
opts: { method?: string; body?: unknown; contentType?: string } = {},
): Promise<{ status: number; body: unknown }> {
const method = opts.method ?? "POST";
const bodyStr =
@@ -21,18 +21,19 @@ async function request(
const res = await fetch(url, { method, headers, body: bodyStr });
const text = await res.text();
let parsed: unknown;
try { parsed = JSON.parse(text); } catch { parsed = text; }
try {
parsed = JSON.parse(text);
} catch {
parsed = text;
}
return { status: res.status, body: parsed };
}
async function makeTestServer() {
const failValidator = Object.assign(
(_obj: unknown): _obj is never => false,
{
errors: [{ message: "required" }] as unknown[],
schema: { $id: "test-schema", title: "Test", description: "" },
}
);
const failValidator = Object.assign((_obj: unknown): _obj is never => false, {
errors: [{ message: "required" }] as unknown[],
schema: { $id: "test-schema", title: "Test", description: "" },
});
const app = new Koa();
const router = new KoaRouter();
@@ -48,7 +49,7 @@ async function makeTestServer() {
async (ctx) => {
ctx.status = 200;
ctx.body = { ok: true };
}
},
);
router.post(
@@ -57,7 +58,7 @@ async function makeTestServer() {
async (ctx) => {
ctx.status = 200;
ctx.body = { ok: true, body: ctx.request.body };
}
},
);
app.use(router.routes());
@@ -69,7 +70,7 @@ async function makeTestServer() {
const url = `http://127.0.0.1:${port}`;
const close = () =>
new Promise<void>((resolve, reject) =>
server.close((err) => (err ? reject(err) : resolve()))
server.close((err) => (err ? reject(err) : resolve())),
);
return { url, close };
}
@@ -107,11 +108,16 @@ describe("jsonBody middleware", () => {
});
it("failing validator → 400 jsonchema_validation_failed with schema and errors", async () => {
const res = await request(url + "/fail-validate", { body: { any: "thing" } });
const res = await request(url + "/fail-validate", {
body: { any: "thing" },
});
assert.equal(res.status, 400);
assert.equal((res.body as any).error, "jsonchema_validation_failed");
assert.ok((res.body as any).schema, "response includes schema");
assert.ok(Array.isArray((res.body as any).errors), "response includes errors");
assert.ok(
Array.isArray((res.body as any).errors),
"response includes errors",
);
});
it("includeParams: param absent from body → merged in", async () => {
+148
View File
@@ -0,0 +1,148 @@
import { after, before, describe, it } from "node:test";
import assert from "node:assert/strict";
import { createTestDb, type TestDb } from "../helpers/sqlite.js";
import { createTestServer, type TestServer } from "../helpers/koa.js";
import { hashPassword } from "../../src/util/hash.js";
import type { ClientUser } from "../../src/db/types/User.js";
const PASSWORD = "user-visibility-password";
function basic(email: string): string {
return `Basic ${Buffer.from(`${email}:${PASSWORD}`).toString("base64")}`;
}
describe("API user email visibility", () => {
let testDb: TestDb;
let server: TestServer;
let admin: ClientUser;
let normal: ClientUser;
let coResident: ClientUser;
let abodeAdmin: ClientUser;
let aid: string;
before(async () => {
testDb = await createTestDb();
server = await createTestServer(testDb.db);
const password = await hashPassword(PASSWORD);
admin = await testDb.db.createUser({
email: "global-admin@test.example",
name: "Global Admin",
password,
flags: { admin: true },
});
normal = await testDb.db.createUser({
email: "normal@test.example",
name: "Normal",
password,
flags: {},
});
coResident = await testDb.db.createUser({
email: "co-resident@test.example",
name: "Co-resident",
password,
flags: {},
});
abodeAdmin = await testDb.db.createUser({
email: "abode-admin@test.example",
name: "Abode Admin",
password,
flags: {},
});
const abode = await testDb.db.createAbode(
{ name: "Shared abode" },
{ uid: admin.uid },
);
aid = abode.aid;
for (const user of [normal, coResident]) {
await testDb.db.createResident(
{ uid: user.uid, aid, flags: {} },
{ uid: admin.uid },
);
}
await testDb.db.createResident(
{ uid: abodeAdmin.uid, aid, flags: { admin: true } },
{ uid: admin.uid },
);
});
after(async () => {
await server.close();
testDb.close();
});
it("shows a normal caller only their own email in GET /users", async () => {
const response = await fetch(`${server.url}/users`, {
headers: { Authorization: basic(normal.email) },
});
assert.equal(response.status, 200);
const users = (await response.json()) as ClientUser[];
assert.equal(
users.find((user) => user.uid === normal.uid)?.email,
normal.email,
);
assert.ok(!("email" in users.find((user) => user.uid === coResident.uid)!));
});
it("hides another user's email in GET /users/:uid", async () => {
const response = await fetch(`${server.url}/users/${coResident.uid}`, {
headers: { Authorization: basic(normal.email) },
});
assert.equal(response.status, 200);
assert.ok(!("email" in ((await response.json()) as object)));
});
it("allows global admins to see user emails", async () => {
const response = await fetch(`${server.url}/users`, {
headers: { Authorization: basic(admin.email) },
});
assert.equal(response.status, 200);
const users = (await response.json()) as ClientUser[];
assert.equal(
users.find((user) => user.uid === coResident.uid)?.email,
coResident.email,
);
});
it("makes lookup by email global-admin-only", async () => {
const denied = await fetch(
`${server.url}/users/by-email?email=${encodeURIComponent(coResident.email)}`,
{ headers: { Authorization: basic(normal.email) } },
);
assert.equal(denied.status, 401);
const allowed = await fetch(
`${server.url}/users/by-email?email=${encodeURIComponent(coResident.email)}`,
{ headers: { Authorization: basic(admin.email) } },
);
assert.equal(allowed.status, 200);
assert.equal(
((await allowed.json()) as ClientUser).email,
coResident.email,
);
});
it("shows co-resident emails to an abode admin", async () => {
const response = await fetch(`${server.url}/abodes/${aid}/users`, {
headers: { Authorization: basic(abodeAdmin.email) },
});
assert.equal(response.status, 200);
const users = (await response.json()) as ClientUser[];
assert.equal(
users.find((user) => user.uid === coResident.uid)?.email,
coResident.email,
);
});
it("hides co-resident emails from a non-admin resident", async () => {
const response = await fetch(`${server.url}/abodes/${aid}/users`, {
headers: { Authorization: basic(normal.email) },
});
assert.equal(response.status, 200);
const users = (await response.json()) as ClientUser[];
assert.equal(
users.find((user) => user.uid === normal.uid)?.email,
normal.email,
);
assert.ok(!("email" in users.find((user) => user.uid === coResident.uid)!));
});
});
+14 -14
View File
@@ -1,15 +1,15 @@
{
"compilerOptions": {
"rootDir": "src",
"strict": true,
"verbatimModuleSyntax": true,
"moduleResolution": "nodenext",
"module": "nodenext",
"target": "esnext",
"allowImportingTsExtensions": false,
"noEmit": true,
"sourceMap": true,
"jsx": "react-jsxdev"
},
"include": ["src/**/*.ts","src/**/*.tsx"]
}
"compilerOptions": {
"rootDir": "src",
"strict": true,
"verbatimModuleSyntax": true,
"moduleResolution": "nodenext",
"module": "nodenext",
"target": "esnext",
"allowImportingTsExtensions": false,
"noEmit": true,
"sourceMap": true,
"jsx": "react-jsxdev"
},
"include": ["src/**/*.ts", "src/**/*.tsx"]
}
+5 -5
View File
@@ -1,7 +1,7 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"rootDir": "."
},
"include": ["src/**/*.ts", "src/**/*.tsx", "test/**/*.ts"]
"extends": "./tsconfig.json",
"compilerOptions": {
"rootDir": "."
},
"include": ["src/**/*.ts", "src/**/*.tsx", "test/**/*.ts"]
}
+12 -12
View File
@@ -25,8 +25,8 @@ export default async (): Promise<Configuration[]> => {
process.env.DB_SOURCES === "dynamic"
? "dynamic"
: process.env.DB_SOURCES === "static"
? "static"
: "shared";
? "static"
: "shared";
let disableDbSqlite = process.env.DISABLE_DB_SQLITE === "1";
let disableBs3 = disableDbSqlite || process.env.DISABLE_BS3 === "1";
@@ -56,7 +56,7 @@ export default async (): Promise<Configuration[]> => {
(await readdir(file("./src/bin"))).map((bin) => [
bin.replace(/\..+$/, ""),
file(`./src/bin/${bin}`),
])
]),
);
for (const bin of Object.keys(binaries)) {
copies.push({
@@ -93,12 +93,12 @@ export default async (): Promise<Configuration[]> => {
} else if (dbSources === "static") {
console.log("Resolving db interfaces statically");
aliases[file("./src/db/dbSources.ts")] = file(
"./src/db/dbSources.static.ts"
"./src/db/dbSources.static.ts",
);
} else {
console.log("Resolving db interfaces shared");
aliases[file("./src/db/dbSources.ts")] = file(
"./src/db/dbSources.shared.ts"
"./src/db/dbSources.shared.ts",
);
}
@@ -111,21 +111,21 @@ export default async (): Promise<Configuration[]> => {
} else if (disableBs3) {
console.log("Disabling better-sqlite3 sqlite db backend");
aliases[file("./src/db/sqlite/impl/implementations.ts")] = file(
"./src/db/sqlite/impl/implementations.node.ts"
"./src/db/sqlite/impl/implementations.node.ts",
);
compiledSources.push("sqlite");
} else if (disableNodeSqlite) {
console.log("Disabling node:sqlite sqlite db backend");
aliases[file("./src/db/sqlite/impl/implementations.ts")] = file(
"./src/db/sqlite/impl/implementations.bs3.ts"
"./src/db/sqlite/impl/implementations.bs3.ts",
);
compiledSources.push("sqlite");
} else {
console.log(
"Enabling sqlite db interface with better-sqlite3 and node:sqlite backends"
"Enabling sqlite db interface with better-sqlite3 and node:sqlite backends",
);
aliases[file("./src/db/sqlite/impl/implementations.ts")] = file(
"./src/db/sqlite/impl/implementations.all.ts"
"./src/db/sqlite/impl/implementations.all.ts",
);
compiledSources.push("sqlite");
}
@@ -145,18 +145,18 @@ export default async (): Promise<Configuration[]> => {
defines.compiledSources = JSON.stringify(compiledSources);
for (const source of existingSources)
defines[`compiledSources.${source}`] = JSON.stringify(
compiledSources.includes(source)
compiledSources.includes(source),
);
if (!compiledSources.length) {
console.warn(
"No db interface enabled, the builds will be completely useless"
"No db interface enabled, the builds will be completely useless",
);
process.exit(1);
}
// log about the natives we have
console.log(
`Using ${Object.values(natives).filter(Boolean).length} natives:`
`Using ${Object.values(natives).filter(Boolean).length} natives:`,
);
for (const [key, path] of Object.entries(natives)) {
if (path) console.log(`- ${key}: ${path}`);