braindump/README.md

# braindump

A headless server that runs brain-dump note-taking sessions into an Obsidian-style vault, exposing a web API meant to be consumed by a web UI or Obsidian plugin. An LLM agent interviews you conversationally and writes linked markdown notes into the vault as the conversation progresses, following the `brain-dump` skill (incremental writes, 1–3 questions per message, no extrapolation, TODO markers, `projects/<name>/` layout with wikilinks).

Two interchangeable LLM backends, selected per session:

- **`claude-sdk`** — the [Claude Agent SDK](https://www.npmjs.com/package/@anthropic-ai/claude-agent-sdk). Gets file tools, skill loading (`~/.claude/skills/brain-dump`), and session resume from the SDK. Uses your Anthropic credentials.
- **`openai-compat`** — any OpenAI-compatible chat-completions endpoint (openrouter, vllm, ollama). A built-in agent loop supplies vault-scoped file tools and injects the skill into the system prompt.

## Requirements

- Node ≥ 22 (uses `node:sqlite`)
- For `claude-sdk`: Anthropic credentials (the SDK inherits Claude Code auth or `ANTHROPIC_API_KEY`)

## Setup

```sh
npm install
cp config.example.jsonc config.jsonc   # edit: vaults, backends, models
npm run web                            # boots, auto-migrates the db, listens
```

Config is JSONC; path from `BRAINDUMP_CONFIG` (default `./config.jsonc`). `"${ENV_VAR}"` string values are resolved from the environment; `~` is expanded in paths. Only vaults listed in `vaults` are writable — that allow-list is the write fence.

> **Note:** an exported `PORT` env var overrides the config file's `port` (watch out under code-server, which exports `PORT` globally). `BRAINDUMP_DB` overrides `dbPath`.

There is **no auth** — put it behind an authenticating reverse proxy. CORS origins are configurable (`app://obsidian.md` is in the example for Obsidian plugins).

## Deployment (Docker)

```sh
docker build -t braindump .
```

The image expects three mounts (paths inside the container are up to you; these match the defaults baked into the image via `BRAINDUMP_CONFIG=/config/config.jsonc` and `BRAINDUMP_DB=/data/braindump.sqlite`):

| Mount | Purpose |
|---|---|
| `/config/config.jsonc` | your config — point `vaults[].path` at the container-side vault paths |
| `/data` | persistent volume for the SQLite db |
| `/vaults/...` | your vault(s), read-write |

Backend-specific requirements:

- **`claude-sdk`**: pass `ANTHROPIC_API_KEY`, and mount the brain-dump skill into the container user's home so the SDK's skill loading finds it: `-v ~/.claude/skills/brain-dump:/home/braindump/.claude/skills/brain-dump:ro`.
- **`openai-compat`**: mount the skill file wherever `skillPath` in your config points (e.g. `/config/SKILL.md`), and pass any `${ENV_VAR}` referenced by the config (e.g. `OPENROUTER_API_KEY`).

Example `compose.yaml`:

```yaml
services:
  braindump:
    build: .   # or image: gitea.codinget.me/codinget/braindump
    ports: ["3000:3000"]
    environment:
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
    volumes:
      - ./config.jsonc:/config/config.jsonc:ro
      - braindump-data:/data
      - ~/git/brain:/vaults/brain
      - ~/.claude/skills/brain-dump:/home/braindump/.claude/skills/brain-dump:ro
volumes:
  braindump-data:
```

with the corresponding config using container paths: `"vaults": [{ "name": "brain", "path": "/vaults/brain" }]` and `"skillPath": "/home/braindump/.claude/skills/brain-dump/SKILL.md"`.

Notes:

- The container runs as a non-root `braindump` user; git `safe.directory` is pre-configured system-wide so the read-only git endpoints work on vaults owned by your host uid. Vault *writes* need the mounted vault to be writable by the container user (`chmod`/`chown` or a matching uid via `user:`).
- The healthcheck and config default both use port 3000; if you change the port, change both (`PORT` env works too, and overrides the config).
- **Reverse proxy + SSE**: disable response buffering for `/api/sessions/*/messages` (nginx: `proxy_buffering off;`, or honor the `X-Accel-Buffering: no` pattern) and raise/disable the proxy read timeout for long turns — heartbeat comments are sent every ~15s to keep the connection alive. This is also where your auth goes.

## API

Base path `/api`. Errors are `{"ok":false,"error":"<code>"}`.

| Method | Path | Description |
|---|---|---|
| GET | `/health` | liveness |
| GET | `/vaults` | configured vaults |
| POST | `/sessions` | create session — `{vault, topic, backend?, model?}` |
| GET | `/sessions[?status=]` | list sessions |
| GET | `/sessions/:id` | session + transcript |
| PATCH | `/sessions/:id` | update `{topic?, status?}` |
| DELETE | `/sessions/:id[?hard=true]` | archive (soft) or delete row; never touches vault files |
| POST | `/sessions/:id/messages` | send a user message — **response is an SSE stream** for the turn |
| GET | `/sessions/:id/file?path=` | read a vault file (path-validated) |
| GET | `/sessions/:id/git/status` | `git status --porcelain`, parsed |
| GET | `/sessions/:id/git/diff[?path=]` | raw diff (read-only; commits are always left to you) |

### The turn stream

`POST /sessions/:id/messages` with `{"text": "..."}` responds with `text/event-stream`:

```
event: text               data: {"delta":"What's the goal of the project?"}
event: tool_start         data: {"tool":"write_file","path":"projects/x/x.md"}
event: file_write         data: {"path":"projects/x/x.md","bytes":412}
event: tool_end           data: {"tool":"write_file","ok":true}
event: assistant_message  data: {"text":"...full assistant text..."}
event: turn_complete      data: {"turnCount":1}
```

Consume with `fetch` + a ReadableStream reader (or `curl -N`). One turn at a time per session — a concurrent POST gets `409 turn_in_progress`. Closing the connection aborts the turn; files already written stay (writes are incremental by design). Heartbeat comments keep proxies from timing out.

## Curl walkthrough

```sh
# create a session
SID=$(curl -s localhost:3000/api/sessions -H 'content-type: application/json' \
  -d '{"vault":"brain","topic":"my-project","backend":"claude-sdk"}' | jq -r .session.id)

# first turn — watch text stream and files land
curl -N localhost:3000/api/sessions/$SID/messages -H 'content-type: application/json' \
  -d '{"text":"Brain dump my-project: a CLI tool that does X, written in Go, status: early prototype."}'

# what changed in the vault?
curl -s localhost:3000/api/sessions/$SID/git/status | jq
curl -s "localhost:3000/api/sessions/$SID/file?path=projects/my-project/my-project.md" | jq -r .content

# keep going — the agent asks 1-3 questions per turn and extends the notes
curl -N localhost:3000/api/sessions/$SID/messages -H 'content-type: application/json' \
  -d '{"text":"It also has a TUI mode. TODO on my side: pick a license."}'

# review + commit yourself when happy (braindump never commits)
git -C ~/git/brain status
```

## Development

```sh
npm run typecheck
npm test                        # pathSafety + Db unit tests
PORT=11500 npx tsx scripts/mock-openai.ts   # deterministic mock chat-completions server
```

`scripts/mock-openai.ts` scripts a full tool-calling session (including a vault-escape probe when the user message contains `ESCAPE`), so the whole openai-compat path can be tested end-to-end without a real model — point `backends.openai-compat.baseUrl` at it.

### Layout

- `src/agent/` — `AgentBackend` interface + the two adapters (`claudeSdk/`, `openaiCompat/`) and the stub
- `src/vault/` — vault allow-list resolution, `pathSafety` (symlink-aware escape prevention), read-only git helpers
- `src/webapi/` — Koa router, SSE plumbing, middleware
- `src/db/` — `node:sqlite` wrapper + migrations (sessions, messages, turn_events)

Backends own message persistence: `openai-compat` stores its full replayable history in SQLite; `claude-sdk` stores display rows and resumes via the SDK's own transcript (`sdk_session_id`).