pwned

A small clone of the HaveIBeenPwned Pwned Passwords API, built for local development and experimentation. It implements the same k-anonymity range API (GET /range/{5-char-hash-prefix}), a thin React client to test it against, and tooling to populate the database from a wordlist, a hash list, or the real HIBP range API.

Note: This project was built with Claude Code (Sonnet 5), from initial scaffolding through the performance work on the bulk importer. It has not been used in production and has not had a from-scratch human security review; treat it as a learning/testing project rather than something to deploy as-is.

Layout

This is an npm workspaces monorepo:

• packages/backend — Koa + @koa/router + pg. Serves GET /range/:prefix, matching the real HIBP API's response shape.
• packages/frontend — React + wouter, bundled with webpack. A thin client that hashes a password locally (SHA-1) and only sends the hash's first 5 characters to the server, same as the real HIBP API's privacy model.
• packages/import — CLI tooling to populate pwned_passwords:
  • index.ts — incremental import of a wordlist or hash list, with optional worker-thread/concurrency parallelism.
  • bulkImport.ts — a much faster full-table rebuild via COPY + deferred index build, for large files (see "Import performance" below).
  • importHibp.ts — pulls the full dataset from the real HIBP range API.

Getting started

npm install
cp .env.example .env.local   # edit if needed
npm run db:up                # starts a local Postgres via docker compose
npm run -w backend migrate   # creates the pwned_passwords table
npm run db:seed              # optional: a handful of well-known weak passwords
npm run dev                  # starts backend (:3001) + frontend (:3000)

Point at an externally hosted Postgres instead of the local compose one by setting DATABASE_URL in .env.local (takes precedence over the discrete PG* vars).

Importing data

# incremental, from a plaintext wordlist or a HASH[:COUNT] file
npm run import -- <file> --format=wordlist --jobs=8
npm run import -- <file> --format=hashes

# full rebuild, much faster for large files (see below)
npm run import:bulk -- <file> --format=wordlist --jobs=8

# pull the real dataset from api.pwnedpasswords.com
npm run import:hibp

Import performance

The incremental importer (index.ts) upserts in batches and is fine for smaller files or topping up an existing table. For large files (e.g. rockyou, ~14.3M lines), it hits real Postgres limits: random-order B-tree maintenance on a table this size becomes commit- and I/O-bound rather than CPU-bound.

bulkImport.ts avoids this by loading through COPY into an unindexed shadow table, merging it with the existing data in one pass, and only building the primary key/index once at the end (a sorted bulk build, instead of ~14M random-order incremental ones). The whole run is a single transaction, so Postgres can skip WAL-logging the new table's data entirely, and a failure at any point rolls back cleanly instead of leaving a half-migrated table.

Measured on the same file/database across the course of this project's development: 42 minutes → 24.7 minutes → ~7.3 minutes → 186 seconds wall clock, as (respectively) batching increased and commits became async, the whole operation became one WAL-skipping transaction, and wordlist hashing moved to a worker-thread pool.

Development

• npm run dev — backend + frontend with live reload
• npm run build — production builds
• npm run db:up / npm run db:down — local Postgres via docker compose (dev only)