VA.2026.014 · Floor -1 · vault · hall 16
“a backup is not a file — it is a promise that can be checked.”
va-backup is the collection's insurance policy. It reads two live databases — Neon PostgreSQL and MongoDB — dumps both read-only, compresses them with zstd, hashes each with SHA-256, uploads to Cloudflare R2, verifies the upload, then writes a manifest back to GitHub. Nothing here writes to production. That single rule shapes every other decision in the machine: the primary database is chosen by a config flag, not a connection string guess, and consistency is checked before a single byte is dumped.
The engine is split by responsibility, not by feature. An external scheduler — cron-job.org — POSTs a trigger and leaves; the API only enqueues, never dumps; a separate worker process claims jobs with an atomic lock, holds a 30-second heartbeat, and executes the pipeline. A duplicate trigger with the same idempotency key returns 200 instead of enqueuing a second copy, and a worker that dies mid-job leaves a stale heartbeat that the next worker requeues. The API can live on Vercel with a 60-second budget; the worker cannot, so it runs on a container. The split is the design.
Before the backup, a consistency gate decides whether the backup is worth taking: PostgreSQL schema and index compared structurally against an expected-state snapshot, Mongo index inspected, both reduced to hashes, drift reported as PASS, WARNING, or FAIL. Only PASS and WARNING proceed — a FAIL stops the pipeline rather than preserving a broken picture. Restore is the same instinct reversed: it refuses to touch production by default, targets staging, then re-verifies schema hash, index hash, table counts, collection counts, and sample records before anyone calls the recovery finished. Every provider has a local filesystem mode, so the whole vault runs on a laptop with no credentials at all.
VA.2026.014
va-backup
Infrastructure Engine · Queue + Async Worker · Neon + Mongo → R2 → GitHub · Elysia + Bun
1,855 lines and 100 numbered sections written before implementation: five providers, one queue, one worker, and 31 closing design rules. Phases 1 through 7 drawn up front — infrastructure, core engine, backup, consistency, reliability, metadata, recovery.
consistency engine built before the dump logic it guards. schema hash, index hash, and drift detection decide whether a backup is allowed to exist; FAIL stops the pipeline rather than archiving a broken picture.
Neon and Mongo dumped read-only, primary first; zstd compression, SHA-256 checksum, R2 upload, then a four-step verification — object exists, size matches, metadata matches, checksum verified.
rule 88 enforced in deployment: the API ships to Vercel under a 60-second function budget, the worker runs on its own host. Restore, needing minutes and a zstd binary, moved out of serverless entirely.
the repository became a master template — one clone per project, each with its own BACKUP_PLATFORM slug and its own expected-state snapshot. A new deployment is a copy and a .env, not a new build.
151 tests across 38 spec files, with live suites against real Neon, Mongo, R2, and GitHub gated behind flags so the default suite stays green offline. Two CI workflows: verify on every push, live on manual dispatch.
concepts — kept & discarded
kept — one trigger, one queue entry, one worker; nobody waits on a backup
kept — consistency as a gate rather than a report
held — one clone per project over multi-tenant configuration
discarded — internal scheduler; cron-job.org stays the source of truth
archive ref — VA.2026.014/CON
cross ref — timeline fig. 01
curator annotations
no internal cron. the schedule lives in cron-job.org and in BACKUP_SCHEDULE, and if they disagree the manifest says so — two sources of truth beat one silent lie.
every provider has a filesystem twin behind STORAGE_DRIVER=local. the vault runs end to end on a laptop with zero credentials — the most useful test harness ever written.
retention may never remove the latest successful backup, a referenced backup, or one in use by a running restore. an expiry policy that can delete the only good copy is not a policy, it is a countdown.
| materials | 1,855-line implementation plan · Neon PostgreSQL + MongoDB sources · Cloudflare R2 artifact store · GitHub manifest repository · zstd compression + SHA-256 · Prisma queue store · OData V4 envelope · exponential backoff + heartbeat | dimensions | phases — 7 · design rules — 31 · internal endpoints — 6 · tests — 151 · artifacts per run — 2 (Neon · Mongo) · default retention — 30 days |
|---|---|---|---|
| condition | beta · 7 phases · 151 tests · 2 CI workflows | acquired | 2026 |
| revision | Rev. A | archive location | DWG-014 · floor Floor -1 · vault |
| internal reference | VA.2026.014/M | origin | drawn as a 1,855-line plan before a line of code, 2026 |
failed experiments
the first deployment put /internal/restore on Vercel. it needs minutes and a zstd binary and lost the race against the function timeout. restore moved to the worker host; only fast queries stayed serverless.
the single-platform model started as a limitation and became the design. no multi-tenancy on the engine; instead, one clone per project with its own platform slug and its own expected-state snapshot.
dumping first and checking later wastes storage on a picture already known to be wrong. the gate was moved in front of the pipeline, where it can still refuse.
draft concepts
research notes
— the API that schedules and the process that executes must be able to fail independently — coupling them couples their timeouts.
— verifying an upload after it succeeds is the difference between 'we sent it' and 'it is there'.
— restoring into staging by default removes the most dangerous operation in the system from the path of least resistance.
— a snapshot of expected schema and indexes turns drift detection into a diff, and diffs are reviewable.
The backup path is read-only by construction, not by discipline. Production databases are never written to, never locked, never used as a scratch space — the engine observes a running system and leaves it exactly as it found it.
A request that waits for a dump is a request that times out, and a timeout is indistinguishable from data loss. The API enqueues and returns 202; the worker owns the work, the heartbeat, and the retries.
A successful upload is a claim, not a fact. Objects are checked for existence, size, metadata, and checksum; restored data is checked for schema hash, index hash, counts, and samples. Nothing in this engine is trusted because it reported success.
within the collection
future vision
The vault widens: incremental dumps beside full ones, per-table restore, retention tiers for daily, weekly, and monthly artifacts, and alert routing that treats a stale backup as an incident rather than a metric. The direction never changes — the copy that exists is worthless until someone has proven they can bring it back.