A single generated Google Colab notebook that restores, discovers, downloads, extracts, and verifies the full Kentucky appellate court document corpus (~98.7k documents). Built to survive Colab disconnects — it auto-resumes, is idempotent, and mirrors all state to Google Drive so nothing is lost between sessions.

What it does

ky_court_pipeline.ipynb walks the corpus through six stages:

restore → discover → backfill → download → extract → sync/verify

Every stage reads/writes one SQLite DB and a Store abstraction (Google Drive in Colab, local filesystem in tests). Any stage stops gracefully at the session time budget; re-running resumes exactly where it left off.

Highlights

Resumable — recover() heals stale state; each stage skips what's done.
Idempotent — safe to re-run as often as you like.
Drive-backed — SQLite DB, deduped PDFs, extracted text, exports, and a live status.json heartbeat all live under MyDrive/ky_court-WIP/.
Never auto-deletes on Drive — original backups are retained.
Polite + fault-tolerant — paced API calls, exponential backoff, capped retries, and a DRY_RUN mode for fully offline exercise.

Layout

Path	Purpose
`ky_court_pipeline.ipynb`	the deliverable (generated notebook)
`notebook_builder/`	sources: `build.py` + one module per cell (`c00_config` … `c90_verify`)
`tests/`	offline pytest suite (18 tests; live ones gated behind `KY_LIVE=1`)
`scripts/smoke_local.py`	full offline pipeline over the real DB
`RUNBOOK.md`	full operational runbook (first run, resuming, failure modes, config knobs)

Quick start

# local test/smoke
python3 -m venv .venv && .venv/bin/pip install -r <pinned deps>
.venv/bin/pytest tests -q
.venv/bin/python scripts/smoke_local.py
.venv/bin/python notebook_builder/build.py   # regenerate the notebook

# on Google Colab
# upload ky_court-WIP-resume-*.tar.gz to MyDrive/ (if Drive is empty),
# open ky_court_pipeline.ipynb, Runtime > Run all.

See RUNBOOK.md for the full runbook — first run, resuming, what "done" looks like, failure modes, and every config knob.

Note: input data (extracted archives, resume tarballs, _WIP backups) is deliberately not in this repo — it lives in Google Drive / locally.