A project-management case study. One person owned scope, schedule, risk, and quality for an autonomous market-making system — and directed an AI engineering agent as the delivery resource. Every claim on this page traces to a git commit or a document in the project repository.
§1 · The project
The product: an automated market-making system for a prediction-market venue (Polymarket) — live market-data feeds, a quoting strategy, a risk layer with a hardware-style kill path, a paper-trading simulator, cloud deployment, full observability, and a market-discovery scanner. It runs today as six concurrent instances plus a scanner on AWS, quoting real order books with simulated capital.
The constraints: a single owner with finite session time; a domain where silent failure costs money (the north-star requirement was written into the project charter: survive venue misbehavior without silent degradation); and a delivery resource — an AI engineering agent — that is tireless, fast, and confidently wrong just often enough that nothing it reports can be accepted without evidence. That last constraint shaped the entire management system.
This project treats AI-assisted delivery as a management problem, and the controls look like classic project controls because they are:
§2 · The management system
Six rungs from "collect data" to "multi-venue scale," with an anti-drift rule: no rung starts before the previous one's evidence exists. Scope pressure gets answered with the ladder, not debate.
source: project charter, north-star §Every roadmap item closes with a commit hash and date written inline — [x] 9a5e33e 2026-07-08. Completion is a pointer into git history, never a memory.
source: implementation-roadmap.mdDeferring a decision is allowed only with a named re-review trigger ("Gate: first calibration that needs adjusted-mid precision"). Vague deadlines rot; triggers fire.
source: deferred-decisions.md, 11 sectionsOne handoff document, revised 85 times, holds the current state, the ordered next actions, and the operational quick-reference. Any session — or any successor — can cold-start from it.
source: Handoff_Prompt.md git historyExternal-system claims must cite a verified document or a source fetched that session. This discipline overturned four design assumptions in a single verification pass — before they became code.
source: docs/notes/*-verified.md (7 documents)§3 · Timeline
Click a phase to expand its record.
Domain research reports, an authoritative architecture document, and — as important — the working conventions: test-driven development, the separation-of-duties rule, the decision register, and two-pass design walkthroughs. The management system was designed before the system was.
Market-data ingestion (websocket feeds with watchdogs and fail-loud semantics), storage, and observability scaffolding. The feeds layer closed with a five-phase completion review rather than a smoke test — a pattern repeated at every layer boundary.
The quoting strategy, a paper-trading simulator, and a time-semantics audit that produced nine findings before they became production bugs. Then the centerpiece: a seven-phase risk layer — inventory caps, drawdown controls, infrastructure health gates, a circuit breaker, and an OS-thread watchdog — delivered in twelve days with all 19 safety invariants pinned by tests.
Deployed to AWS under systemd. Launch week surfaced four bugs the 700-test green suite had missed — an empty runtime registry, no graceful shutdown, a config gap, a data-capture overwrite. All four were fixed with regression tests the same week, and the lesson ("live operation is a verification layer, not a victory lap") entered the record.
The quoted market died overnight (its underlying event resolved); the bot went down for 18 hours and nothing paged, because alerting didn't exist yet. The milestone review that followed found something bigger: a configuration parameter was 20× too aggressive, silently zeroing earnings for ~85% of the collection window. The response was textbook: root-cause memo, fleet retune, relaunch on a longer-lived market, and alerting promoted to the next milestone.
Dashboards and alerting shipped as code (Prometheus + Grafana, provisioned from the repo), then every alert rule was deliberately live-fired to a pager — no rule is trusted until it has actually paged. A pre-registered data-quality discriminator caught a subtle feed bug (crossed order books) that had two root causes; both were fixed with regression tests. A stale-process incident produced the project's most durable rule: a deploy isn't deployed until the running process proves it — enforced afterward by a build-version gauge in code. The fleet expanded from one market to six.
Three commits all month — by design. The fleet ran alone while management shifted to operational review. The multi-week review found that reward pools oscillate violently (vindicating the register's "no retirement without multi-day dwell evidence" rule — a snapshot-based call would have retired the fleet's largest earner right before it recovered) and that crash-restarts wiped simulated position state, which became the next quarter's first fix.
Position persistence closed the crash-wipe gap. Two primary-source verification passes (fees, venue APIs) overturned four design assumptions before implementation. A market-discovery scanner was built and censused 16,954 funded markets on its first production-scale run. Then the payoff: when a scheduled market swap collided with a same-day market resolution, a depth-based ranking — run ahead of schedule — showed the staged replacement plan was no longer viable and that the competitive landscape had shifted fleet-wide. The swap was re-decided on fresh evidence and deployed the same day with zero downtime for the remaining fleet.
§4 · The numbers
tests passing at each pinned count · Jun 28 – Sep 15, 2026
Counts as recorded in commit messages at each milestone. The suite is a delivery gate: no session closes red. Flat stretch in August = unattended-operation phase.
211 total · the shape of a phased delivery
August's near-silence is the point: the system ran unattended while management shifted to operational reviews. September is lean because sessions grew larger, not slower.
layered verification: each net catches what the others miss
Conservative counts from session records. The tail matters most: the four found by data forensics — including a mis-calibration silently zeroing revenue — were invisible to every earlier layer.
§5 · Exhibits
Excerpts from the project's real management documents, lightly condensed.
excerpt · docs/implementation-roadmap.md · convention: an item is closed by writing its commit hash inline
| Item | Scope | Gate evidence | Status |
|---|---|---|---|
| Risk layer P1–P7 | 19 safety invariants, kill path, watchdog | 75eb023 · 2026-06-29 | SHIPPED |
| Production launch | AWS + systemd, first paper market | 2026-07-05 | SHIPPED |
| S1 replay fix | hard edge: must land before calibration | 9a5e33e · 2026-07-08 | SHIPPED |
| S2 telemetry | 14 metric families, liveness-aware | b92a499 · 2026-07-10 | SHIPPED |
| S4 dashboards + alerting | provisioned as code; rules live-fired | 539828d · 2026-07-18 | SHIPPED |
| S7 alert test-fire | every rule proven to page, on schedule | 2026-07-30 | SHIPPED |
| S9b position persistence | crash-wipe fix; gated before calibration | 2026-09-10 | SHIPPED |
| S12 scanner phase 1b | 16,954-market census, first ranking | 88b5b52 · 2026-09-15 | SHIPPED |
| S13 alert dispatch + depth | makes rankings depth-real, pages humans | — | NEXT |
excerpt · docs/deferred-decisions.md · rule: "every deferred decision names a concrete Gate event; vague deadlines rot silently"
| Decision | Call | Re-review trigger (Gate) | Outcome so far |
|---|---|---|---|
| Retire an underperforming market? | Only on multi-day dwell evidence, never a snapshot | pool history available in telemetry | Prevented retiring the fleet's largest earner during a temporary collapse; vindicated twice more since |
| Arbitrage strategy (new revenue line) | Parked — high value, wrong order | behind scanner infra (S9–S13) | Resisted a mid-session scope pull; still correctly parked |
| Fully automatic market retirement | Keep a human confirm step | ≥5 assisted cycles where the confirm added nothing | Still assisted — the asymmetry (a wrong remove destroys standing income) justifies the friction |
| Pager burst-loss fix | Deferred with documented risk | first real multi-instance fire, or fleet ≥10 | Open, with the failure mode written down where the on-call will find it |
The register's best moment: an early snapshot said a market's reward pool had died — the obvious call was to retire it. The dwell rule forced a wait. The pool recovered to full size within two weeks. The rule now carries its own evidence.
condensed · 2026-07-30 · full record in the ops guide, invariant #7
| When | Event |
|---|---|
| 07-28 19:22 | Primary instance (re)starts — on the code available at that moment |
| 07-29 14:57 | Critical data-integrity fix is pulled to the host; deploy record written as "deployed + verified" |
| 07-30 | A routine drift check reads zero where a healthy fix produces signal. Three independent evidence lines converge: no network calls from the process where the fix makes them; no fix-specific log lines; a metric added by the fix absent for exactly this instance |
| 07-30 15:05 | One restart; all three signals go green within minutes. Exposure: 43 hours of stale code on the exact market the fix targeted |
Root cause (process, not code): verification checked the artifact on disk, not the running process. Five sibling instances started after the pull and ran the fix — aggregate fleet health masked the one stale member.
Corrective actions, layered:
The class of lesson, as recorded: "Deployed + verified" that checks the artifact but not the runtime is a claim, not a fact.
structure excerpt · docs/Handoff_Prompt.md · 85 revisions over the project's life
Two details worth noticing: the next actions carry their own commands (the 2 a.m. rule — no future reader should have to re-derive an operational step), and the report instructs its own maintenance. The process survives context loss because it is written to.
§6 · What the record demonstrates
| Competency | Where the record shows it |
|---|---|
| Scope & schedule control | Stage-gate ladder + gate-tagged roadmap; an attractive new revenue line parked mid-session because it sat on the wrong rung — and stayed parked |
| Risk management | 19 test-pinned safety invariants; a decision register where every deferral carries a trigger; alert rules proven by live fire, not by review |
| Quality management | Layered verification (tests → smoke → production → forensics), each layer's catches counted; no session closes with a red suite |
| Incident management | Every incident closed with a postmortem, and every lesson routed to at least three audiences — including code, the only audience that never forgets to read |
| Evidence-based decisions | Two major pivots (market strategy, replacement plan) executed on same-day data that contradicted the staged plan; the dwell rule that prevented a wrong retirement |
| Communication & continuity | 85 revisions of a self-perpetuating status report; cold-start reading lists; 28 research documents that cite primary sources |
| Managing AI-augmented delivery | Separation of duties, verification-before-claims, and knowledge routing — controls designed for a resource that is fast, tireless, and requires audit |
§7 · Retrospective