Firstpass

First review in seconds, before a human looks.

Overview

A working GitHub App prototype where opening a pull request gets an automatic summary comment within seconds — signed webhooks, an idempotent Postgres job queue, and a worker that claims jobs with FOR UPDATE SKIP LOCKED. Deterministic first, no LLM spend: the intelligence layer has a reserved home in core/llm but v1 proves every integration point with plain diff summaries.

Role: Backend · bots + queues
Year: 2026
Stack: Python · FastAPI · Postgres 16 · Docker Compose · GitHub Apps · pytest
Source: GitHub ↗

Proving the loop before the intelligence

Waiting on first review is dead time: a developer opens a pull request and then waits while obvious context sits one API call away. Most review bots reach for the model first and treat delivery as plumbing; I sequenced this one the other way — milestones 1 and 2 prove the full loop with a hardcoded comment so that webhook verification, storage, queueing, and commenting each earn trust before any diff or model is allowed in.

Postgres as the queue

Postgres is the queue because Postgres is already there: webhook_events plus a jobs table, idempotent on delivery ID so GitHub redeliveries are absorbed, claimed with FOR UPDATE SKIP LOCKED and exponential backoff. The milestone 7 hardening pass treats duplicate and stale comments as the product defect they are — payload validation, retry jitter, a reaper for jobs stuck running after a crash, supersede-cancel on new pushes, and editing the bot's comment in place instead of stacking duplicates.

Firstpass pipeline board showing the review comment mock, Postgres queue query, job lifecycle, and deterministic pipeline before the model — The full loop — webhook to queue to comment, model last

A deterministic v1 with the model seams reserved

Firstpass architecture diagram from GitHub events through FastAPI and Postgres to workers and PR comments — GitHub events to AI review via Postgres queue

Milestone 3 fetches the diff through the Files API and posts a deterministic summary — per-file additions and removals plus the short hash, truncated gracefully on large PRs. Static analyzers, the LLM pass behind core/llm, inline line-mapping, and the dashboard stay explicitly deferred with reserved seams, gated on real decisions: analyzer set, provider and budget, hosting. Useful the day the loop works, not the day the model arrives.

Structure

01
Signed GitHub webhook
02
Idempotent event store
03
Postgres job queue
04
SKIP LOCKED worker
05
Diff summary comment

The result

The loop works end to end on a local Docker Compose stack — webhook in, summary comment on the PR within seconds — guarded by 16 pytest gates covering signatures, diff rendering, and queue hardening with mocked GitHub calls. The repo carries its own memory: a PROJECT.md that scopes v1 as deterministic-only under a free-only constraint, ADRs for the stack choices, and a risk ledger that names the known weak spots — init-only schema mounts, uncached per-job tokens — instead of letting them hide.

PR comment in seconds

16 pytest gates

idempotent webhook queue

If I built it again

Integration-first sequencing was the whole bet: a review bot that drops or duplicates comments is worse than none, so reliability came before intelligence. Next is milestone 4's static analyzers, then the milestone 5 model pass — but only after the provider and budget decision the free-only constraint demands.