SignalML

Problem

arXiv publishes hundreds of ML papers a day; nobody reads a firehose. The long-term product is a reader that surfaces what matters. But a reader is only as good as its data layer, so the build order was deliberate: the ingestion and classification pipeline first, with production-grade operational behavior, before a single pixel of UI. This case study covers that pipeline — the product surface on top is future work, and this page says so plainly.

A second, more personal goal: measure how far disciplined agent-assisted development can go. The entire working pipeline below went from written spec to deployed in one evening (~86 minutes of wall clock, two commits) — with the human doing architecture and review, and the agent doing hands.

Approach

Two Deno edge functions on Supabase, chained by pg_cron:

Ingestion (06:00 UTC). Pages the arXiv Atom API across five ML categories with polite backoff (3s between pages, 3-attempt exponential retry, 30s timeouts, an identifying User-Agent), normalizes arXiv IDs so paper revisions collapse onto one row, dedupes in memory, and lands everything in Postgres with a single batched upsert. The cron job’s URL and shared secret are resolved from Supabase Vault at execution time — secrets never appear in migration SQL.

Classification (06:30 UTC). Picks up pending papers and asks gpt-5-nano (Responses API) for structured flags — domain, innovation level, key architecture, topic tags — under a strict JSON schema, then re-validates every field by hand anyway: belt and braces against the refusal and truncation cases strict mode doesn’t cover. A prompt_cache_key is set and cached-token counts are extracted and summed per run — cost is instrumented, not estimated.

The design center is operational legibility. Both functions write to run-log tables (ingestion_runs, classification_runs) with a running → succeeded | failed state machine — the run row is inserted before any external I/O, so even a hard crash leaves a findable orphan. Each paper carries its own status machine (pending | processing | succeeded | failed, with attempt counts and error text), so one bad abstract can’t poison a batch. Both functions take a dryRun that shows exactly what would be selected without spending a token. Cron migrations are idempotent (unschedule-then-schedule). Indexes mirror the actual queries — the classifier’s selection has its exact composite index — rather than being sprinkled generically. RLS is on for every table, including deny-all policies on the operational logs so only the service role can touch them.

Result

What runs end to end today: cron fires → arXiv paged and upserted → classifier picks up pending rows → schema-validated flags land → both runs logged, with failure email via Resend. Deployed on Supabase + Vercel and exercised against production at least once.

What doesn’t exist yet, on purpose: everything a user would see. The feed, reader, auth, bookmarks (the schema is ready — bookmarks FKs auth.users with owner-scoped RLS) are phases 3–5. There is no ranking or embedding layer yet either; today “signal” means one LLM-judged categorical per paper.

An honest audit of the fast build, recorded here because it’s the actual lesson: the edge functions ship under @ts-nocheck (1,200 untypechecked lines — the price of that evening), classification is serial and won’t finish a 200-paper batch inside an edge-function budget, papers orphaned in processing are never reaped, and there are no tests. That’s the trade agent-speed bought, and it’s the fix list for phase 3 — written down before any feed work begins.