metrics-server-dashboard

Problem

I wanted a self-hosted “Mission Control” for my machines: host and Go-runtime vitals streaming live into a dense, dark, single-screen dashboard, with short-term history, multi-app support, and an agent SDK small enough to drop into any Go service. And I wanted to build it spec-first: the full architecture document — wire protocol, endpoint table, a class diagram — and seven UI mockups were finished and committed before the first line of implementation. The build itself took four days.

Approach

Collection. A 1 Hz collector samples the host via gopsutil (CPU total and per-core, memory, disk I/O, network) and the Go runtime (goroutines, heap, stack, GC pauses). Network throughput is derived, not sampled — a stateful delta across ticks. Remote services push their own runtime metrics through a tiny agent SDK that imports nothing outside the standard library — the genuinely embeddable piece.

Fan-out. A central hub goroutine owns all state and multiplexes three separate channels for metrics, logs, and app-registry updates — a noisy log stream can never crowd the metrics path. Each browser connection gets a read/write goroutine pair with a 256-message buffer, ping/pong keepalives, and the load-shedding rule that defines the system’s behavior under stress: sends to slow consumers are non-blocking — a laggy tab silently drops frames rather than stalling everyone else.

Memory is bounded at every layer, by design: 256-message client queues → a 600-snapshot archive buffer (flushed every 10 s without holding the lock across I/O) → a 720-snapshot in-memory store (~12 minutes at 1 Hz, with a Firestore provider behind the same interface) → 60 points per on-screen series → 100 log lines client-side. Nothing in the pipeline can grow without limit.

Density over decoration. The frontend is templ-rendered HTML with one Alpine store as the single source of truth and ApexCharts with animations disabled — at 1 Hz, animation queues are backlog. The CPU chart adapts its form to the data: five or fewer visible cores render as lines with a total and a max-core trace; more than five switches to a per-core heatmap — the right encoding at each density, chosen automatically.

Result

The pipeline works end to end: collection → hub → WebSocket → live charts, with history hydration on connect, multi-app switching, pause, and a filterable log console. Update rate 1 Hz, ~12 minutes of hot history, reconnects with capped exponential backoff.

Writing this case study forced a re-audit of the code, and the audit is part of the result. It found: the ingestion endpoints ship unauthenticated while the read endpoints require a token; a cross-goroutine data race on the per-client subscription filter; a README claiming HTMX that the build never shipped (a scaffold leftover — the real design, one Alpine store fed by a socket, is the better fit for stateful 1 Hz charts and now the documented one); and four panels padding real data with placeholder values. Those are now the fix list, in the open. A dashboard that exists to tell you the truth about your servers should hold itself to the same standard.