Architecture
QuantumLane is a single-box data platform that demonstrates production patterns at the smallest scale that still requires real engineering. The full architecture document with ADRs lives in the repo; what follows is the summary.
Principles
- Boring tech that runs forever beats novel tech that runs for a month.
- Observability is a first-class feature, not an afterthought.
- Document the trade-off, not the tool.
- Schema is contract. Forward-only migrations, never
ALTER TABLEin psql. - Local dev = production in a smaller box.
- Cost discipline is part of the design. Target: under CAD $20/month all-in.
Stack
| Layer | Choice | Why |
|---|---|---|
| Orchestration | Dagster | Asset-centric model; lighter than Airflow at single-box scale. |
| Database | PostgreSQL 16 + PostGIS | One store at this scale; best-in-class geospatial. |
| Object storage | Cloudflare R2 | Free egress matters for public dataset publishing. |
| API | FastAPI + Pydantic v2 | Async, auto OpenAPI, modern validation. |
| Website | Plain HTML + Tailwind CDN | 4 pages, monthly updates — no build system needed. |
| Reverse proxy | Caddy | Automatic TLS, dead-simple config. |
| Host | Hetzner CAX21 (~CAD $11/mo) | 10x cheaper than AWS for the same patterns. |
What's in v0.1
- TTC GTFS-RT ingestion: vehicle positions (every 30s), trip updates (every 30s), service alerts (every 5m).
- Static GTFS daily refresh.
- PostgreSQL 16 + PostGIS, daily-partitioned realtime tables, retention to 30 days.
- FastAPI public API with rate limiting.
- Per-feed freshness telemetry surfaced on this site.
- Schema-drift detection via field-population signatures.
- Daily parquet exports to R2 (cold storage).
What's deliberately not in v0.1
- Other agencies (GO, MiWay, Brampton) — coming in v0.2 with the schema heterogeneity story.
- Streaming framework (Kafka, etc.) — overkill at <100 msg/sec.
- ML / forecasting — needs more historical data first.
- Authentication — public data, public API.
- Pretty map UIs — not the point.
Cost
Under CAD $15/month all-in: one Hetzner CAX21 ARM box (~$11), Cloudflare R2 free tier (10 GB), Cloudflare DNS free, GitHub Actions free, domain ~$15/year.