Nodus
All field notes
Field note001 Filed19 Apr 2026 TagControl loop Read10 min ByPranav Varshney · Viswa Kotra

Five systems, one control loop.

Nodus did not invent market making, ad auctions, graph routing, generation reconciler, or durable checkpointing. We borrowed each from places that already solved them under load, then joined them so an AI workload can finish when capacity disappears.

The product is the join

Customers submit outcomes: budget, deadline, workload signals, continuity mode. They do not pick a machine SKU, a cloud catalog row, or a recovery plan. Nodus derives provider, route, reservation, execution, and portable progress from that brief.

That derivation is not one clever service. It is five boundaries with hard contracts between them:

  • Market: continuous normalized capacity book (quant inventory, not a static catalog).
  • Routing: pure optimizer over snapshots (dispatch graph, not a provider SDK call).
  • Bidding: auditable capacity commitment (campaign budget / auction, not hope).
  • Execution: desired vs observed generations (controller, not “VM created”).
  • Checkpointing: verified portable progress (GKE-style durability, not local disk).

Anyone can bolt five subsystems onto a slide. The engineering is making reclaim re-enter the same path without inventing a sixth one.

Happy path, then the loop

Nodus control loop Profile and route consume market snapshots, bidding reserves, execution runs, checkpoint commits concurrently, reclaim re-enters market and route. Profile envelope Route + ceiling Reserve lease Execute gen N Settle ledger Market snapshots (async) Checkpoint (concurrent) reclaim / restore
Market runs continuously. Checkpointing runs with the workload. Reclaim does not invent a parallel stack; it re-enters route from verified progress.

Authority, on purpose

Three durable systems, intentionally separate:

  • PostgreSQL: business truth: workload, route, lease, generation fence, latest-manifest CAS, ledger.
  • Temporal: orchestration progress: timers, retries, waits, recovery saga. Never customer-visible state.
  • Kafka: published facts / transport. Never assumed atomic with DB or provider side effects.

Every customer-visible transition writes Postgres state and an outbox row in one transaction. Consumers are idempotent and at-least-once. Generation fencing and latest-manifest compare-and-set live in Postgres (plus generation-scoped storage credentials), never Redis locks, never a separate lock service.

Control-plane services are Go on EKS. Customer SDKs are Python and Go. Object storage holds checkpoints and artifacts. Redis is for ephemeral limits only.

Invariants that force the join

  • Router is pure. Reads immutable snapshots; returns a plan. Never calls providers. Reservation is a separate, auditable side effect.
  • Cost means completion cost. Routing and bidding optimize expected cost to finish (run plus expected recovery), not minimum hourly price.
  • Completed means durable. Never report completed until the final artifact manifest is verified and the CAS pointer advances (when the continuity mode requires it).
  • At most one writer. Fence before replace. Old runners lose CAS authority immediately.

What this series covers

Each following note opens one boundary: what we took from an existing architecture, what we changed for GPU reclaim, and what we are still building.

Next after the five: fencing and the reclaim saga, then multi-stage DAGs with manifest-only handoffs.

What we are not claiming yet

Design and pilot are not the same maturity. External wholesale adapters, parallel hedging, and fully automatic signed reclaim are phase-gated. We publish the contracts now so every shipping decision is made under the assumption that reclaim, settlement, and replay will exist later. You cannot retrofit those boundaries. You can only plan for them.

All field notes