tracker: Auction delta sync

[squadgazzz]

Description

This is a tracker for the Delta Sync feature, which is one of the prerequisites for the Unified Auction epic.

Problem

Currently, the autopilot broadcasts a full orderbook snapshot to solvers every auction cycle. On mainnet alone, >90% of the payload is duplicated between consecutive auctions (mostly static limit orders). As we move toward a unified mega-auction across all chains, this payload grows linearly with the number of supported networks, making it the single biggest scalability bottleneck.

Delta sync replaces full snapshots with incremental updates: only new, removed, and changed orders are sent each cycle. Solvers maintain a local replica and receive full snapshots only on initial connection or resync.

Sub-issues

• Refactor SolvableOrdersCache::update() to emit structured change events (e.g., OrderAdded, OrderRemoved, OrderUpdated, PriceChanged) - use internally first to speed up the autopilot's own cache rebuild.
• Expose change events to the driver (probably, via a streaming endpoint)
• Driver maintains its own order cache, still forwards full auctions to solvers until they support the new approach
• Snapshot endpoint for newly connected solvers to bootstrap their state

[ashleychandy]

Hey team, I've been looking into this and wanted to propose a fix for the delta sync bottleneck. Here's what I have in mind:

Proposed Approach

Instead of sending full orderbook snapshots every cycle, we compute a structured diff between the previous and current auction state and emit typed change events:

• OrderAdded
• OrderRemoved
• OrderUpdated
• PriceChanged
• BlockChanged
• AuctionChanged
• JitOwnersChanged

These are broadcast over a tokio broadcast channel with a rolling history buffer so clients can replay missed events on reconnect.

New API Endpoints

Three new endpoints handle the lifecycle:

Endpoint	Description
GET /delta/snapshot	Full state bootstrap for new/reconnecting clients
GET /delta/stream	SSE stream of incremental delta envelopes with built-in replay for gap recovery
GET /delta/checksum	SHA-256 hash of current orders and prices for integrity cross-validation

Driver-Side Replica

The driver maintains a local in-memory replica that subscribes to the stream. Autopilot restarts are detected via a bootId field in each envelope, which triggers an automatic re-snapshot on the driver side.

Thin Solve Requests

To actually reduce payload size end-to-end, drivers that opt in via supports_thin_solve_request: true receive thin solve requests — the order list is stripped from the body and lightweight metadata is passed in HTTP headers instead:

• X-Auction-Id
• X-Auction-Deadline
• X-Auction-Tokens
• X-Auction-Body-Mode

The driver reconstructs the full auction from its local replica.

Rollout Strategy

Rollout is fully opt-in behind feature flags so existing solvers are completely unaffected:

• AUTOPILOT_DELTA_SYNC_ENABLED — enables autopilot-side endpoints
• DRIVER_DELTA_SYNC_ENABLED / DRIVER_DELTA_SYNC_AUTOPILOT_URL — enables driver-side replica
• supports_thin_solve_request: true — opts a solver into thin requests

A shadow-compare mode (AUTOPILOT_DELTA_SYNC_SHADOW_COMPARE) runs the incremental path alongside the full rebuild and validates output before going primary, which should give confidence before full cutover.

---

Happy to walk through any part of this in more detail. Let me know if there are concerns or alternative approaches worth considering.