Skip to content

feat(agent-memory): ADR-003 Phase 2a — POST /memory/sync with mode + dedup#191

Merged
lilyshen0722 merged 1 commit intomainfrom
feat/adr-003-phase2-memory-sync
Apr 14, 2026
Merged

feat(agent-memory): ADR-003 Phase 2a — POST /memory/sync with mode + dedup#191
lilyshen0722 merged 1 commit intomainfrom
feat/adr-003-phase2-memory-sync

Conversation

@lilyshen0722
Copy link
Copy Markdown
Contributor

Summary

New kernel promotion endpoint every runtime driver targets. Phase 2b (OpenClaw tools that call it) is a follow-up in the submodule.

Endpoint: `POST /api/agents/runtime/memory/sync` with required `{ sections, mode: 'full' | 'patch', sourceRuntime? }`

  • `full`: replaces the entire sections envelope; omitted sections are cleared
  • `patch`: per-key merge for single-object sections; element-level merge for arrays (`daily` by `date`, `relationships` by `otherInstanceId`)
  • Idempotent within the same UTC day via canonical-stringify dedup key; repeated identical payloads return `{ deduped: true }`
  • byteSize/updatedAt server-stamped; schemaVersion auto-set to 2; v1 `content` mirror rules documented in the handler docstring

17 new tests (8 unit-merge + 4 unit-YMD + 8 unit-dedup + 13 integration); 95/95 agent-memory pass, 705/705 backend total.

Critical fix (reviewer caught pre-commit)

`PUT /memory` and `nativeRuntimeService.commonly_write_memory` now clear `lastSyncKey`/`lastSyncAt` on every write. Without this, a non-sync writer mutating state between two identical syncs caused the second sync to be silently deduped, leaving the kernel stuck on the intervening write while the driver believed its promotion succeeded. Regression test locks the invariant.

Other review-driven changes

  • Canonical stringify in dedup-key computation so webhook/Python drivers with different emit order still collapse identical payloads
  • Full-mode v1 mirror rule made symmetric: full replace → full mirror refresh, including blanking `content` when `long_term` is omitted (prevents phantom data on GET)
  • Structured logs on dedup hits + validation rejects (REVIEW.md §Maintainability)
  • Concurrency caveat on `mergePatchSections` documents read-merge-write assumes per-instance serialization; revisit when webhook drivers arrive

Deferred to Phase 2b / later

  • OpenClaw `commonly_read_my_memory` / `commonly_save_my_memory` / `commonly_ask_agent` tools (submodule work)
  • Optimistic-concurrency patch merge (currently relies on serialized drivers — fine for heartbeat cadence, revisit with webhooks)

Changes

  • `backend/models/AgentMemory.ts` — adds `lastSyncKey?`, `lastSyncAt?`
  • `backend/services/agentMemoryService.ts` — `isValidYMD`, `mergePatchSections`, `computeSyncDedupKey` (canonical-stringify based)
  • `backend/routes/agentsRuntime.ts` — new `POST /memory/sync`, stricter YYYY-MM-DD validation on `daily[].date`, PUT now invalidates dedup
  • `backend/services/nativeRuntimeService.ts` — `commonly_write_memory` invalidates dedup
  • 17 new tests

Test plan

  • 95/95 agent-memory tests pass locally
  • 705/705 backend total pass locally
  • Reviewer agent pass with Critical + 4 Important fixes applied
  • CI green
  • Post-merge: build, deploy, live-verify `/memory/sync` full + patch + dedup + PUT-invalidation

🤖 Generated with Claude Code

…dedup

Adds the kernel promotion endpoint every runtime driver will target. Phase 2b
(new OpenClaw tools that call it) is a separate change in the submodule.

## What ships

**New endpoint: `POST /api/agents/runtime/memory/sync`**
- Required body: `{ sections, sourceRuntime?, mode }`
- `mode: "full"` — replaces the entire sections envelope. Sections omitted
  from the payload are cleared. Use for driver snapshots.
- `mode: "patch"` — merges with existing state. Single-object sections $set
  per-key (siblings preserved). Array sections (daily, relationships) merge
  element-wise keyed by `date` / `otherInstanceId`. Use for incremental
  promotion.
- Idempotent within the same UTC day: repeated identical payloads return
  `{ deduped: true }` without writing. Dedup key is
  `(dayBucket, sourceRuntime, sha256-trunc32 of canonical-stringified body)`.
  Canonical stringify sorts object keys recursively so semantically
  identical payloads with different key order collapse to one write.

**Supporting work**
- `AgentMemory.lastSyncKey` + `lastSyncAt` track the dedup cache state.
- Strict `'YYYY-MM-DD'` validation on `daily[].date` — regex + Date
  round-trip so calendar-invalid dates (2026-02-30) are rejected.
- `mergePatchSections` in agentMemoryService pure-fns the element-level
  array merge; unit-tested separately from route wiring.
- Full-mode handler always updates v1 `content` mirror from final sections,
  including blanking it when the new snapshot omits long_term. Avoids
  phantom data on GET when v1 readers still exist.

## Critical fix caught by review

- `PUT /memory` and `nativeRuntimeService.commonly_write_memory` now clear
  `lastSyncKey`/`lastSyncAt` on every write. Without this, a non-sync
  writer mutating state between two identical syncs caused the second sync
  to be silently deduped, leaving the kernel stuck on the intervening
  write while the driver believed its promotion succeeded. Regression
  test locks the invariant.

## Other review-driven changes

- Canonical stringify (json-stable-order) replaces JSON.stringify in the
  dedup key so webhook/Python drivers with different emit order don't
  silently miss dedup.
- Full-mode v1 mirror rule made symmetric with sections (full replace →
  full mirror refresh, including empty when long_term omitted).
- Structured logs now fire on both dedup hits and validation rejects
  (REVIEW.md §Maintainability, kernel-surface observability).
- Concurrency caveat noted inline on `mergePatchSections` — read-merge-
  write assumes per-instance serialization; revisit when webhook drivers
  arrive in Phase 2b/later.

## Tests (17 new; 95/95 agent-memory pass; 705/705 backend total)

Unit (on top of existing):
- `isValidYMD` (4 tests — accept valid, reject malformed, reject
  calendar-invalid incl. leap years, reject non-strings)
- `mergePatchSections` (6 tests — single-object per-key merge, replacement,
  daily by date, relationships by otherInstanceId, missing-existing,
  preserve-omitted)
- `computeSyncDedupKey` (8 tests — same-day-same-payload collapse,
  mode/runtime/day/content sensitivity, key prefix, canonical-stringify
  order invariance across object keys AND across top-level sections)

Integration (on top of existing):
- Rejects without sections, without valid mode, with malformed date,
  with calendar-invalid date
- full mode wipes envelope, patch mode preserves siblings + merges daily
  by date and relationships by otherInstanceId
- full mode WITHOUT long_term blanks v1 content mirror (Important fix)
- Patch mode with long_term mirrors v1 content
- Same-day dedup returns deduped:true; cross-content re-sync does not
- PUT /memory invalidates sync dedup cache (Critical fix)
- Server-stamps byteSize; rejects unauthenticated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lilyshen0722 lilyshen0722 merged commit ddecc09 into main Apr 14, 2026
10 checks passed
@lilyshen0722 lilyshen0722 deleted the feat/adr-003-phase2-memory-sync branch April 14, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants