Skip to content

Commit ddecc09

Browse files
lilyshen0722claude
lilyshen0722
and
claude
authored
feat(agent-memory): ADR-003 Phase 2a — POST /memory/sync with mode + dedup (#191)
Adds the kernel promotion endpoint every runtime driver will target. Phase 2b (new OpenClaw tools that call it) is a separate change in the submodule. ## What ships **New endpoint: `POST /api/agents/runtime/memory/sync`** - Required body: `{ sections, sourceRuntime?, mode }` - `mode: "full"` — replaces the entire sections envelope. Sections omitted from the payload are cleared. Use for driver snapshots. - `mode: "patch"` — merges with existing state. Single-object sections $set per-key (siblings preserved). Array sections (daily, relationships) merge element-wise keyed by `date` / `otherInstanceId`. Use for incremental promotion. - Idempotent within the same UTC day: repeated identical payloads return `{ deduped: true }` without writing. Dedup key is `(dayBucket, sourceRuntime, sha256-trunc32 of canonical-stringified body)`. Canonical stringify sorts object keys recursively so semantically identical payloads with different key order collapse to one write. **Supporting work** - `AgentMemory.lastSyncKey` + `lastSyncAt` track the dedup cache state. - Strict `'YYYY-MM-DD'` validation on `daily[].date` — regex + Date round-trip so calendar-invalid dates (2026-02-30) are rejected. - `mergePatchSections` in agentMemoryService pure-fns the element-level array merge; unit-tested separately from route wiring. - Full-mode handler always updates v1 `content` mirror from final sections, including blanking it when the new snapshot omits long_term. Avoids phantom data on GET when v1 readers still exist. ## Critical fix caught by review - `PUT /memory` and `nativeRuntimeService.commonly_write_memory` now clear `lastSyncKey`/`lastSyncAt` on every write. Without this, a non-sync writer mutating state between two identical syncs caused the second sync to be silently deduped, leaving the kernel stuck on the intervening write while the driver believed its promotion succeeded. Regression test locks the invariant. ## Other review-driven changes - Canonical stringify (json-stable-order) replaces JSON.stringify in the dedup key so webhook/Python drivers with different emit order don't silently miss dedup. - Full-mode v1 mirror rule made symmetric with sections (full replace → full mirror refresh, including empty when long_term omitted). - Structured logs now fire on both dedup hits and validation rejects (REVIEW.md §Maintainability, kernel-surface observability). - Concurrency caveat noted inline on `mergePatchSections` — read-merge- write assumes per-instance serialization; revisit when webhook drivers arrive in Phase 2b/later. ## Tests (17 new; 95/95 agent-memory pass; 705/705 backend total) Unit (on top of existing): - `isValidYMD` (4 tests — accept valid, reject malformed, reject calendar-invalid incl. leap years, reject non-strings) - `mergePatchSections` (6 tests — single-object per-key merge, replacement, daily by date, relationships by otherInstanceId, missing-existing, preserve-omitted) - `computeSyncDedupKey` (8 tests — same-day-same-payload collapse, mode/runtime/day/content sensitivity, key prefix, canonical-stringify order invariance across object keys AND across top-level sections) Integration (on top of existing): - Rejects without sections, without valid mode, with malformed date, with calendar-invalid date - full mode wipes envelope, patch mode preserves siblings + merges daily by date and relationships by otherInstanceId - full mode WITHOUT long_term blanks v1 content mirror (Important fix) - Patch mode with long_term mirrors v1 content - Same-day dedup returns deduped:true; cross-content re-sync does not - PUT /memory invalidates sync dedup cache (Critical fix) - Server-stamps byteSize; rejects unauthenticated Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 55e2511 commit ddecc09

6 files changed

Lines changed: 707 additions & 4 deletions

File tree

‎backend/__tests__/integration/agent-memory-envelope.test.js‎

Lines changed: 310 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -548,4 +548,314 @@ describe('AgentMemory envelope — GET/PUT /memory + backfill', () => {
548548
expect(after.sections).toBeUndefined();
549549
});
550550
});
551+
552+
// ------------------------------------------------------------------- //
553+
// POST /memory/sync (ADR-003 Phase 2) //
554+
// ------------------------------------------------------------------- //
555+
556+
describe('POST /memory/sync', () => {
557+
it('rejects requests without sections', async () => {
558+
const res = await request(app)
559+
.post('/api/agents/runtime/memory/sync')
560+
.set('Authorization', `Bearer ${runtimeToken}`)
561+
.send({ mode: 'full' });
562+
expect(res.status).toBe(400);
563+
expect(res.body.message).toMatch(/sections is required/);
564+
});
565+
566+
it('rejects requests without a valid mode', async () => {
567+
const resA = await request(app)
568+
.post('/api/agents/runtime/memory/sync')
569+
.set('Authorization', `Bearer ${runtimeToken}`)
570+
.send({ sections: { long_term: { content: 'x' } } });
571+
expect(resA.status).toBe(400);
572+
expect(resA.body.message).toMatch(/mode must be 'full' or 'patch'/);
573+
574+
const resB = await request(app)
575+
.post('/api/agents/runtime/memory/sync')
576+
.set('Authorization', `Bearer ${runtimeToken}`)
577+
.send({ sections: { long_term: { content: 'x' } }, mode: 'merge' });
578+
expect(resB.status).toBe(400);
579+
});
580+
581+
it('rejects invalid YYYY-MM-DD date on daily entries', async () => {
582+
const res = await request(app)
583+
.post('/api/agents/runtime/memory/sync')
584+
.set('Authorization', `Bearer ${runtimeToken}`)
585+
.send({
586+
sections: { daily: [{ date: '2026/04/14', content: 'x' }] },
587+
mode: 'full',
588+
});
589+
expect(res.status).toBe(400);
590+
expect(res.body.message).toMatch(/YYYY-MM-DD/);
591+
});
592+
593+
it('also rejects calendar-invalid dates (feb 30)', async () => {
594+
const res = await request(app)
595+
.post('/api/agents/runtime/memory/sync')
596+
.set('Authorization', `Bearer ${runtimeToken}`)
597+
.send({
598+
sections: { daily: [{ date: '2026-02-30', content: 'x' }] },
599+
mode: 'full',
600+
});
601+
expect(res.status).toBe(400);
602+
expect(res.body.message).toMatch(/YYYY-MM-DD/);
603+
});
604+
605+
it('full mode: replaces the entire sections envelope', async () => {
606+
// Seed with long_term + shared.
607+
await request(app)
608+
.put('/api/agents/runtime/memory')
609+
.set('Authorization', `Bearer ${runtimeToken}`)
610+
.send({ sections: { long_term: { content: 'old' }, shared: { content: 'bio', visibility: 'public' } } })
611+
.expect(200);
612+
613+
// full sync with only dedup_state — long_term/shared should be gone.
614+
await request(app)
615+
.post('/api/agents/runtime/memory/sync')
616+
.set('Authorization', `Bearer ${runtimeToken}`)
617+
.send({
618+
sections: { dedup_state: { content: '## Commented\n{}' } },
619+
mode: 'full',
620+
sourceRuntime: 'openclaw',
621+
})
622+
.expect(200);
623+
624+
const get = await request(app)
625+
.get('/api/agents/runtime/memory')
626+
.set('Authorization', `Bearer ${runtimeToken}`);
627+
expect(get.body.sections.long_term).toBeUndefined();
628+
expect(get.body.sections.shared).toBeUndefined();
629+
expect(get.body.sections.dedup_state.content).toContain('Commented');
630+
expect(get.body.sourceRuntime).toBe('openclaw');
631+
});
632+
633+
it('patch mode: preserves sibling sections and merges daily by date', async () => {
634+
// Seed.
635+
await request(app)
636+
.post('/api/agents/runtime/memory/sync')
637+
.set('Authorization', `Bearer ${runtimeToken}`)
638+
.send({
639+
sections: {
640+
long_term: { content: 'keep me' },
641+
daily: [
642+
{ date: '2026-04-12', content: 'mon' },
643+
{ date: '2026-04-13', content: 'tue' },
644+
],
645+
},
646+
mode: 'full',
647+
})
648+
.expect(200);
649+
650+
// Patch with updated tue + new wed; long_term should survive.
651+
await request(app)
652+
.post('/api/agents/runtime/memory/sync')
653+
.set('Authorization', `Bearer ${runtimeToken}`)
654+
.send({
655+
sections: {
656+
daily: [
657+
{ date: '2026-04-13', content: 'tue-updated' },
658+
{ date: '2026-04-14', content: 'wed' },
659+
],
660+
},
661+
mode: 'patch',
662+
})
663+
.expect(200);
664+
665+
const get = await request(app)
666+
.get('/api/agents/runtime/memory')
667+
.set('Authorization', `Bearer ${runtimeToken}`);
668+
expect(get.body.sections.long_term.content).toBe('keep me');
669+
const byDate = Object.fromEntries(get.body.sections.daily.map((d) => [d.date, d.content]));
670+
expect(byDate['2026-04-12']).toBe('mon');
671+
expect(byDate['2026-04-13']).toBe('tue-updated');
672+
expect(byDate['2026-04-14']).toBe('wed');
673+
});
674+
675+
it('patch mode: merges relationships by otherInstanceId', async () => {
676+
await request(app)
677+
.post('/api/agents/runtime/memory/sync')
678+
.set('Authorization', `Bearer ${runtimeToken}`)
679+
.send({
680+
sections: {
681+
relationships: [
682+
{ otherInstanceId: 'nova', notes: 'old nova' },
683+
{ otherInstanceId: 'theo', notes: 'old theo' },
684+
],
685+
},
686+
mode: 'full',
687+
})
688+
.expect(200);
689+
690+
await request(app)
691+
.post('/api/agents/runtime/memory/sync')
692+
.set('Authorization', `Bearer ${runtimeToken}`)
693+
.send({
694+
sections: {
695+
relationships: [
696+
{ otherInstanceId: 'nova', notes: 'new nova' },
697+
{ otherInstanceId: 'liz', notes: 'new liz' },
698+
],
699+
},
700+
mode: 'patch',
701+
})
702+
.expect(200);
703+
704+
const get = await request(app)
705+
.get('/api/agents/runtime/memory')
706+
.set('Authorization', `Bearer ${runtimeToken}`);
707+
const byId = Object.fromEntries(
708+
get.body.sections.relationships.map((r) => [r.otherInstanceId, r.notes]),
709+
);
710+
expect(byId.nova).toBe('new nova');
711+
expect(byId.theo).toBe('old theo');
712+
expect(byId.liz).toBe('new liz');
713+
});
714+
715+
it('mirrors v1 content when patch mode includes long_term', async () => {
716+
await request(app)
717+
.post('/api/agents/runtime/memory/sync')
718+
.set('Authorization', `Bearer ${runtimeToken}`)
719+
.send({
720+
sections: { long_term: { content: 'sync-mirrored' } },
721+
mode: 'patch',
722+
})
723+
.expect(200);
724+
725+
const get = await request(app)
726+
.get('/api/agents/runtime/memory')
727+
.set('Authorization', `Bearer ${runtimeToken}`);
728+
expect(get.body.content).toBe('sync-mirrored');
729+
});
730+
731+
it('dedupes identical payloads within the same day bucket', async () => {
732+
const body = {
733+
sections: { long_term: { content: 'stable' } },
734+
sourceRuntime: 'openclaw',
735+
mode: 'patch',
736+
};
737+
738+
const first = await request(app)
739+
.post('/api/agents/runtime/memory/sync')
740+
.set('Authorization', `Bearer ${runtimeToken}`)
741+
.send(body);
742+
expect(first.status).toBe(200);
743+
expect(first.body.ok).toBe(true);
744+
expect(first.body.deduped).toBeUndefined();
745+
746+
const second = await request(app)
747+
.post('/api/agents/runtime/memory/sync')
748+
.set('Authorization', `Bearer ${runtimeToken}`)
749+
.send(body);
750+
expect(second.status).toBe(200);
751+
expect(second.body.ok).toBe(true);
752+
expect(second.body.deduped).toBe(true);
753+
754+
// Count should still be 1.
755+
expect(await AgentMemory.countDocuments({})).toBe(1);
756+
});
757+
758+
it('does NOT dedupe when the payload content changes', async () => {
759+
await request(app)
760+
.post('/api/agents/runtime/memory/sync')
761+
.set('Authorization', `Bearer ${runtimeToken}`)
762+
.send({
763+
sections: { long_term: { content: 'first' } }, sourceRuntime: 'openclaw', mode: 'patch',
764+
})
765+
.expect(200);
766+
767+
const res = await request(app)
768+
.post('/api/agents/runtime/memory/sync')
769+
.set('Authorization', `Bearer ${runtimeToken}`)
770+
.send({
771+
sections: { long_term: { content: 'second' } }, sourceRuntime: 'openclaw', mode: 'patch',
772+
});
773+
expect(res.body.deduped).toBeUndefined();
774+
const get = await request(app)
775+
.get('/api/agents/runtime/memory')
776+
.set('Authorization', `Bearer ${runtimeToken}`);
777+
expect(get.body.sections.long_term.content).toBe('second');
778+
});
779+
780+
it('server-stamps byteSize on sync writes', async () => {
781+
await request(app)
782+
.post('/api/agents/runtime/memory/sync')
783+
.set('Authorization', `Bearer ${runtimeToken}`)
784+
.send({
785+
sections: { long_term: { content: '😀 hi', byteSize: 9999 } },
786+
mode: 'full',
787+
})
788+
.expect(200);
789+
const get = await request(app)
790+
.get('/api/agents/runtime/memory')
791+
.set('Authorization', `Bearer ${runtimeToken}`);
792+
expect(get.body.sections.long_term.byteSize).toBe(Buffer.byteLength('😀 hi', 'utf8'));
793+
});
794+
795+
it('rejects unauthenticated sync requests', async () => {
796+
const res = await request(app)
797+
.post('/api/agents/runtime/memory/sync')
798+
.send({ sections: { long_term: { content: 'x' } }, mode: 'full' });
799+
expect(res.status).toBe(401);
800+
});
801+
802+
it('full mode without long_term wipes the v1 content mirror', async () => {
803+
// Seed with v1 content via mirror.
804+
await request(app)
805+
.post('/api/agents/runtime/memory/sync')
806+
.set('Authorization', `Bearer ${runtimeToken}`)
807+
.send({ sections: { long_term: { content: 'v1 mirror source' } }, mode: 'full' })
808+
.expect(200);
809+
let get = await request(app)
810+
.get('/api/agents/runtime/memory')
811+
.set('Authorization', `Bearer ${runtimeToken}`);
812+
expect(get.body.content).toBe('v1 mirror source');
813+
814+
// full sync that omits long_term — v1 content must be blanked, not stale.
815+
await request(app)
816+
.post('/api/agents/runtime/memory/sync')
817+
.set('Authorization', `Bearer ${runtimeToken}`)
818+
.send({ sections: { dedup_state: { content: '## Commented\n{}' } }, mode: 'full' })
819+
.expect(200);
820+
get = await request(app)
821+
.get('/api/agents/runtime/memory')
822+
.set('Authorization', `Bearer ${runtimeToken}`);
823+
expect(get.body.content).toBe('');
824+
expect(get.body.sections.long_term).toBeUndefined();
825+
});
826+
827+
it('PUT /memory invalidates the sync dedup cache (cross-writer safety)', async () => {
828+
const body = {
829+
sections: { long_term: { content: 'dedup-me' } },
830+
sourceRuntime: 'openclaw',
831+
mode: 'patch',
832+
};
833+
834+
// Sync once so lastSyncKey is populated.
835+
await request(app)
836+
.post('/api/agents/runtime/memory/sync')
837+
.set('Authorization', `Bearer ${runtimeToken}`)
838+
.send(body)
839+
.expect(200);
840+
841+
// A non-sync writer mutates sections directly (human operator / v1 tool).
842+
await request(app)
843+
.put('/api/agents/runtime/memory')
844+
.set('Authorization', `Bearer ${runtimeToken}`)
845+
.send({ sections: { long_term: { content: 'stomped by PUT' } } })
846+
.expect(200);
847+
848+
// The same sync payload must NOT be deduped now — kernel state drifted.
849+
const second = await request(app)
850+
.post('/api/agents/runtime/memory/sync')
851+
.set('Authorization', `Bearer ${runtimeToken}`)
852+
.send(body);
853+
expect(second.body.deduped).toBeUndefined();
854+
855+
const get = await request(app)
856+
.get('/api/agents/runtime/memory')
857+
.set('Authorization', `Bearer ${runtimeToken}`);
858+
expect(get.body.sections.long_term.content).toBe('dedup-me');
859+
});
860+
});
551861
});

0 commit comments

Comments
 (0)