From 55c8683a0678039a754a7b9a6504bf794d698951 Mon Sep 17 00:00:00 2001 From: Divyansh Kumar Date: Fri, 27 Feb 2026 10:42:08 +0000 Subject: [PATCH 1/2] =?UTF-8?q?synapses-intelligence=20v0.3.0=20=E2=80=94?= =?UTF-8?q?=20Context=20Packet,=20SDLC,=20Learning=20Loop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v0.2.0: Context Packet Builder - POST /v1/context-packet: phase-aware semantic document replacing raw graph nodes (~800 token cap vs 4000+ for raw output) - 7-section packet: root summary, dep summaries, LLM insight, constraints, team status, quality gate, pattern hints, phase guidance - SDLC phase system (planning/development/testing/review/deployment) controls which sections are active per phase - Quality modes (quick/standard/enterprise) drive the quality gate checklist - Co-occurrence learner: POST /v1/decision logs agent work; Brain builds pattern hints from symmetric co-occurrence counts (no LLM, pure Go) - New HTTP endpoints: /v1/sdlc, /v1/sdlc/phase, /v1/sdlc/mode, /v1/decision, /v1/patterns - New CLI: brain sdlc, brain decisions, brain patterns - New SQLite tables: sdlc_config, context_patterns, decision_log - NullBrain extended with 6 new no-op methods - Full JSON tags on all HTTP request/response types v0.2.1: Correctness fixes - Extract shared ExtractJSON + Truncate into internal/llm/util.go; remove 4 duplicate copies from ingestor/enricher/guardian/orchestrator - Store pruneOldData: silent db.Exec errors now logged to stderr - Store GetSummaryWithTags, GetRecentDecisions: JSON unmarshal errors logged - LLMUsed bool added to enricher.Response, brain.EnrichResponse, brain.ContextPacket v0.3.0: Insight Cache + Quality Signal - insight_cache table: caches LLM insight per (node_id, phase), 6-hour TTL - Insight cache invalidation on re-ingest: UpsertSummary deletes stale cache - Builder is cache-first: checks insight_cache before calling Ollama; stores result after live call; LLMUsed=false on cache hits - violation_cache TTL prune added (7-day expiry at startup) - PacketQuality float64 (0.0-1.0) heuristic on every context packet - Reset() now clears insight_cache (bug fix) - INTELLIGENCE.md: comprehensive integration guide for Synapses core including 3 injection points, BrainClient design, 6 integration challenges, 7 planned improvements for v0.4.0 - Version bumped to 0.3.0 Co-Authored-By: Claude Sonnet 4.6 --- INTELLIGENCE.md | 716 ++++++++++++++++++++++++ cmd/brain/main.go | 171 +++++- internal/contextbuilder/builder.go | 72 ++- internal/contextbuilder/builder_test.go | 65 +++ internal/enricher/enricher.go | 33 +- internal/guardian/guardian.go | 30 +- internal/ingestor/ingestor.go | 35 +- internal/ingestor/ingestor_test.go | 4 +- internal/llm/util.go | 34 ++ internal/orchestrator/orchestrator.go | 21 +- internal/store/store.go | 94 +++- pkg/brain/brain.go | 31 +- pkg/brain/null.go | 2 + pkg/brain/types.go | 66 ++- server/server.go | 130 +++++ 15 files changed, 1335 insertions(+), 169 deletions(-) create mode 100644 INTELLIGENCE.md create mode 100644 internal/llm/util.go diff --git a/INTELLIGENCE.md b/INTELLIGENCE.md new file mode 100644 index 0000000..24bab17 --- /dev/null +++ b/INTELLIGENCE.md @@ -0,0 +1,716 @@ +# synapses-intelligence v0.3.0 +## The Thinking Brain for AI Agent Systems + +--- + +## What This Is + +`synapses-intelligence` is a **local LLM sidecar** that adds semantic reasoning to code agent systems. It runs as a lightweight HTTP server on `localhost:11435`, backed by [Ollama](https://ollama.com) (local LLM, no API keys, no cloud, no data leaves the machine) and its own `brain.sqlite` for persistent storage. + +It is designed as **Layer 3** in a multi-layer AI operating system stack: + +``` +[ Agent (Claude / GPT / Gemini) ] ← consumes Context Packets + ↑ +[ synapses-intelligence (Brain) ] ← this module — semantic enrichment + ↑ +[ Synapses (Code Graph MCP) ] ← structural graph, BFS traversal + ↑ +[ Codebase ] +``` + +**Primary job:** Transform raw structural graph data (node IDs, edge types, code snippets) into compact, phase-aware **Context Packets** — structured semantic documents that tell an LLM agent exactly what it needs to know, in ~600-800 tokens, instead of 4,000+ tokens of raw code. + +**Secondary jobs:** Learn project patterns over time, enforce SDLC phase discipline, coordinate multi-agent work, explain architectural violations in plain English. + +--- + +## Architecture Overview + +``` +brain.sqlite Ollama (local LLM) + │ │ + ▼ ▼ +┌─────────────────────────────────────────────┐ +│ pkg/brain (Brain interface) │ +│ │ +│ ┌──────────────┐ ┌────────────────────┐ │ +│ │ Ingestor │ │ Context Builder │ │ +│ │ (on save) │ │ (on get_context) │ │ +│ └──────────────┘ └────────────────────┘ │ +│ ┌──────────────┐ ┌────────────────────┐ │ +│ │ Enricher │ │ SDLC Manager │ │ +│ │ (LLM call) │ │ (phase/mode) │ │ +│ └──────────────┘ └────────────────────┘ │ +│ ┌──────────────┐ ┌────────────────────┐ │ +│ │ Guardian │ │ Co-occurrence │ │ +│ │ (violations)│ │ Learner │ │ +│ └──────────────┘ └────────────────────┘ │ +│ ┌──────────────┐ │ +│ │ Orchestrator │ │ +│ │ (conflicts) │ │ +│ └──────────────┘ │ +└─────────────────────────────────────────────┘ + ↓ + HTTP server :11435 +``` + +**brain.sqlite tables:** + +| Table | Purpose | TTL | +|-------|---------|-----| +| `semantic_summaries` | 1-sentence intent summaries per node | Permanent (updated on re-ingest) | +| `violation_cache` | Cached violation explanations per (rule_id, file) | 7 days | +| `insight_cache` | Cached LLM insight per (node_id, phase) | 6 hours; invalidated on re-ingest | +| `sdlc_config` | Current project phase + quality mode | Permanent (1 row) | +| `context_patterns` | Learned co-occurrence patterns | Pruned if stale after 14 days | +| `decision_log` | Agent action history for learning | Pruned after 30 days | + +--- + +## Capabilities + +### 1. Semantic Ingestor — `POST /v1/ingest` + +Generates a 1-sentence intent summary and domain tags for any code entity. Called when a file changes. + +**Request:** +```json +{ + "node_id": "pkg:func:AuthService.Validate", + "node_name": "Validate", + "node_type": "method", + "package": "auth", + "code": "func (a *AuthService) Validate(token string) (Claims, error) { ... }" +} +``` + +**Response:** +```json +{ + "node_id": "pkg:func:AuthService.Validate", + "summary": "Validates JWT tokens by verifying signature and checking expiry.", + "tags": ["auth", "jwt", "validation"] +} +``` + +Summaries are stored in `semantic_summaries`. When a node is re-ingested, its entry in `insight_cache` is automatically invalidated (code changed → old insight stale). + +--- + +### 2. Context Enricher — `POST /v1/enrich` + +Returns stored summaries for a set of node IDs plus an optional 2-sentence LLM insight about the root entity's architectural role. + +**Request:** +```json +{ + "root_id": "pkg:struct:AuthService", + "root_name": "AuthService", + "root_type": "struct", + "all_node_ids": ["pkg:struct:AuthService", "pkg:func:TokenValidator.Verify"], + "callee_names": ["TokenValidator", "RateLimiter"], + "caller_names": ["HTTPHandler"], + "task_context": "Adding OAuth2 support" +} +``` + +**Response:** +```json +{ + "insight": "AuthService is the central authentication boundary, coordinating JWT verification and session management for all protected routes.", + "concerns": ["token expiry handling", "rate limit bypass risk"], + "summaries": { + "pkg:struct:AuthService": "Central coordinator for user authentication and session lifecycle.", + "pkg:func:TokenValidator.Verify": "Cryptographically verifies JWT signatures against the configured signing key." + }, + "llm_used": true +} +``` + +--- + +### 3. Context Packet Builder — `POST /v1/context-packet` *(v0.2.0+)* + +**The primary integration endpoint.** Builds a complete, phase-aware Context Packet — a structured semantic document replacing raw graph nodes. This is the main value the Brain provides to Synapses. + +**Request:** +```json +{ + "agent_id": "claude-session-42", + "snapshot": { + "root_node_id": "pkg:struct:AuthService", + "root_name": "AuthService", + "root_type": "struct", + "root_file": "internal/auth/service.go", + "callee_names": ["TokenValidator", "RateLimiter", "UserRepository"], + "caller_names": ["HTTPHandler", "GRPCHandler"], + "applicable_rules": [ + {"rule_id": "no-db-in-handler", "severity": "error", "description": "handlers must not call db directly"} + ], + "active_claims": [ + {"agent_id": "claude-backend", "scope": "internal/session/", "scope_type": "directory", "expires_at": "2026-02-27T15:00:00Z"} + ], + "task_context": "Adding OAuth2 support", + "task_id": "task-123" + }, + "phase": "development", + "quality_mode": "standard", + "enable_llm": true +} +``` + +**Response — a full Context Packet (~600-800 tokens):** +```json +{ + "agent_id": "claude-session-42", + "entity_name": "AuthService", + "entity_type": "struct", + "generated_at": "2026-02-27T14:30:00Z", + "phase": "development", + "quality_mode": "standard", + "root_summary": "Central coordinator for user authentication and session lifecycle.", + "dependency_summaries": { + "TokenValidator": "Cryptographically verifies JWT signatures against the configured signing key.", + "RateLimiter": "Enforces per-user sliding window request limits using Redis counters.", + "UserRepository": "Fetches and caches user records from PostgreSQL by ID or email." + }, + "insight": "AuthService is the central authentication boundary that coordinates JWT verification and session management; changes here affect all protected API routes.", + "concerns": ["token expiry edge cases", "concurrent session invalidation"], + "active_constraints": [ + { + "rule_id": "no-db-in-handler", + "severity": "error", + "description": "handlers must not call db directly", + "hint": "inject a repository interface instead of calling db.Query() directly" + } + ], + "team_status": [ + {"agent_id": "claude-backend", "scope": "internal/session/", "scope_type": "directory", "expires_in_seconds": 1800} + ], + "quality_gate": { + "require_tests": true, + "require_docs": false, + "require_pr_check": false, + "checklist": [ + "Write unit tests for new/modified functions", + "Run validate_plan — no rule violations", + "Exported symbols should have doc comments" + ] + }, + "pattern_hints": [ + {"trigger": "AuthService", "co_change": "TokenValidator", "reason": "edit during development", "confidence": 0.87} + ], + "phase_guidance": "You are in development phase. Implement per the plan. Claim work before editing (claim_work). Respect all active constraints. Run validate_plan before major changes.", + "llm_used": true, + "packet_quality": 1.0 +} +``` + +**Context Packet section matrix — what's included per SDLC phase:** + +| Section | planning | development | testing | review | deployment | +|---------|----------|-------------|---------|--------|------------| +| Root summary | ✓ | ✓ | ✓ | ✓ | ✓ | +| Dep summaries | ✓ | ✓ | — | ✓ | — | +| LLM insight | ✓ | ✓ | — | ✓ | — | +| Constraints | — | ✓ | ✓ | ✓ | — | +| Team status | ✓ | ✓ | ✓ | ✓ | ✓ | +| Quality gate | — | ✓ | ✓ | ✓ | — | +| Pattern hints | — | ✓ | — | ✓ | — | +| Phase guidance | ✓ | ✓ | ✓ | ✓ | ✓ | + +**`packet_quality` field:** 0.0–1.0 heuristic. `0.0` = no summaries ingested. `0.4` = root summary present. `0.5` = root + dep summaries. `1.0` = all sections including insight. Callers can use this to decide whether to request a follow-up LLM pass. + +**Insight caching:** On the LLM path, insight is cached in `insight_cache` for 6 hours per `(node_id, phase)` pair. Subsequent calls for the same entity/phase within 6 hours are served from cache with zero LLM cost. `llm_used` is `false` on cache hits. + +**Nil response (HTTP 204):** If the Brain is unavailable or `context_builder` is disabled, the server returns 204. Callers must treat this as "use raw context" and fall back without error. + +--- + +### 4. Rule Guardian — `POST /v1/explain-violation` + +Explains an architectural rule violation in plain English with a concrete fix suggestion. Results are cached per `(rule_id, source_file)` for 7 days. + +**Request:** +```json +{ + "rule_id": "no-db-in-handler", + "rule_severity": "error", + "description": "view/handler files must not import db packages", + "source_file": "internal/handlers/auth.go", + "target_name": "db.QueryRow" +} +``` + +**Response:** +```json +{ + "explanation": "The handler auth.go is directly calling database functions, bypassing the repository layer. This couples HTTP handling to database implementation details, making the handler untestable and violating the clean architecture boundary.", + "fix": "Create an AuthRepository interface in internal/auth/ and inject it into the handler constructor. Move all db.QueryRow calls into a concrete PostgresAuthRepository implementation." +} +``` + +--- + +### 5. Task Orchestrator — `POST /v1/coordinate` + +Suggests work distribution when multiple agents claim overlapping scopes. + +**Request:** +```json +{ + "new_agent_id": "claude-frontend", + "new_scope": "internal/auth/", + "conflicting_claims": [ + {"agent_id": "claude-backend", "scope": "internal/auth/service.go", "scope_type": "file"} + ] +} +``` + +**Response:** +```json +{ + "suggestion": "Agent claude-backend is actively modifying auth/service.go. You can safely work on the auth handler layer (internal/handlers/auth.go) or the auth types (internal/auth/types.go) without conflict.", + "alternative_scope": "internal/handlers/auth.go" +} +``` + +--- + +### 6. SDLC Phase Management + +**GET /v1/sdlc** — Returns current phase and quality mode: +```json +{"phase": "development", "quality_mode": "standard", "updated_at": "...", "updated_by": "agent-1"} +``` + +**PUT /v1/sdlc/phase** — Set phase: +```json +{"phase": "testing", "agent_id": "agent-1"} +``` + +Valid phases: `planning` → `development` → `testing` → `review` → `deployment` + +**PUT /v1/sdlc/mode** — Set quality mode: +```json +{"mode": "enterprise", "agent_id": "agent-1"} +``` + +Valid modes: `quick` | `standard` | `enterprise` + +Phase and mode are stored in `brain.sqlite`. They persist across server restarts and affect every subsequent context packet. + +--- + +### 7. Decision Log & Learning — `POST /v1/decision` + +Agents log completed work. The Brain extracts co-occurrence patterns from the `related_entities` field, building a project-specific knowledge base of "when you edit X, you usually also edit Y." + +**Request:** +```json +{ + "agent_id": "claude-session-42", + "phase": "development", + "entity_name": "AuthService", + "action": "edit", + "related_entities": ["TokenValidator", "SessionStore"], + "outcome": "success", + "notes": "Added OAuth2 token validation path" +} +``` + +Internally: For each `(AuthService, TokenValidator)` and `(AuthService, SessionStore)` pair, `co_count` is incremented in `context_patterns`. `confidence = co_count / total_count`. These patterns surface in future context packets as `pattern_hints`. + +--- + +### 8. Pattern Query — `GET /v1/patterns?trigger=AuthService&limit=5` + +Returns the top learned co-occurrence patterns for an entity. Used for debugging the learning loop. + +**Response:** +```json +{ + "patterns": [ + {"trigger": "AuthService", "co_change": "TokenValidator", "reason": "edit during development", "confidence": 0.87}, + {"trigger": "AuthService", "co_change": "SessionStore", "reason": "edit during development", "confidence": 0.65} + ], + "count": 2 +} +``` + +--- + +### 9. Health — `GET /v1/health` + +```json +{"status": "ok", "model": "qwen2.5-coder:1.5b", "available": true} +``` + +`available: false` means Ollama is not reachable. All fast-path (SQLite-only) operations still work. Only LLM calls fail. + +--- + +## CLI Reference + +``` +brain serve Start the HTTP sidecar (default port 11435) + -port Override port + -model Override Ollama model + +brain status Show Ollama connectivity, model, stats, SDLC config +brain ingest Manually ingest a code snippet +brain summaries List all stored summaries +brain sdlc Show current phase and mode +brain sdlc phase

Set phase (planning|development|testing|review|deployment) +brain sdlc mode Set mode (quick|standard|enterprise) +brain decisions List recent agent decision log +brain patterns List all learned co-occurrence patterns +brain reset Clear all brain data (prompts for confirmation) +brain version Print version +``` + +--- + +## Configuration + +Set via `BRAIN_CONFIG` environment variable pointing to a JSON file: + +```json +{ + "enabled": true, + "model": "qwen2.5-coder:1.5b", + "ollama_url": "http://localhost:11434", + "timeout_ms": 3000, + "db_path": "~/.synapses/brain.sqlite", + "port": 11435, + "ingest": true, + "enrich": true, + "guardian": true, + "orchestrate": true, + "context_builder": true, + "learning_enabled": true, + "default_phase": "development", + "default_mode": "standard" +} +``` + +**Model tiers by RAM:** + +| RAM | Model | Size | Notes | +|-----|-------|------|-------| +| 4 GB | `qwen2.5-coder:1.5b` | ~900 MB | Default, good enough | +| 4 GB+ | `qwen3:1.7b` | ~1.1 GB | Recommended upgrade | +| 8 GB+ | `qwen3:4b` | ~2.5 GB | Noticeably better quality | +| 16 GB+ | `qwen3:8b` | ~5 GB | Enterprise quality | + +--- + +## Integration with Synapses Core + +### Overview + +Synapses is the structural layer — it parses codebases into a graph and serves relevance-ranked context to agents. The Brain is the semantic layer — it adds meaning, phase awareness, and learned patterns on top of that structure. + +Integration replaces Synapses' raw `get_context` graph dump with a Brain-assembled Context Packet. The fallback (raw graph) is always preserved. + +### 3 Integration Points + +#### Point 1: `get_context` → Context Packet (highest priority) + +**Location:** `internal/mcp/tools.go` in the `handleGetContext` function, after `CarveEgoGraph()` returns the subgraph. + +**What Synapses must send:** + +```go +// After carving the ego graph: +snapshot := brain.SynapsesSnapshotInput{ + RootNodeID: rootNode.ID, // stable graph node ID + RootName: rootNode.Name, // "AuthService" + RootType: rootNode.Type, // "struct" + RootFile: rootNode.Metadata["file"], // "internal/auth/service.go" + CalleeNames: extractNames(subgraph, CALLS_OUT), // direct callees + CallerNames: extractNames(subgraph, CALLS_IN), // direct callers + RelatedNames: extractNames(subgraph, ALL), // transitive neighbours + ApplicableRules: matchRulesForFile(rootNode.Metadata["file"]), // see below + ActiveClaims: getAllActiveClaims(), // from claims table + TaskContext: req.TaskContext, // from tool call params + TaskID: req.TaskID, +} +pkt, err := brainClient.BuildContextPacket(ctx, brain.ContextPacketRequest{ + AgentID: req.AgentID, // from MCP session or generated + Snapshot: snapshot, + Phase: "", // "" = use stored project phase + QualityMode: "", // "" = use stored project mode + EnableLLM: cfg.BrainEnableLLM, // configurable, default true +}) +if err != nil || pkt == nil { + // Brain unavailable or returned nil — fall back to raw graph output, no error + return formatRawGraph(subgraph) +} +return formatContextPacket(pkt) +``` + +**ApplicableRules extraction** — Synapses has architectural rules in its SQLite (from synapses.json). For each rule, check if `from_file_pattern` glob-matches `rootNode.Metadata["file"]`. Pass only matching rules: + +```go +func matchRulesForFile(file string) []brain.RuleInput { + rules := store.GetAllRules() + var out []brain.RuleInput + for _, r := range rules { + if r.FromFilePattern == "" || glob.Match(r.FromFilePattern, file) { + out = append(out, brain.RuleInput{ + RuleID: r.ID, + Severity: r.Severity, + Description: r.Description, + }) + } + } + return out +} +``` + +#### Point 2: File indexing → Auto-Ingest (background, non-blocking) + +**Location:** Synapses' file watcher / indexer pipeline, after a node is indexed or re-indexed. + +**What Synapses must do:** + +```go +// Fire-and-forget: do not block the indexing pipeline +go func() { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + brainClient.Ingest(ctx, brain.IngestRequest{ + NodeID: node.ID, + NodeName: node.Name, + NodeType: node.Type, + Package: node.Package, + Code: node.Code, // truncated to 500 chars by the Brain + }) +}() +``` + +This is the cold-start solution: every time Synapses indexes a file, the Brain learns its meaning. After a full project index, all nodes have summaries. The Brain doesn't need any bootstrap step — it learns continuously as Synapses works. + +#### Point 3: `get_violations` → Explanation Enrichment (optional, cached) + +**Location:** `internal/mcp/tools.go` in the `handleGetViolations` function, after collecting violations. + +```go +for i, v := range violations { + resp, err := brainClient.ExplainViolation(ctx, brain.ViolationRequest{ + RuleID: v.RuleID, + RuleSeverity: v.Severity, + Description: v.Description, + SourceFile: v.SourceFile, + TargetName: v.TargetName, + }) + if err == nil && resp.Explanation != "" { + violations[i].Explanation = resp.Explanation + violations[i].Fix = resp.Fix + } +} +``` + +Results are cached by the Brain for 7 days — the first call for each `(rule_id, file)` pair is slow (~2s); all subsequent calls for the same pair are instant. + +### BrainClient Synapses Needs + +A minimal HTTP client with fail-silent semantics: + +```go +type BrainClient struct { + baseURL string // "http://localhost:11435" + httpClient *http.Client // with Timeout: 5s +} + +func (c *BrainClient) BuildContextPacket(ctx context.Context, req brain.ContextPacketRequest) (*brain.ContextPacket, error) { + // POST /v1/context-packet + // On HTTP 204: return nil, nil + // On any error: return nil, nil (fail-silent) +} + +func (c *BrainClient) Ingest(ctx context.Context, req brain.IngestRequest) { + // POST /v1/ingest — fire-and-forget, ignore all errors +} + +func (c *BrainClient) ExplainViolation(ctx context.Context, req brain.ViolationRequest) (brain.ViolationResponse, error) { + // POST /v1/explain-violation + // On any error: return empty response, nil (fail-silent) +} + +func (c *BrainClient) Available(ctx context.Context) bool { + // GET /v1/health — used at startup only +} +``` + +### `synapses.json` Configuration (proposed additions) + +```json +{ + "brain": { + "enabled": true, + "url": "http://localhost:11435", + "timeout_ms": 5000, + "enable_llm": true + } +} +``` + +--- + +## Integration Challenges + +### Challenge 1: Node ID Stability Contract + +**Problem:** The Brain stores summaries keyed by Synapses node IDs. If a function is moved to a different file or package, its node ID changes — but the old summary stays in `semantic_summaries` under the old ID, and the new node ID has no summary. + +**Current state:** Unresolved. Silent miss — `get_context` falls back to raw output for that node. + +**Planned fix (v0.4.0):** Node lookup fallback by `(node_name, package)` when `node_id` lookup misses. Two-step: try exact ID, then try name-based lookup. Add `InvalidateNode(oldID)` to the BrainClient so Synapses can clean up on rename. + +--- + +### Challenge 2: Agent ID Propagation + +**Problem:** The context packet is personalized per agent (`agent_id` affects TeamStatus filtering). MCP clients don't advertise a stable agent ID. Claude Code doesn't pass `agent_id` as a tool parameter. + +**Current state:** `agent_id` is empty in most real calls. TeamStatus shows all other agents including "self" if self-filtering fails. + +**Planned fix:** Synapses generates a stable session ID per MCP connection (UUID on connect, stored in session state). Pass this as `agent_id` on every context packet request. Document the convention in the MCP tool schema. + +--- + +### Challenge 3: Cold Start — No Summaries on Fresh Install + +**Problem:** A developer installs the Brain, starts it, and immediately calls `get_context`. The Brain knows nothing — zero summaries in `semantic_summaries`. Context packets have `packet_quality: 0.0` and provide no value until nodes are ingested. + +**Current state:** Auto-ingest on file-save (Integration Point 2) solves this progressively, but only after the developer edits files. + +**Planned fix (v0.4.0):** On Brain startup (or on first `brain serve`), call Synapses' `GET /synapses/nodes` (new endpoint) to get all currently-indexed nodes with their code, and bulk-ingest them in a background goroutine. Progress reported via `brain status`. + +Alternatively: Synapses calls `POST /v1/ingest` for every node at startup if the brain is available. This is the cleaner approach — Synapses initiates the bulk ingest, Brain processes it. + +--- + +### Challenge 4: Context Packet Latency on LLM Path + +**Problem:** `get_context` is expected to respond in <100ms. The LLM insight path adds 2-3 seconds. Even with the insight cache, a cold miss blocks the entire `get_context` call. + +**Current state:** The 6-hour insight cache handles warm requests. Cold requests (new entity or cache expired) are slow. + +**Planned fix (v0.4.0):** Two-phase response: +1. Return the packet immediately with `llm_used: false` and whatever is in SQLite (fast path, <5ms). +2. Asynchronously generate the insight and store it in `insight_cache`. +3. On the next call for the same entity, the cache hit serves the full packet instantly. + +This changes the contract: `enable_llm: true` means "generate insight if not cached; do not block." + +--- + +### Challenge 5: Rules Data Is in Synapses, Not in Brain + +**Problem:** To populate `ApplicableRules` in the context packet request, Synapses must do a glob-match of every architectural rule against the root file. Synapses stores rules in its own SQLite. The Brain has no direct access to Synapses' data. + +**Current state:** The Brain accepts `ApplicableRules` in the request — Synapses must compute and pass them. This is the correct architecture (Synapses knows its rules; Brain enriches with explanations). + +**Implementation detail:** Synapses needs a `matchRulesForFile(file string) []RuleInput` helper that reads from its `rules` table and pattern-matches. This is ~20 lines of Go and straightforward. + +--- + +### Challenge 6: Claims Data Lives in Synapses' SQLite + +**Problem:** `TeamStatus` in the context packet requires `ActiveClaims` — which agent is working on what. But work claims are stored in Synapses' `claims` table (managed by the `claim_work` MCP tool). The Brain has no access to that table. + +**Current state:** Synapses must read its own claims table and pass all active (non-expired) claims in the context packet request. This is correct architecture. + +**Implementation detail:** `getAllActiveClaims()` — a SQL query against Synapses' `claims` table: +```sql +SELECT agent_id, scope, scope_type, expires_at FROM claims WHERE expires_at > datetime('now') +``` +Pass results as `ClaimInput` array in the request. + +--- + +## Improvements for Next Version (v0.4.0) + +### Improvement 1: Async Insight Generation (Critical for UX) + +Change `BuildContextPacket` to never block on LLM. Return fast-path packet immediately. Generate insight asynchronously. The client gets `packet_quality: 0.5` first time, `1.0` on next call (cache hit). This eliminates the 2-3s cold-miss latency entirely from the user's perspective. + +### Improvement 2: Bulk Bootstrap Ingest + +`brain ingest-all --synapses-url http://localhost:11434` — reads all indexed nodes from Synapses via a new REST endpoint, ingest them all in parallel goroutines (pool of 4 workers). Prints progress. Solves cold start. + +### Improvement 3: Node ID Change Notification + +Add `DELETE /v1/summary/{nodeId}` endpoint. When Synapses detects a node rename or deletion during re-indexing, it calls this to clean up stale summaries. Prevents ghost summaries accumulating for deleted code. + +### Improvement 4: PacketQuality Scoring Refinement + +Current weights (root +0.4, deps +0.1, insight +0.5) over-weight insight. A packet with 8 dep summaries but no insight scores 0.5, while a packet with insight but no dep summaries scores 0.9. Better formula: + +``` +quality = 0.0 +if root_summary: +0.35 +if dep_summaries > 0: +0.25 (scaled: min(len/8, 1.0) * 0.25) +if insight: +0.40 +``` + +This makes dep summaries worth 25% and insight worth 40% — reflecting that a full semantic map of the neighborhood is more useful than a single insight sentence. + +### Improvement 5: Language-Aware Prompts + +Current prompts are language-agnostic ("describe this code entity"). Add a `Language` field to `IngestRequest` (detectable from file extension by Synapses). Use language-specific prompt variants: Go prompts emphasize interfaces and goroutines; TypeScript prompts emphasize React components and async patterns; Python prompts emphasize class hierarchies and generators. + +### Improvement 6: Insight Cache Warming on Phase Change + +When `PUT /v1/sdlc/phase` is called, the new phase has different section flags. For the testing phase, LLM insight is disabled entirely — the cache for `development` phase is irrelevant. On phase transitions, proactively warm the cache for entities that were recently active (top-N from `decision_log`). This eliminates cold-start latency at the beginning of each phase. + +### Improvement 7: Confidence Decay in Pattern Learning + +Current co-occurrence learning has no temporal decay. A pattern from 6 months ago with 50 co-occurrences has the same confidence as a recent pattern. Add `last_seen_at` tracking. Decay confidence of old patterns: `effective_confidence = confidence * exp(-days_since_last / 30)`. Recent patterns rank higher than stale ones. + +--- + +## Fail-Silent Guarantee + +Every caller integration must treat the Brain as optional. The fail-silent contract: + +| Situation | Brain Response | Synapses Behavior | +|-----------|---------------|-------------------| +| Brain process not running | Connection refused | Use raw graph output | +| Brain running, Ollama down | HTTP 200, llm_used=false | Use fast-path packet (summaries only) | +| Brain returns HTTP 204 | No content | Use raw graph output | +| Brain times out (>5s) | Timeout error | Use raw graph output | +| Brain returns malformed JSON | Parse error | Use raw graph output | + +The NullBrain (`pkg/brain.NullBrain`) implements the full Brain interface with zero-value returns and no errors — safe to use when the Brain module is not compiled in. + +--- + +## Quick Start + +```bash +# 1. Install Ollama +curl -fsSL https://ollama.com/install.sh | sh +ollama pull qwen2.5-coder:1.5b + +# 2. Start the Brain +brain serve + +# 3. Verify +curl http://localhost:11435/v1/health +# {"status":"ok","model":"qwen2.5-coder:1.5b","available":true} + +# 4. Ingest a code snippet +curl -X POST http://localhost:11435/v1/ingest \ + -H 'Content-Type: application/json' \ + -d '{"node_id":"pkg:auth:Validate","node_name":"Validate","node_type":"func","package":"auth","code":"func Validate(token string) (Claims, error) { ... }"}' + +# 5. Build a context packet +curl -X POST http://localhost:11435/v1/context-packet \ + -H 'Content-Type: application/json' \ + -d '{"snapshot":{"root_node_id":"pkg:auth:Validate","root_name":"Validate"},"enable_llm":false}' +``` diff --git a/cmd/brain/main.go b/cmd/brain/main.go index 2eca55b..bfaa68f 100644 --- a/cmd/brain/main.go +++ b/cmd/brain/main.go @@ -26,7 +26,7 @@ import ( "github.com/synapses/synapses-intelligence/server" ) -const version = "0.1.0" +const version = "0.3.0" func main() { if len(os.Args) < 2 { @@ -60,6 +60,12 @@ func main() { cmdIngest(cfg, os.Args[2:]) case "summaries": cmdSummaries(cfg) + case "sdlc": + cmdSDLC(cfg, os.Args[2:]) + case "decisions": + cmdDecisions(cfg, os.Args[2:]) + case "patterns": + cmdPatterns(cfg) case "reset": cmdReset(cfg) case "version", "--version", "-v": @@ -139,6 +145,14 @@ func cmdStatus(cfg config.BrainConfig) { printFeature("enrich", cfg.Enrich) printFeature("guardian", cfg.Guardian) printFeature("orchestrate", cfg.Orchestrate) + printFeature("context_builder", cfg.ContextBuilder) + printFeature("learning", cfg.LearningEnabled) + + // SDLC config (reuse b from above). + sdlcCfg := b.GetSDLCConfig() + fmt.Printf("\nSDLC:\n") + fmt.Printf(" %-12s %s\n", "phase", sdlcCfg.Phase) + fmt.Printf(" %-12s %s\n", "mode", sdlcCfg.QualityMode) } func printFeature(name string, enabled bool) { @@ -222,6 +236,133 @@ func cmdSummaries(cfg config.BrainConfig) { fmt.Printf("\nTotal: %d summaries\n", len(summaries)) } +// cmdSDLC shows or sets the SDLC phase and quality mode. +// +// brain sdlc Show current phase and mode +// brain sdlc phase

Set phase (planning|development|testing|review|deployment) +// brain sdlc mode Set mode (quick|standard|enterprise) +func cmdSDLC(cfg config.BrainConfig, args []string) { + b := brain.New(cfg) + + if len(args) == 0 { + // Show current SDLC config. + cfg := b.GetSDLCConfig() + fmt.Printf("Phase: %s\n", cfg.Phase) + fmt.Printf("Quality Mode: %s\n", cfg.QualityMode) + if cfg.UpdatedAt != "" { + fmt.Printf("Updated: %s", cfg.UpdatedAt) + if cfg.UpdatedBy != "" { + fmt.Printf(" by %s", cfg.UpdatedBy) + } + fmt.Println() + } + return + } + + if len(args) < 2 { + fmt.Fprintln(os.Stderr, "brain sdlc: usage: brain sdlc phase OR brain sdlc mode ") + os.Exit(1) + } + + switch args[0] { + case "phase": + if err := b.SetSDLCPhase(brain.SDLCPhase(args[1]), "cli"); err != nil { + fmt.Fprintf(os.Stderr, "brain sdlc phase: %v\n", err) + os.Exit(1) + } + fmt.Printf("Phase set to: %s\n", args[1]) + case "mode": + if err := b.SetQualityMode(brain.QualityMode(args[1]), "cli"); err != nil { + fmt.Fprintf(os.Stderr, "brain sdlc mode: %v\n", err) + os.Exit(1) + } + fmt.Printf("Quality mode set to: %s\n", args[1]) + default: + fmt.Fprintf(os.Stderr, "brain sdlc: unknown subcommand %q — use 'phase' or 'mode'\n", args[0]) + os.Exit(1) + } +} + +// cmdDecisions lists recent decision log entries. +// +// brain decisions Show last 20 decisions +// brain decisions Show decisions for a specific entity +func cmdDecisions(cfg config.BrainConfig, args []string) { + st, err := store.Open(cfg.DBPath) + if err != nil { + fmt.Fprintf(os.Stderr, "brain decisions: open store: %v\n", err) + os.Exit(1) + } + defer st.Close() + + entity := "" + if len(args) > 0 { + entity = args[0] + } + + entries, err := st.GetRecentDecisions(entity, 20) + if err != nil { + fmt.Fprintf(os.Stderr, "brain decisions: query: %v\n", err) + os.Exit(1) + } + + if len(entries) == 0 { + if entity != "" { + fmt.Printf("No decisions recorded for entity %q.\n", entity) + } else { + fmt.Println("No decisions recorded yet. Agents call POST /v1/decision to log decisions.") + } + return + } + + fmt.Printf("%-20s %-12s %-20s %-8s %s\n", "ENTITY", "PHASE", "ACTION", "OUTCOME", "AGENT") + fmt.Println(repeat("-", 90)) + for _, e := range entries { + fmt.Printf("%-20s %-12s %-20s %-8s %s\n", + truncate(e.EntityName, 18), + truncate(e.Phase, 10), + truncate(e.Action, 18), + truncate(e.Outcome, 6), + truncate(e.AgentID, 20), + ) + } + fmt.Printf("\nShowing %d entries\n", len(entries)) +} + +// cmdPatterns lists learned co-occurrence patterns. +func cmdPatterns(cfg config.BrainConfig) { + st, err := store.Open(cfg.DBPath) + if err != nil { + fmt.Fprintf(os.Stderr, "brain patterns: open store: %v\n", err) + os.Exit(1) + } + defer st.Close() + + patterns, err := st.AllPatterns() + if err != nil { + fmt.Fprintf(os.Stderr, "brain patterns: query: %v\n", err) + os.Exit(1) + } + + if len(patterns) == 0 { + fmt.Println("No patterns learned yet. Patterns are built from POST /v1/decision calls.") + return + } + + fmt.Printf("%-25s %-25s %6s %6s %s\n", "TRIGGER", "CO-CHANGE", "CONF", "COUNT", "REASON") + fmt.Println(repeat("-", 100)) + for _, p := range patterns { + fmt.Printf("%-25s %-25s %6.2f %6d %s\n", + truncate(p.Trigger, 23), + truncate(p.CoChange, 23), + p.Confidence, + p.CoCount, + truncate(p.Reason, 40), + ) + } + fmt.Printf("\nTotal: %d patterns\n", len(patterns)) +} + // cmdReset clears all brain data. func cmdReset(cfg config.BrainConfig) { fmt.Printf("This will delete all summaries and violation cache from %s.\n", cfg.DBPath) @@ -255,13 +396,21 @@ Usage: brain [flags] Commands: - serve Start the HTTP sidecar server (default port: 11435) - Flags: -port , -model - status Show Ollama connectivity, model, and SQLite stats - ingest Manually ingest a code snippet (JSON via argument or stdin) - summaries List all stored semantic summaries - reset Clear all brain data (prompts for confirmation) - version Print version + serve Start the HTTP sidecar server (default port: 11435) + Flags: -port , -model + status Show Ollama connectivity, model, SQLite stats, SDLC config + ingest Manually ingest a code snippet (JSON via argument or stdin) + summaries List all stored semantic summaries + sdlc Show or set SDLC phase and quality mode + Examples: + brain sdlc + brain sdlc phase testing + brain sdlc mode enterprise + decisions List recent agent decision log entries + brain decisions [entity_name] + patterns List learned co-occurrence patterns + reset Clear all brain data (prompts for confirmation) + version Print version Environment: BRAIN_CONFIG Path to a JSON config file (optional) @@ -277,7 +426,11 @@ Config example (brain.json): "enabled": true, "model": "qwen2.5-coder:1.5b", "ollama_url": "http://localhost:11434", - "timeout_ms": 3000 + "timeout_ms": 3000, + "context_builder": true, + "learning_enabled": true, + "default_phase": "development", + "default_mode": "standard" } Model tiers (by system RAM): diff --git a/internal/contextbuilder/builder.go b/internal/contextbuilder/builder.go index 33f5bfc..521b141 100644 --- a/internal/contextbuilder/builder.go +++ b/internal/contextbuilder/builder.go @@ -63,8 +63,14 @@ type Packet struct { Insight string Concerns []string + LLMUsed bool // true when the LLM was called for this packet ActiveConstraints []ConstraintItem + // PacketQuality is a 0.0-1.0 heuristic reflecting how complete the packet is: + // 1.0 = root summary + dep summaries + LLM insight all present + // 0.5 = root summary present, no LLM insight + // 0.0 = empty packet (no summaries ingested yet) + PacketQuality float64 TeamStatus []AgentItem Gate sdlc.Gate PatternHints []PatternItem @@ -150,22 +156,32 @@ func (b *Builder) Build(ctx context.Context, req Request) (*Packet, error) { } } - // Section 2: LLM Insight (slow path — optional). - if sections.LLMInsight && req.EnableLLM && b.enr != nil { - r, err := b.enr.Enrich(ctx, enricher.Request{ - RootID: req.RootNodeID, - RootName: req.RootName, - RootType: req.RootType, - CalleeNames: req.CalleeNames, - CallerNames: req.CallerNames, - RelatedNames: req.RelatedNames, - TaskContext: req.TaskContext, - }) - if err == nil { - pkt.Insight = r.Insight - pkt.Concerns = r.Concerns + // Section 2: LLM Insight (cache-first; slow path only on cache miss). + if sections.LLMInsight && req.EnableLLM && b.enr != nil && req.RootNodeID != "" { + // Fast path: check cache first (entries live 6h, pruned at startup). + if cached, ok := b.store.GetInsightCache(req.RootNodeID, phase); ok { + pkt.Insight = cached.Insight + pkt.Concerns = cached.Concerns + // LLMUsed stays false — served from cache, no live LLM call + } else { + r, err := b.enr.Enrich(ctx, enricher.Request{ + RootID: req.RootNodeID, + RootName: req.RootName, + RootType: req.RootType, + CalleeNames: req.CalleeNames, + CallerNames: req.CallerNames, + RelatedNames: req.RelatedNames, + TaskContext: req.TaskContext, + }) + if err == nil { + pkt.Insight = r.Insight + pkt.Concerns = r.Concerns + pkt.LLMUsed = r.LLMUsed + // Store in cache for future requests. + _ = b.store.UpsertInsightCache(req.RootNodeID, phase, r.Insight, r.Concerns) + } + // Error is non-fatal — insight section stays empty. } - // Error is non-fatal — insight section stays empty. } // Section 3: Active constraints. @@ -195,6 +211,9 @@ func (b *Builder) Build(ctx context.Context, req Request) (*Packet, error) { pkt.PhaseGuidance = sdlc.PhaseGuidance(phase, mode) } + // Compute packet quality: 0.0 (empty) → 0.5 (summaries) → 1.0 (full with insight). + pkt.PacketQuality = computeQuality(pkt) + return pkt, nil } @@ -303,3 +322,26 @@ func toPatternItems(patterns []store.ContextPattern) []PatternItem { } return out } + +// computeQuality returns a 0.0–1.0 heuristic for how complete a packet is. +// +// Scoring: +// - RootSummary present: +0.4 +// - At least one DependencySummary: +0.1 +// - LLM Insight present (live or cached): +0.5 +func computeQuality(pkt *Packet) float64 { + var q float64 + if pkt.RootSummary != "" { + q += 0.4 + } + if len(pkt.DependencySummaries) > 0 { + q += 0.1 + } + if pkt.Insight != "" { + q += 0.5 + } + if q > 1.0 { + q = 1.0 + } + return q +} diff --git a/internal/contextbuilder/builder_test.go b/internal/contextbuilder/builder_test.go index 08bebfd..3e2c05e 100644 --- a/internal/contextbuilder/builder_test.go +++ b/internal/contextbuilder/builder_test.go @@ -241,3 +241,68 @@ func TestBuild_EmptySnapshot_NoErrors(t *testing.T) { t.Fatal("Build returned nil on empty request") } } + +func TestBuild_PacketQuality_NoData(t *testing.T) { + b, _ := newTestBuilder(t, "") + pkt, err := b.Build(context.Background(), Request{ + Phase: sdlc.PhaseDevelopment, + QualityMode: sdlc.ModeStandard, + RootName: "UnknownEntity", + }) + if err != nil { + t.Fatalf("Build error: %v", err) + } + if pkt.PacketQuality != 0.0 { + t.Errorf("PacketQuality with no data want 0.0, got %f", pkt.PacketQuality) + } +} + +func TestBuild_PacketQuality_WithSummary(t *testing.T) { + b, st := newTestBuilder(t, "") + st.UpsertSummary("node:svc:Foo", "Foo", "Does the foo thing.", []string{}) + + pkt, err := b.Build(context.Background(), Request{ + Phase: sdlc.PhaseDevelopment, + QualityMode: sdlc.ModeStandard, + RootNodeID: "node:svc:Foo", + RootName: "Foo", + }) + if err != nil { + t.Fatalf("Build error: %v", err) + } + // Root summary present → 0.4; no dep summaries, no insight. + if pkt.PacketQuality != 0.4 { + t.Errorf("PacketQuality with root summary want 0.4, got %f", pkt.PacketQuality) + } +} + +func TestBuild_InsightCache_HitSkipsLLM(t *testing.T) { + // Builder with a real mock LLM (to ensure LLM is NOT called on cache hit). + b, st := newTestBuilder(t, `{"insight": "should not appear", "concerns": []}`) + + // Real-world order: ingest first, then cache the insight. + // UpsertSummary invalidates insight cache, so it must come before UpsertInsightCache. + st.UpsertSummary("node:svc:Bar", "Bar", "Does bar.", []string{}) + st.UpsertInsightCache("node:svc:Bar", sdlc.PhaseDevelopment, "Cached insight.", []string{"cached concern"}) + + pkt, err := b.Build(context.Background(), Request{ + Phase: sdlc.PhaseDevelopment, + QualityMode: sdlc.ModeStandard, + EnableLLM: true, + RootNodeID: "node:svc:Bar", + RootName: "Bar", + }) + if err != nil { + t.Fatalf("Build error: %v", err) + } + if pkt.Insight != "Cached insight." { + t.Errorf("expected cached insight, got %q", pkt.Insight) + } + if len(pkt.Concerns) == 0 || pkt.Concerns[0] != "cached concern" { + t.Errorf("expected cached concerns, got %v", pkt.Concerns) + } + // Cache hit means LLMUsed should be false. + if pkt.LLMUsed { + t.Error("LLMUsed should be false on insight cache hit") + } +} diff --git a/internal/enricher/enricher.go b/internal/enricher/enricher.go index bae95d8..92d7d8b 100644 --- a/internal/enricher/enricher.go +++ b/internal/enricher/enricher.go @@ -47,6 +47,7 @@ type Request struct { type Response struct { Insight string Concerns []string + LLMUsed bool // true when the LLM was called; false on cache hit (future) } type insightJSON struct { @@ -83,7 +84,7 @@ func (e *Enricher) Enrich(ctx context.Context, req Request) (Response, error) { result, err := parseInsight(raw) if err != nil { - return Response{}, fmt.Errorf("parse insight: %w (raw: %q)", err, truncate(raw, 100)) + return Response{}, fmt.Errorf("parse insight: %w (raw: %q)", err, llm.Truncate(raw, 100)) } return result, nil @@ -116,7 +117,7 @@ func (e *Enricher) buildPrompt(req Request) string { } func parseInsight(raw string) (Response, error) { - raw = extractJSON(raw) + raw = llm.ExtractJSON(raw) var result insightJSON if err := json.Unmarshal([]byte(raw), &result); err != nil { return Response{}, fmt.Errorf("unmarshal: %w", err) @@ -125,7 +126,7 @@ func parseInsight(raw string) (Response, error) { if insight == "" { return Response{}, fmt.Errorf("empty insight in response") } - return Response{Insight: insight, Concerns: result.Concerns}, nil + return Response{Insight: insight, Concerns: result.Concerns, LLMUsed: true}, nil } // joinNames joins up to n names into a comma-separated string. @@ -135,29 +136,3 @@ func joinNames(names []string, n int) string { } return strings.Join(names, ", ") } - -func extractJSON(s string) string { - s = strings.TrimSpace(s) - if idx := strings.Index(s, "```"); idx >= 0 { - s = s[idx:] - s = strings.TrimPrefix(s, "```json") - s = strings.TrimPrefix(s, "```") - if end := strings.Index(s, "```"); end >= 0 { - s = s[:end] - } - } - if start := strings.Index(s, "{"); start >= 0 { - s = s[start:] - } - if end := strings.LastIndex(s, "}"); end >= 0 { - s = s[:end+1] - } - return strings.TrimSpace(s) -} - -func truncate(s string, n int) string { - if len(s) <= n { - return s - } - return s[:n] + "..." -} diff --git a/internal/guardian/guardian.go b/internal/guardian/guardian.go index 22f4e7a..f941633 100644 --- a/internal/guardian/guardian.go +++ b/internal/guardian/guardian.go @@ -79,7 +79,7 @@ func (g *Guardian) Explain(ctx context.Context, req Request) (Response, error) { result, err := parseViolation(raw) if err != nil { - return Response{}, fmt.Errorf("parse response: %w (raw: %q)", err, truncate(raw, 100)) + return Response{}, fmt.Errorf("parse response: %w (raw: %q)", err, llm.Truncate(raw, 100)) } // Cache for future calls with the same (rule, file) pair. @@ -108,7 +108,7 @@ func (g *Guardian) buildPrompt(req Request) string { } func parseViolation(raw string) (Response, error) { - raw = extractJSON(raw) + raw = llm.ExtractJSON(raw) var result violationJSON if err := json.Unmarshal([]byte(raw), &result); err != nil { return Response{}, fmt.Errorf("unmarshal: %w", err) @@ -122,29 +122,3 @@ func parseViolation(raw string) (Response, error) { Fix: strings.TrimSpace(result.Fix), }, nil } - -func extractJSON(s string) string { - s = strings.TrimSpace(s) - if idx := strings.Index(s, "```"); idx >= 0 { - s = s[idx:] - s = strings.TrimPrefix(s, "```json") - s = strings.TrimPrefix(s, "```") - if end := strings.Index(s, "```"); end >= 0 { - s = s[:end] - } - } - if start := strings.Index(s, "{"); start >= 0 { - s = s[start:] - } - if end := strings.LastIndex(s, "}"); end >= 0 { - s = s[:end+1] - } - return strings.TrimSpace(s) -} - -func truncate(s string, n int) string { - if len(s) <= n { - return s - } - return s[:n] + "..." -} diff --git a/internal/ingestor/ingestor.go b/internal/ingestor/ingestor.go index 0dd9e0b..94732b8 100644 --- a/internal/ingestor/ingestor.go +++ b/internal/ingestor/ingestor.go @@ -89,7 +89,7 @@ func (ing *Ingestor) Summarize(ctx context.Context, req Request) (Response, erro summary, tags, err := parseSummary(raw) if err != nil { - return Response{NodeID: req.NodeID}, fmt.Errorf("parse summary: %w (raw: %q)", err, truncate(raw, 100)) + return Response{NodeID: req.NodeID}, fmt.Errorf("parse summary: %w (raw: %q)", err, llm.Truncate(raw, 100)) } if err := ing.store.UpsertSummary(req.NodeID, req.NodeName, summary, tags); err != nil { @@ -118,7 +118,7 @@ func (ing *Ingestor) buildPrompt(req Request) string { // Handles cases where the model wraps the JSON in markdown code fences. // Tags are optional — returns nil if the model did not include them. func parseSummary(raw string) (summary string, tags []string, err error) { - raw = extractJSON(raw) + raw = llm.ExtractJSON(raw) var result summaryJSON if err = json.Unmarshal([]byte(raw), &result); err != nil { return "", nil, fmt.Errorf("unmarshal: %w", err) @@ -130,29 +130,6 @@ func parseSummary(raw string) (summary string, tags []string, err error) { return summary, result.Tags, nil } -// extractJSON strips markdown code fences if the LLM wrapped its output. -func extractJSON(s string) string { - s = strings.TrimSpace(s) - // Strip ```json ... ``` or ``` ... ``` - if idx := strings.Index(s, "```"); idx >= 0 { - s = s[idx:] - s = strings.TrimPrefix(s, "```json") - s = strings.TrimPrefix(s, "```") - if end := strings.Index(s, "```"); end >= 0 { - s = s[:end] - } - } - // Find the first { to skip any leading text - if start := strings.Index(s, "{"); start >= 0 { - s = s[start:] - } - // Find the last } to skip any trailing text - if end := strings.LastIndex(s, "}"); end >= 0 { - s = s[:end+1] - } - return strings.TrimSpace(s) -} - // truncateCode caps the code snippet at maxCodeChars runes. func truncateCode(code string) string { code = strings.TrimSpace(code) @@ -162,11 +139,3 @@ func truncateCode(code string) string { runes := []rune(code) return string(runes[:maxCodeChars]) + "..." } - -// truncate shortens a string for error messages. -func truncate(s string, n int) string { - if len(s) <= n { - return s - } - return s[:n] + "..." -} diff --git a/internal/ingestor/ingestor_test.go b/internal/ingestor/ingestor_test.go index 741dfd8..00a177d 100644 --- a/internal/ingestor/ingestor_test.go +++ b/internal/ingestor/ingestor_test.go @@ -128,9 +128,9 @@ func TestExtractJSON(t *testing.T) { {" \n{\"summary\": \"hello\"}\n", `{"summary": "hello"}`}, } for _, tc := range cases { - got := extractJSON(tc.input) + got := llm.ExtractJSON(tc.input) if got != tc.want { - t.Errorf("extractJSON(%q) = %q, want %q", tc.input, got, tc.want) + t.Errorf("ExtractJSON(%q) = %q, want %q", tc.input, got, tc.want) } } } diff --git a/internal/llm/util.go b/internal/llm/util.go new file mode 100644 index 0000000..4712e66 --- /dev/null +++ b/internal/llm/util.go @@ -0,0 +1,34 @@ +package llm + +import "strings" + +// ExtractJSON strips markdown code fences and extracts the JSON object from raw LLM output. +// Many small models wrap JSON responses in ```json ... ``` blocks despite instructions. +// This function handles that gracefully so callers always get raw JSON to unmarshal. +func ExtractJSON(s string) string { + s = strings.TrimSpace(s) + if idx := strings.Index(s, "```"); idx >= 0 { + s = s[idx:] + s = strings.TrimPrefix(s, "```json") + s = strings.TrimPrefix(s, "```") + if end := strings.Index(s, "```"); end >= 0 { + s = s[:end] + } + } + if start := strings.Index(s, "{"); start >= 0 { + s = s[start:] + } + if end := strings.LastIndex(s, "}"); end >= 0 { + s = s[:end+1] + } + return strings.TrimSpace(s) +} + +// Truncate shortens s to at most n bytes for use in error messages. +// Appends "..." when truncation occurs. +func Truncate(s string, n int) string { + if len(s) <= n { + return s + } + return s[:n] + "..." +} diff --git a/internal/orchestrator/orchestrator.go b/internal/orchestrator/orchestrator.go index 6e88616..deb2797 100644 --- a/internal/orchestrator/orchestrator.go +++ b/internal/orchestrator/orchestrator.go @@ -112,7 +112,7 @@ func (o *Orchestrator) fallbackResponse(req Request) Response { } func parseCoordinate(raw string) (Response, error) { - raw = extractJSON(raw) + raw = llm.ExtractJSON(raw) var result coordinateJSON if err := json.Unmarshal([]byte(raw), &result); err != nil { return Response{}, fmt.Errorf("unmarshal: %w", err) @@ -126,22 +126,3 @@ func parseCoordinate(raw string) (Response, error) { AlternativeScope: strings.TrimSpace(result.AlternativeScope), }, nil } - -func extractJSON(s string) string { - s = strings.TrimSpace(s) - if idx := strings.Index(s, "```"); idx >= 0 { - s = s[idx:] - s = strings.TrimPrefix(s, "```json") - s = strings.TrimPrefix(s, "```") - if end := strings.Index(s, "```"); end >= 0 { - s = s[:end] - } - } - if start := strings.Index(s, "{"); start >= 0 { - s = s[start:] - } - if end := strings.LastIndex(s, "}"); end >= 0 { - s = s[:end+1] - } - return strings.TrimSpace(s) -} diff --git a/internal/store/store.go b/internal/store/store.go index aa7bd4e..218132b 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -56,6 +56,18 @@ CREATE TABLE IF NOT EXISTS context_patterns ( CREATE INDEX IF NOT EXISTS idx_patterns_trigger ON context_patterns(trigger); CREATE INDEX IF NOT EXISTS idx_patterns_confidence ON context_patterns(confidence DESC); +-- Caches LLM-generated insight per (node_id, phase). TTL: 6 hours. +-- Avoids repeat LLM calls for unchanged code during the same work session. +CREATE TABLE IF NOT EXISTS insight_cache ( + node_id TEXT NOT NULL, + phase TEXT NOT NULL, + insight TEXT NOT NULL, + concerns TEXT NOT NULL DEFAULT '[]', + cached_at TEXT NOT NULL, + PRIMARY KEY (node_id, phase) +); +CREATE INDEX IF NOT EXISTS idx_insight_node ON insight_cache(node_id); + CREATE TABLE IF NOT EXISTS decision_log ( id TEXT PRIMARY KEY, agent_id TEXT NOT NULL DEFAULT '', @@ -107,15 +119,31 @@ func (s *Store) pruneOldData() { now := time.Now().UTC() // Prune decision log entries older than 30 days. cutoff30d := now.Add(-30 * 24 * time.Hour).Format(time.RFC3339) - s.db.Exec(`DELETE FROM decision_log WHERE created_at < ?`, cutoff30d) + if _, err := s.db.Exec(`DELETE FROM decision_log WHERE created_at < ?`, cutoff30d); err != nil { + fmt.Fprintf(os.Stderr, "brain store: prune decision_log: %v\n", err) + } // Prune weak, stale patterns (seen < 2 times AND older than 14 days). cutoff14d := now.Add(-14 * 24 * time.Hour).Format(time.RFC3339) - s.db.Exec(`DELETE FROM context_patterns WHERE co_count < 2 AND updated_at < ?`, cutoff14d) + if _, err := s.db.Exec(`DELETE FROM context_patterns WHERE co_count < 2 AND updated_at < ?`, cutoff14d); err != nil { + fmt.Fprintf(os.Stderr, "brain store: prune context_patterns: %v\n", err) + } + // Prune insight cache entries older than 6 hours (stale insight). + cutoff6h := now.Add(-6 * time.Hour).Format(time.RFC3339) + if _, err := s.db.Exec(`DELETE FROM insight_cache WHERE cached_at < ?`, cutoff6h); err != nil { + fmt.Fprintf(os.Stderr, "brain store: prune insight_cache: %v\n", err) + } + // Prune violation cache entries older than 7 days (rules can change). + cutoff7d := now.Add(-7 * 24 * time.Hour).Format(time.RFC3339) + if _, err := s.db.Exec(`DELETE FROM violation_cache WHERE cached_at < ?`, cutoff7d); err != nil { + fmt.Fprintf(os.Stderr, "brain store: prune violation_cache: %v\n", err) + } } // --- Semantic Summaries --- // UpsertSummary stores or updates the semantic summary and tags for a node. +// If the node already exists (re-ingest), the insight cache is invalidated for +// all phases — the old insight may no longer match the updated code. func (s *Store) UpsertSummary(nodeID, nodeName, summary string, tags []string) error { if tags == nil { tags = []string{} @@ -132,7 +160,12 @@ func (s *Store) UpsertSummary(nodeID, nodeName, summary string, tags []string) e updated_at = excluded.updated_at`, nodeID, nodeName, summary, string(tagsJSON), now, ) - return err + if err != nil { + return err + } + // Invalidate cached insight — code has changed, old insight is stale. + _, _ = s.db.Exec(`DELETE FROM insight_cache WHERE node_id = ?`, nodeID) + return nil } // GetSummary returns the stored summary for a node, or "" if not found. @@ -156,7 +189,9 @@ func (s *Store) GetSummaryWithTags(nodeID string) (summary string, tags []string if err != nil { return "", nil } - json.Unmarshal([]byte(tagsJSON), &tags) + if err := json.Unmarshal([]byte(tagsJSON), &tags); err != nil { + fmt.Fprintf(os.Stderr, "brain store: decode tags for node: %v\n", err) + } return summary, tags } @@ -268,6 +303,52 @@ func (s *Store) GetViolationExplanation(ruleID, sourceFile string) (explanation, return explanation, fix, true } +// --- Insight Cache --- + +// InsightCacheEntry is a stored LLM-generated insight for a (node_id, phase) pair. +type InsightCacheEntry struct { + Insight string + Concerns []string +} + +// GetInsightCache returns the cached insight for a (nodeID, phase) pair. +// Returns ("", nil, false) if not cached or if the entry was pruned (>6h old). +func (s *Store) GetInsightCache(nodeID, phase string) (entry InsightCacheEntry, found bool) { + var insight, concernsJSON string + err := s.db.QueryRow( + `SELECT insight, concerns FROM insight_cache WHERE node_id = ? AND phase = ?`, + nodeID, phase, + ).Scan(&insight, &concernsJSON) + if err != nil { + return InsightCacheEntry{}, false + } + var concerns []string + if err := json.Unmarshal([]byte(concernsJSON), &concerns); err != nil { + fmt.Fprintf(os.Stderr, "brain store: decode concerns for insight cache %s/%s: %v\n", nodeID, phase, err) + } + return InsightCacheEntry{Insight: insight, Concerns: concerns}, true +} + +// UpsertInsightCache stores a (nodeID, phase) → insight mapping. +// Existing entries are replaced (insight content may have improved). +func (s *Store) UpsertInsightCache(nodeID, phase, insight string, concerns []string) error { + if concerns == nil { + concerns = []string{} + } + concernsJSON, _ := json.Marshal(concerns) + now := time.Now().UTC().Format(time.RFC3339) + _, err := s.db.Exec(` + INSERT INTO insight_cache (node_id, phase, insight, concerns, cached_at) + VALUES (?, ?, ?, ?, ?) + ON CONFLICT(node_id, phase) DO UPDATE SET + insight = excluded.insight, + concerns = excluded.concerns, + cached_at = excluded.cached_at`, + nodeID, phase, insight, string(concernsJSON), now, + ) + return err +} + // --- SDLC Config --- // SDLCConfigRow is the stored project SDLC state. @@ -463,7 +544,9 @@ func (s *Store) GetRecentDecisions(entityName string, limit int) ([]DecisionLogE &e.Action, &relJSON, &e.Outcome, &e.Notes, &e.CreatedAt); err != nil { continue } - json.Unmarshal([]byte(relJSON), &e.RelatedEntities) + if err := json.Unmarshal([]byte(relJSON), &e.RelatedEntities); err != nil { + fmt.Fprintf(os.Stderr, "brain store: decode related_entities for decision %s: %v\n", e.ID, err) + } out = append(out, e) } return out, rows.Err() @@ -476,6 +559,7 @@ func (s *Store) Reset() error { _, err := s.db.Exec(` DELETE FROM semantic_summaries; DELETE FROM violation_cache; + DELETE FROM insight_cache; DELETE FROM sdlc_config; DELETE FROM context_patterns; DELETE FROM decision_log; diff --git a/pkg/brain/brain.go b/pkg/brain/brain.go index 0c25ad2..dc132af 100644 --- a/pkg/brain/brain.go +++ b/pkg/brain/brain.go @@ -67,6 +67,11 @@ type Brain interface { // GetSDLCConfig returns the current SDLC config. GetSDLCConfig() SDLCConfig + + // GetPatterns returns learned co-occurrence patterns sorted by confidence. + // If trigger is non-empty, only patterns with that trigger are returned. + // limit caps the number of results (0 = default of 20). + GetPatterns(trigger string, limit int) []PatternHint } // impl is the production Brain backed by Ollama + SQLite. @@ -133,7 +138,7 @@ func (b *impl) Ingest(ctx context.Context, req IngestRequest) (IngestResponse, e if err != nil { return IngestResponse{NodeID: req.NodeID}, err } - return IngestResponse{NodeID: r.NodeID, Summary: r.Summary}, nil + return IngestResponse{NodeID: r.NodeID, Summary: r.Summary, Tags: r.Tags}, nil } func (b *impl) Enrich(ctx context.Context, req EnrichRequest) (EnrichResponse, error) { @@ -161,6 +166,7 @@ func (b *impl) Enrich(ctx context.Context, req EnrichRequest) (EnrichResponse, e Insight: r.Insight, Concerns: r.Concerns, Summaries: summaries, + LLMUsed: r.LLMUsed, }, nil } @@ -282,6 +288,27 @@ func (b *impl) GetSDLCConfig() SDLCConfig { } } +func (b *impl) GetPatterns(trigger string, limit int) []PatternHint { + if limit <= 0 { + limit = 20 + } + var raw []store.ContextPattern + if trigger != "" { + raw = b.store.GetPatternsForTriggers([]string{trigger}, limit) + } else { + all, _ := b.store.AllPatterns() + if len(all) > limit { + all = all[:limit] + } + raw = all + } + out := make([]PatternHint, len(raw)) + for i, p := range raw { + out[i] = PatternHint{Trigger: p.Trigger, CoChange: p.CoChange, Reason: p.Reason, Confidence: p.Confidence} + } + return out +} + // --- conversion helpers --- func toBuilderRules(rules []RuleInput) []contextbuilder.RuleRef { @@ -313,6 +340,8 @@ func toContextPacket(p *contextbuilder.Packet) *ContextPacket { Insight: p.Insight, Concerns: p.Concerns, PhaseGuidance: p.PhaseGuidance, + LLMUsed: p.LLMUsed, + PacketQuality: p.PacketQuality, QualityGate: QualityGate{ RequireTests: p.Gate.RequireTests, RequireDocs: p.Gate.RequireDocs, diff --git a/pkg/brain/null.go b/pkg/brain/null.go index 8417228..fe369f2 100644 --- a/pkg/brain/null.go +++ b/pkg/brain/null.go @@ -39,3 +39,5 @@ func (n *NullBrain) SetQualityMode(_ QualityMode, _ string) error { return nil } func (n *NullBrain) GetSDLCConfig() SDLCConfig { return SDLCConfig{Phase: PhaseDevelopment, QualityMode: QualityStandard} } + +func (n *NullBrain) GetPatterns(_ string, _ int) []PatternHint { return nil } diff --git a/pkg/brain/types.go b/pkg/brain/types.go index 91c5fae..d177daa 100644 --- a/pkg/brain/types.go +++ b/pkg/brain/types.go @@ -21,8 +21,9 @@ type IngestRequest struct { // IngestResponse is returned after summarization. type IngestResponse struct { - NodeID string `json:"node_id"` - Summary string `json:"summary"` // 1-sentence intent summary + NodeID string `json:"node_id"` + Summary string `json:"summary"` // 1-sentence intent summary + Tags []string `json:"tags,omitempty"` // 1-3 domain labels, e.g. ["auth","http"] } // --- Enrich (Context Enricher) --- @@ -56,6 +57,9 @@ type EnrichResponse struct { // Summaries maps nodeID → 1-sentence summary for nodes that have been ingested. // These are loaded from brain.sqlite (no LLM call needed — fast lookup). Summaries map[string]string `json:"summaries"` + // LLMUsed is true when the LLM was called to generate the Insight field. + // False means Insight is empty (LLM unavailable, feature disabled, or timed out). + LLMUsed bool `json:"llm_used"` } // --- ExplainViolation (Rule Guardian) --- @@ -172,6 +176,15 @@ type ContextPacket struct { // Section 7: Phase Guidance — what the agent should do next PhaseGuidance string `json:"phase_guidance,omitempty"` + + // LLMUsed indicates whether a live LLM call was made during packet assembly. + // False means all data came from brain.sqlite (sub-millisecond path). + LLMUsed bool `json:"llm_used,omitempty"` + + // PacketQuality is a 0.0–1.0 heuristic reflecting how complete this packet is. + // 0.0 = no summaries ingested yet; 0.5 = summaries present, no insight; 1.0 = full. + // Agents can use this to decide whether to request a follow-up LLM enrichment pass. + PacketQuality float64 `json:"packet_quality"` } // ConstraintItem is a single architectural rule the agent must respect. @@ -212,45 +225,44 @@ type PatternHint struct { // --- Context Packet request/input types --- // ContextPacketRequest is the input to Brain.BuildContextPacket(). +// All fields are optional — empty Phase/QualityMode fall back to the stored project config. type ContextPacketRequest struct { - AgentID string - Snapshot SynapsesSnapshotInput - Phase SDLCPhase // "" = use stored project phase - QualityMode QualityMode // "" = use stored project mode - EnableLLM bool // true = allow LLM insight generation (adds ~2s) + AgentID string `json:"agent_id,omitempty"` + Snapshot SynapsesSnapshotInput `json:"snapshot"` + Phase SDLCPhase `json:"phase,omitempty"` // "" = use stored project phase + QualityMode QualityMode `json:"quality_mode,omitempty"` // "" = use stored project mode + EnableLLM bool `json:"enable_llm"` // true = allow LLM insight (~2s) } // SynapsesSnapshotInput carries the raw structural data from a Synapses get_context call. // Synapses (or the HTTP caller) populates this; the Brain uses it to build the packet. type SynapsesSnapshotInput struct { - RootNodeID string - RootName string - RootType string - RootFile string // used for constraint file pattern matching - CalleeNames []string // direct callees (what root calls) - CallerNames []string // direct callers (what calls root) - RelatedNames []string // transitive neighbors - // ApplicableRules are the architectural rules whose file pattern matches RootFile. - ApplicableRules []RuleInput - // ActiveClaims are work claims from other agents (for team coordination section). - ActiveClaims []ClaimInput - TaskContext string - TaskID string + RootNodeID string `json:"root_node_id,omitempty"` + RootName string `json:"root_name"` + RootType string `json:"root_type,omitempty"` + RootFile string `json:"root_file,omitempty"` // used for constraint hint lookups + CalleeNames []string `json:"callee_names,omitempty"` // what root calls directly + CallerNames []string `json:"caller_names,omitempty"` // what calls root directly + RelatedNames []string `json:"related_names,omitempty"` // transitive neighbours + ApplicableRules []RuleInput `json:"applicable_rules,omitempty"` // rules whose pattern matches RootFile + ActiveClaims []ClaimInput `json:"active_claims,omitempty"` // work claims from other agents + TaskContext string `json:"task_context,omitempty"` + TaskID string `json:"task_id,omitempty"` } // RuleInput is a single architectural rule reference. type RuleInput struct { - RuleID string - Severity string - Description string + RuleID string `json:"rule_id"` + Severity string `json:"severity"` + Description string `json:"description"` } // ClaimInput is a single work claim from another agent. type ClaimInput struct { - AgentID string - Scope string - ScopeType string - ExpiresAt string // RFC3339 + AgentID string `json:"agent_id"` + Scope string `json:"scope"` + ScopeType string `json:"scope_type"` + ExpiresAt string `json:"expires_at,omitempty"` // RFC3339 } // --- Decision log (learning loop) --- diff --git a/server/server.go b/server/server.go index 48fa532..8d3d833 100644 --- a/server/server.go +++ b/server/server.go @@ -13,6 +13,7 @@ import ( "log" "net" "net/http" + "strconv" "time" "github.com/synapses/synapses-intelligence/pkg/brain" @@ -36,6 +37,14 @@ func New(b brain.Brain, port int) *Server { mux.HandleFunc("POST /v1/explain-violation", s.handleExplainViolation) mux.HandleFunc("POST /v1/coordinate", s.handleCoordinate) + // v0.2.0 endpoints + mux.HandleFunc("POST /v1/context-packet", s.handleContextPacket) + mux.HandleFunc("GET /v1/sdlc", s.handleGetSDLC) + mux.HandleFunc("PUT /v1/sdlc/phase", s.handleSetPhase) + mux.HandleFunc("PUT /v1/sdlc/mode", s.handleSetMode) + mux.HandleFunc("POST /v1/decision", s.handleLogDecision) + mux.HandleFunc("GET /v1/patterns", s.handleGetPatterns) + s.server = &http.Server{ Addr: fmt.Sprintf("127.0.0.1:%d", port), Handler: mux, @@ -167,6 +176,127 @@ func (s *Server) handleCoordinate(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, resp) } +// --- v0.2.0 Handlers --- + +// handleContextPacket assembles a Context Packet for the calling agent. +// POST /v1/context-packet +// Body: brain.ContextPacketRequest (JSON) +// Returns: brain.ContextPacket or 204 if Brain unavailable (caller uses raw context). +func (s *Server) handleContextPacket(w http.ResponseWriter, r *http.Request) { + var req brain.ContextPacketRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid request body: "+err.Error()) + return + } + + pkt, err := s.brain.BuildContextPacket(r.Context(), req) + if err != nil { + writeError(w, http.StatusInternalServerError, err.Error()) + return + } + if pkt == nil { + // Brain is unavailable or feature disabled — caller falls back to raw context. + w.WriteHeader(http.StatusNoContent) + return + } + writeJSON(w, http.StatusOK, pkt) +} + +// handleGetSDLC returns the current SDLC config. +// GET /v1/sdlc +func (s *Server) handleGetSDLC(w http.ResponseWriter, r *http.Request) { + cfg := s.brain.GetSDLCConfig() + writeJSON(w, http.StatusOK, cfg) +} + +// handleSetPhase updates the project SDLC phase. +// PUT /v1/sdlc/phase +// Body: {"phase": "testing", "agent_id": "agent-1"} +func (s *Server) handleSetPhase(w http.ResponseWriter, r *http.Request) { + var body struct { + Phase brain.SDLCPhase `json:"phase"` + AgentID string `json:"agent_id"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + writeError(w, http.StatusBadRequest, "invalid request body: "+err.Error()) + return + } + if body.Phase == "" { + writeError(w, http.StatusBadRequest, "phase is required") + return + } + if err := s.brain.SetSDLCPhase(body.Phase, body.AgentID); err != nil { + writeError(w, http.StatusBadRequest, err.Error()) + return + } + cfg := s.brain.GetSDLCConfig() + writeJSON(w, http.StatusOK, cfg) +} + +// handleSetMode updates the project quality mode. +// PUT /v1/sdlc/mode +// Body: {"mode": "enterprise", "agent_id": "agent-1"} +func (s *Server) handleSetMode(w http.ResponseWriter, r *http.Request) { + var body struct { + Mode brain.QualityMode `json:"mode"` + AgentID string `json:"agent_id"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + writeError(w, http.StatusBadRequest, "invalid request body: "+err.Error()) + return + } + if body.Mode == "" { + writeError(w, http.StatusBadRequest, "mode is required") + return + } + if err := s.brain.SetQualityMode(body.Mode, body.AgentID); err != nil { + writeError(w, http.StatusBadRequest, err.Error()) + return + } + cfg := s.brain.GetSDLCConfig() + writeJSON(w, http.StatusOK, cfg) +} + +// handleLogDecision records an agent decision and updates learning patterns. +// POST /v1/decision +// Body: brain.DecisionRequest (JSON) +func (s *Server) handleLogDecision(w http.ResponseWriter, r *http.Request) { + var req brain.DecisionRequest + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid request body: "+err.Error()) + return + } + if req.EntityName == "" { + writeError(w, http.StatusBadRequest, "entity_name is required") + return + } + if err := s.brain.LogDecision(r.Context(), req); err != nil { + writeError(w, http.StatusInternalServerError, err.Error()) + return + } + writeJSON(w, http.StatusOK, map[string]string{"status": "recorded"}) +} + +// handleGetPatterns returns learned co-occurrence patterns from brain.sqlite. +// GET /v1/patterns +// Query params: +// - trigger (optional): filter to patterns for a specific entity name +// - limit (optional): max results, default 20 +func (s *Server) handleGetPatterns(w http.ResponseWriter, r *http.Request) { + trigger := r.URL.Query().Get("trigger") + limit := 20 + if ls := r.URL.Query().Get("limit"); ls != "" { + if n, err := strconv.Atoi(ls); err == nil && n > 0 { + limit = n + } + } + patterns := s.brain.GetPatterns(trigger, limit) + writeJSON(w, http.StatusOK, map[string]interface{}{ + "patterns": patterns, + "count": len(patterns), + }) +} + // --- Helpers --- func writeJSON(w http.ResponseWriter, status int, v interface{}) { From 432826d55a0aef4a81dd0bdffa6e960c9d23e4f1 Mon Sep 17 00:00:00 2001 From: Divyansh Kumar Date: Fri, 27 Feb 2026 13:18:44 +0000 Subject: [PATCH 2/2] docs: add README and CHANGELOG for v0.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README.md — comprehensive developer and user guide: - Quick start (build → pull model → serve → health check) - Full configuration reference with all fields and defaults - Model tier selection guide (4GB → 16GB+) - Complete CLI reference: serve, status, ingest, summaries, sdlc, decisions, patterns, reset, version - HTTP API reference for all 12 endpoints with request/response examples - Go library usage with Brain interface summary - packet_quality interpretation table - Phase→sections matrix and SDLC workflow docs - Quality mode comparison table - brain.sqlite table inventory with TTL/pruning notes - Developer guide: package conventions and import rules CHANGELOG.md — full version history: - v0.0.1: initial scaffolding - v0.1.0: Ingest, Enrich, Guardian, Orchestrate, HTTP sidecar, CLI - v0.2.0: Context Packet, SDLC phases, quality modes, co-occurrence learning (context_patterns, decision_log, sdlc_config tables) - v0.3.0: insight cache (6h TTL), cache invalidation on re-ingest, PacketQuality, LLMUsed, shared llm/util.go, 6 new HTTP endpoints, 5 new CLI subcommands, Reset() fix, INTELLIGENCE.md Co-Authored-By: Claude Sonnet 4.6 --- CHANGELOG.md | 137 +++++++++ README.md | 827 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 964 insertions(+) create mode 100644 CHANGELOG.md create mode 100644 README.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d40bc53 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,137 @@ +# Changelog + +All notable changes to synapses-intelligence are documented here. +Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). + +--- + +## [0.3.0] — 2026-02-27 + +### Added +- **Insight cache** (`insight_cache` table in brain.sqlite): LLM-generated insights + are now cached per `(node_id, phase)` pair with a 6-hour TTL. Subsequent calls to + `BuildContextPacket` or `Enrich` return the cached result instantly without hitting + Ollama — the slow path only runs on cache miss. +- **Cache invalidation on re-ingest**: `UpsertSummary` now automatically deletes all + `insight_cache` rows for the re-ingested node. Code changes always produce fresh insights. +- **`PacketQuality float64`** field on `ContextPacket` (0.0–1.0 heuristic): + - `0.0` — no summaries ingested yet (empty packet) + - `0.4` — root summary present, no dependencies, no insight + - `0.5` — root + dependency summaries, no LLM insight + - `1.0` — root summary + dep summaries + LLM insight (fully enriched) + Agents can read this field to decide whether to request a follow-up enrichment pass. +- **`LLMUsed bool`** field on `EnrichResponse` and `ContextPacket`: `true` means a live + Ollama call was made; `false` means all data came from brain.sqlite (sub-millisecond). +- **`internal/llm/util.go`**: Shared `ExtractJSON()` and `Truncate()` utilities extracted + from four internal packages that previously duplicated the code. +- **`INTELLIGENCE.md`**: Comprehensive machine-readable reference for LLM agents and + developers. Covers all 9 capabilities, 3 Synapses injection points, integration + challenges, and v0.4.0 improvement proposals. +- **New CLI subcommands**: + - `brain sdlc` — show current SDLC phase and quality mode + - `brain sdlc phase ` — set phase (planning/development/testing/review/deployment) + - `brain sdlc mode ` — set mode (quick/standard/enterprise) + - `brain decisions [entity]` — list recent agent decision log entries + - `brain patterns` — list learned co-occurrence patterns by confidence +- **New HTTP endpoints**: + - `POST /v1/context-packet` — assemble a phase-aware Context Packet + - `GET /v1/sdlc` — get current SDLC config + - `PUT /v1/sdlc/phase` — update SDLC phase + - `PUT /v1/sdlc/mode` — update quality mode + - `POST /v1/decision` — log an agent decision (feeds learning loop) + - `GET /v1/patterns` — list learned co-occurrence patterns + +### Fixed +- `Reset()` now clears all 6 tables including `insight_cache` (previously omitted). +- Store pruning functions (`pruneOldData`) now log errors to stderr instead of silently + discarding them. Violation cache is also pruned (7-day TTL). +- JSON unmarshal errors in `GetSummaryWithTags` and `GetRecentDecisions` are now + logged to stderr. + +--- + +## [0.2.0] — 2026-02-27 + +### Added +- **Context Packet** (`BuildContextPacket`): Phase-aware structured semantic document + replacing raw graph nodes in LLM prompts. Delivers ~90% token savings vs raw nodes. + Capped at ~800 tokens. 7 sections gated by phase matrix. +- **SDLC phase system**: 5 phases — `planning`, `development`, `testing`, `review`, + `deployment`. Each phase controls which Context Packet sections are assembled. + Stored per-project in `brain.sqlite` (`sdlc_config` table). +- **Quality modes**: `quick`, `standard`, `enterprise`. Drive the `QualityGate` + checklist injected into every Context Packet. +- **`internal/sdlc/profiles.go`**: Static phase→section matrix (`SectionsForPhase`), + quality gate profiles (`GateForMode`), and phase guidance strings (`PhaseGuidance`). + Pure data package — no external imports. +- **`internal/sdlc/manager.go`**: `Manager` for getting/setting phase and mode in + brain.sqlite. `ResolvePhase` and `ResolveMode` apply per-request overrides on top + of stored project config. +- **`internal/contextbuilder/builder.go`**: Two-step `Builder.Build()` pipeline: + 1. Fast path (<5ms): SQLite lookups for root summary, dependency summaries, + constraints with fix hints, team status, pattern hints, phase guidance. + 2. Optional LLM path (+2–3s): calls enricher for insight + concerns (only when + `EnableLLM=true` and phase has `LLMInsight` section enabled). +- **`internal/contextbuilder/learner.go`**: `Learner.RecordDecision()` — pure Go + co-occurrence counter, no LLM. Writes symmetric pairs (A→B and B→A) to + `context_patterns` table. Confidence = `co_count / total_count`. +- **Co-occurrence learning tables** (`context_patterns`, `decision_log`): patterns + are pruned after 14 days if co_count < 2; decision log pruned after 30 days. +- **New public types** in `pkg/brain/types.go`: `ContextPacket`, `SDLCPhase`, + `QualityMode`, `ContextPacketRequest`, `DecisionRequest`, `ConstraintItem`, + `AgentStatus`, `QualityGate`, `PatternHint`, `SDLCConfig`. +- **New `Brain` interface methods**: `BuildContextPacket`, `LogDecision`, + `SetSDLCPhase`, `SetQualityMode`, `GetSDLCConfig`, `GetPatterns`. +- **New config fields** in `config.BrainConfig`: `ContextBuilder`, `LearningEnabled`, + `DefaultPhase`, `DefaultMode`. +- **`NullBrain`** updated: all new methods return zero-value safe responses. + +### Changed +- **Ingestor prompt upgraded** to produce `{"summary": "...", "tags": ["tag1", "tag2"]}`. + Tags stored in `semantic_summaries.tags` column (JSON). Enables domain-based pattern + matching. Existing rows default to `[]`. +- **`semantic_summaries` schema** gains `tags TEXT NOT NULL DEFAULT '[]'`. +- **`brain.IngestResponse`** gains `Tags []string`. +- **`brain.EnrichResponse`** gains `Summaries map[string]string` (bulk node ID→summary, + loaded from SQLite, no LLM). + +--- + +## [0.1.0] — 2026-02-27 + +### Added +- **`Brain` interface** (`pkg/brain`): public contract for all Brain capabilities. + `NullBrain` is the fail-silent zero-value implementation. +- **Semantic Ingestor** (`internal/ingestor`): generates a 1-sentence summary for a + code entity via Ollama. Stored in `semantic_summaries` table. Called on file-save. +- **Context Enricher** (`internal/enricher`): generates a 2-sentence architectural + insight + concerns for an entity and its callee/caller neighbourhood. Called during + `get_context`. +- **Rule Guardian** (`internal/guardian`): generates a plain-English explanation and + fix suggestion for an architectural rule violation. Cached in `violation_cache` by + `(rule_id, source_file)`. Called during `get_violations`. +- **Task Orchestrator** (`internal/orchestrator`): suggests work distribution when two + agents conflict on the same scope. Falls back to a deterministic response without + LLM if conflicts are simple. +- **LLM client layer** (`internal/llm`): `LLMClient` interface, Ollama HTTP client + (`POST /api/generate`), and `MockLLMClient` for tests. +- **SQLite store** (`internal/store`): `semantic_summaries` and `violation_cache` + tables. Pure-Go SQLite (`modernc.org/sqlite`), no CGO. +- **HTTP sidecar server** (`server`): REST API on `localhost:11435`. Endpoints: + `GET /v1/health`, `GET /v1/summary/{nodeId}`, `POST /v1/ingest`, + `POST /v1/enrich`, `POST /v1/explain-violation`, `POST /v1/coordinate`. +- **CLI binary** (`cmd/brain`): `brain serve|status|ingest|summaries|reset|version`. +- **`config.BrainConfig`**: JSON config file support with `BRAIN_CONFIG` env override. + All features individually toggleable (`ingest`, `enrich`, `guardian`, `orchestrate`). +- Default model: `qwen2.5-coder:1.5b` (~900MB, works on 4GB RAM). + +--- + +## [0.0.1] — 2026-02-27 + +Initial project scaffolding for synapses-intelligence. + +- Go module `github.com/synapses/synapses-intelligence` +- Repository structure: `cmd/`, `config/`, `internal/`, `pkg/brain/`, `server/` +- `go.mod` with `modernc.org/sqlite` as the only non-stdlib dependency +- `Makefile` with `build`, `test`, `lint`, `tidy`, `serve`, `status`, `pull-model` targets diff --git a/README.md b/README.md new file mode 100644 index 0000000..b668165 --- /dev/null +++ b/README.md @@ -0,0 +1,827 @@ +# synapses-intelligence + +**The Thinking Brain for Synapses** — a local LLM sidecar that adds semantic reasoning, +SDLC phase awareness, and co-occurrence learning to the +[Synapses](https://github.com/synapses/synapses) code-graph MCP server. + +--- + +## What it does + +Raw code-graph nodes give an LLM too much noise and not enough signal. synapses-intelligence +replaces raw nodes with **Context Packets** — structured semantic documents assembled from +brain.sqlite and (optionally) a local Ollama model. A fully-enriched packet delivers the +same information in ~800 tokens that raw nodes would need 4,000+ tokens to express. + +### Capabilities at a glance + +| Capability | Method | LLM? | Latency | +|---|---|---|---| +| Summarise a code entity (1 sentence) | `Ingest` | yes | 1–3 s | +| Context Packet (summaries + constraints + guidance) | `BuildContextPacket` | optional | <5 ms fast path | +| Architectural insight for a neighbourhood | `Enrich` | yes | 1–3 s | +| Explain a rule violation in plain English | `ExplainViolation` | yes (cached) | <1 ms cached | +| Agent conflict work distribution | `Coordinate` | yes | 1–3 s | +| SDLC phase + quality mode management | `SetSDLCPhase` / `SetQualityMode` | no | <1 ms | +| Co-occurrence learning ("also check Y when editing X") | `LogDecision` | no | <1 ms | +| Get learned patterns | `GetPatterns` | no | <1 ms | +| Get summary for a node | `Summary` | no | <1 ms | + +**Fail-silent guarantee**: if Ollama is unreachable, all methods return zero-value safe +responses. The caller always gets something usable. + +--- + +## Prerequisites + +| Requirement | Version | +|---|---| +| Go | 1.22+ | +| [Ollama](https://ollama.com) | any recent | +| Ollama model | see [Model tiers](#model-tiers) | + +No CGO, no external databases, no network dependencies beyond Ollama. + +--- + +## Quick start + +```sh +# 1. Install Ollama and pull the default model +ollama pull qwen2.5-coder:1.5b + +# 2. Build the binary +make build + +# 3. Start the brain sidecar +./bin/brain serve + +# 4. Verify it's running +curl http://localhost:11435/v1/health +# {"status":"ok","model":"qwen2.5-coder:1.5b","available":true} +``` + +--- + +## Installation + +### Build from source + +```sh +git clone https://github.com/synapses/synapses-intelligence +cd synapses-intelligence +make build # produces ./bin/brain +make test # runs all tests +``` + +### Go module (library mode) + +```go +import "github.com/synapses/synapses-intelligence/pkg/brain" + +cfg := config.DefaultConfig() +cfg.Enabled = true +b := brain.New(cfg) +``` + +--- + +## Configuration + +Configuration is a JSON file loaded via the `BRAIN_CONFIG` environment variable. +All fields are optional — sensible defaults apply. + +```json +{ + "enabled": true, + "ollama_url": "http://localhost:11434", + "model": "qwen2.5-coder:1.5b", + "timeout_ms": 3000, + "db_path": "~/.synapses/brain.sqlite", + "port": 11435, + + "ingest": true, + "enrich": true, + "guardian": true, + "orchestrate": true, + "context_builder": true, + "learning_enabled": true, + + "default_phase": "development", + "default_mode": "standard" +} +``` + +```sh +BRAIN_CONFIG=/path/to/brain.json brain serve +``` + +### Configuration reference + +| Field | Default | Description | +|---|---|---| +| `enabled` | `false` | Master switch. Set `true` to activate all features. | +| `ollama_url` | `http://localhost:11434` | Ollama server base URL. | +| `model` | `qwen2.5-coder:1.5b` | Ollama model tag. See [Model tiers](#model-tiers). | +| `timeout_ms` | `3000` | Per-LLM-request timeout in milliseconds. | +| `db_path` | `~/.synapses/brain.sqlite` | SQLite database path (created if missing). | +| `port` | `11435` | HTTP sidecar port. | +| `ingest` | `true` | Enable `POST /v1/ingest` (semantic summaries). | +| `enrich` | `true` | Enable `POST /v1/enrich` (neighbourhood insight). | +| `guardian` | `true` | Enable `POST /v1/explain-violation` (rule explanations). | +| `orchestrate` | `true` | Enable `POST /v1/coordinate` (agent conflict resolution). | +| `context_builder` | `true` | Enable `POST /v1/context-packet` (Context Packets). | +| `learning_enabled` | `true` | Enable `POST /v1/decision` (co-occurrence learning). | +| `default_phase` | `development` | Initial SDLC phase when brain.sqlite is first created. | +| `default_mode` | `standard` | Initial quality mode when brain.sqlite is first created. | + +--- + +## Model tiers + +| System RAM | Model | Size | Notes | +|---|---|---|---| +| 4 GB | `qwen2.5-coder:1.5b` | ~900 MB | Default. Works on any dev machine. | +| 4 GB+ | `qwen3:1.7b` | ~1.1 GB | Recommended upgrade. Better reasoning. | +| 8 GB+ | `qwen3:4b` | ~2.5 GB | Power user. Noticeably better summaries. | +| 16 GB+ | `qwen3:8b` | ~5 GB | Enterprise. Best quality, higher latency. | + +```sh +# Pull a specific model +ollama pull qwen3:1.7b + +# Start brain with a different model +brain serve -model qwen3:1.7b + +# Or set in config +{ "model": "qwen3:1.7b" } +``` + +--- + +## CLI reference + +``` +brain [flags] +``` + +### `serve` + +Start the HTTP sidecar server. + +```sh +brain serve # default port 11435, default model +brain serve -port 11436 # custom port +brain serve -model qwen3:1.7b # custom model +``` + +The server binds to `127.0.0.1` only (not exposed to the network). + +### `status` + +Show Ollama connectivity, model, SQLite stats, feature flags, and SDLC config. + +```sh +brain status +# Ollama: connected (http://localhost:11434) +# Model: qwen2.5-coder:1.5b +# Store: /home/user/.synapses/brain.sqlite +# Summaries: 42 stored +# +# Features: +# ingest enabled +# enrich enabled +# ... +# +# SDLC: +# phase development +# mode standard +``` + +### `ingest` + +Manually trigger ingestion of a code snippet. The JSON body accepts the same fields +as `POST /v1/ingest`. + +```sh +# Inline JSON +brain ingest '{"node_id":"auth:AuthService","node_name":"AuthService","node_type":"struct","package":"auth","code":"type AuthService struct { ... }"}' + +# From stdin +echo '{"node_id":"...","code":"..."}' | brain ingest - +``` + +### `summaries` + +List all stored semantic summaries in a formatted table. + +```sh +brain summaries +``` + +### `sdlc` + +Show or set the project's SDLC phase and quality mode. + +```sh +brain sdlc # show current phase and mode +brain sdlc phase testing # set phase +brain sdlc phase development # back to development +brain sdlc mode enterprise # set quality mode +brain sdlc mode standard # reset to standard +``` + +**Valid phases**: `planning`, `development`, `testing`, `review`, `deployment` + +**Valid modes**: `quick`, `standard`, `enterprise` + +### `decisions` + +List recent agent decision log entries. Used to audit what LLM agents did and +to verify that the learning loop is receiving data. + +```sh +brain decisions # last 20 decisions +brain decisions AuthService # decisions involving AuthService +``` + +### `patterns` + +List learned co-occurrence patterns sorted by confidence descending. +A pattern means: "when an agent edits TRIGGER, also check CO-CHANGE." + +```sh +brain patterns +# TRIGGER CO-CHANGE CONF COUNT REASON +# AuthService TokenValidator 0.87 13 co-edited in same session +# UserRepository SessionStore 0.72 9 ... +``` + +### `reset` + +Clear all brain data from brain.sqlite (prompts for confirmation). + +```sh +brain reset +``` + +### `version` + +```sh +brain version +# synapses-intelligence v0.3.0 +``` + +--- + +## HTTP API reference + +All endpoints accept and return `application/json`. The server binds to +`127.0.0.1:11435` by default. + +--- + +### `GET /v1/health` + +Health check and LLM availability probe. + +**Response** +```json +{ + "status": "ok", + "model": "qwen2.5-coder:1.5b", + "available": true +} +``` + +--- + +### `POST /v1/ingest` + +Generate and store a 1-sentence semantic summary (+ topic tags) for a code entity. +Call this whenever a function, struct, or method is saved. + +**Request** +```json +{ + "node_id": "auth:AuthService", + "node_name": "AuthService", + "node_type": "struct", + "package": "auth", + "code": "type AuthService struct { jwtKey []byte; store *UserStore }" +} +``` + +**Response** +```json +{ + "node_id": "auth:AuthService", + "summary": "Central authentication coordinator that validates JWTs and manages user sessions.", + "tags": ["auth", "session", "jwt"] +} +``` + +**Notes** +- `node_id` and `code` are required. `node_type` and `package` are optional but improve summary quality. +- Re-ingesting the same `node_id` overwrites the old summary and invalidates the insight cache. +- Returns immediately (with no summary) if Ollama is unavailable. + +--- + +### `GET /v1/summary/{nodeId}` + +Fetch the stored summary for a single node. Fast (SQLite, no LLM). + +```sh +curl http://localhost:11435/v1/summary/auth:AuthService +# {"summary": "Central authentication coordinator..."} +``` + +Returns `404` if the node has not been ingested. + +--- + +### `POST /v1/enrich` + +Generate a 2-sentence architectural insight + concerns for an entity and its +neighbourhood. Results cached by `(node_id, phase)` for 6 hours. + +**Request** +```json +{ + "root_id": "auth:AuthService", + "root_name": "AuthService", + "root_type": "struct", + "callee_names": ["TokenValidator", "RateLimiter"], + "caller_names": ["LoginHandler"], + "task_context": "adding refresh token support" +} +``` + +**Response** +```json +{ + "insight": "AuthService orchestrates JWT validation and session lifecycle; it is the sole entry point for all authentication state changes.", + "concerns": ["Handles sensitive JWT signing keys — verify no plaintext leaks in logs"], + "summaries": { + "auth:AuthService": "Central auth coordinator...", + "auth:TokenValidator": "Parses and verifies JWT signatures..." + }, + "llm_used": true +} +``` + +--- + +### `POST /v1/explain-violation` + +Get a plain-English explanation and fix suggestion for an architectural rule violation. +Results cached by `(rule_id, source_file)` indefinitely (pruned after 7 days). + +**Request** +```json +{ + "rule_id": "no-db-in-handler", + "rule_severity": "error", + "description": "Handler files must not import db packages directly", + "source_file": "internal/handlers/user.go", + "target_name": "db.QueryRow" +} +``` + +**Response** +```json +{ + "explanation": "Handler 'user.go' imports a database function directly, bypassing the repository layer. This couples HTTP routing to storage details and makes the code hard to test.", + "fix": "Move db.QueryRow into a UserRepository method and inject it into the handler via the constructor." +} +``` + +--- + +### `POST /v1/coordinate` + +Suggest work distribution when two agents conflict on the same scope. + +**Request** +```json +{ + "new_agent_id": "claude-backend-2", + "new_scope": "internal/auth/", + "conflicting_claims": [ + {"agent_id": "claude-backend-1", "scope": "internal/auth/", "scope_type": "directory"} + ] +} +``` + +**Response** +```json +{ + "suggestion": "Agent 'claude-backend-1' already owns internal/auth/. Consider working on the tests (internal/auth/*_test.go) or a related package like internal/session/.", + "alternative_scope": "internal/session/" +} +``` + +--- + +### `POST /v1/context-packet` + +Assemble a phase-aware Context Packet for an agent. This is the primary endpoint — +it replaces raw graph nodes with a compact, structured semantic document. + +Returns `204 No Content` when `context_builder` is disabled or Brain is unavailable +(the caller should fall back to raw context). + +**Request** +```json +{ + "agent_id": "claude-backend-1", + "phase": "development", + "quality_mode": "standard", + "enable_llm": false, + "snapshot": { + "root_node_id": "auth:AuthService", + "root_name": "AuthService", + "root_type": "struct", + "root_file": "internal/auth/service.go", + "callee_names": ["TokenValidator", "RateLimiter", "UserRepository"], + "caller_names": ["LoginHandler", "RefreshHandler"], + "related_names": ["SessionStore"], + "applicable_rules": [ + {"rule_id": "no-db-in-handler", "severity": "error", "description": "..."} + ], + "active_claims": [ + {"agent_id": "claude-backend-2", "scope": "internal/session/", "scope_type": "directory", "expires_at": "2026-02-27T15:00:00Z"} + ], + "task_context": "adding refresh token support", + "task_id": "task-42" + } +} +``` + +**Response** +```json +{ + "agent_id": "claude-backend-1", + "entity_name": "AuthService", + "entity_type": "struct", + "generated_at": "2026-02-27T12:00:00Z", + "phase": "development", + "quality_mode": "standard", + "root_summary": "Central auth coordinator that validates JWTs and manages sessions.", + "dependency_summaries": { + "TokenValidator": "Parses and verifies JWT signatures against the signing key.", + "RateLimiter": "Enforces per-user sliding window request limits.", + "UserRepository": "Fetches user records from PostgreSQL by ID or email." + }, + "insight": "", + "concerns": [], + "llm_used": false, + "packet_quality": 0.5, + "active_constraints": [ + { + "rule_id": "no-db-in-handler", + "severity": "error", + "description": "Handler files must not import db packages directly", + "hint": "Move db.QueryRow into a UserRepository method" + } + ], + "team_status": [ + {"agent_id": "claude-backend-2", "scope": "internal/session/", "scope_type": "directory", "expires_in": 3600} + ], + "quality_gate": { + "require_tests": true, + "require_docs": false, + "require_pr_check": false, + "checklist": [ + "Write unit tests for new/modified functions", + "Run validate_plan — no rule violations", + "Exported symbols should have doc comments" + ] + }, + "pattern_hints": [ + {"trigger": "AuthService", "co_change": "TokenValidator", "reason": "co-edited in same session", "confidence": 0.87} + ], + "phase_guidance": "You are in development phase. Claim scope via claim_work before editing. Respect all active constraints. Run validate_plan before major changes.", + "packet_quality": 0.5 +} +``` + +**`packet_quality` interpretation** + +| Value | Meaning | +|---|---| +| `0.0` | No summaries ingested yet. Packet carries only static data (constraints, guidance). | +| `0.4` | Root summary present, no dependency summaries. | +| `0.5` | Root + at least one dependency summary. No LLM insight. | +| `1.0` | Full packet: root summary + dependencies + LLM insight (cached or live). | + +**Phase → sections matrix** + +| Section | planning | development | testing | review | deployment | +|---|---|---|---|---|---| +| Root summary | ✓ | ✓ | ✓ | ✓ | ✓ | +| Dep summaries | ✓ | ✓ | — | ✓ | — | +| LLM insight | ✓ | ✓ | — | ✓ | — | +| Constraints | — | ✓ | ✓ | ✓ | — | +| Team status | ✓ | ✓ | ✓ | ✓ | ✓ | +| Quality gate | — | ✓ | ✓ | ✓ | — | +| Pattern hints | — | ✓ | — | ✓ | — | +| Phase guidance | ✓ | ✓ | ✓ | ✓ | ✓ | + +--- + +### `GET /v1/sdlc` + +Get the current SDLC config. + +**Response** +```json +{ + "phase": "development", + "quality_mode": "standard", + "updated_at": "2026-02-27T12:00:00Z", + "updated_by": "claude-backend-1" +} +``` + +--- + +### `PUT /v1/sdlc/phase` + +Set the project SDLC phase. Returns the updated config. + +**Request** +```json +{"phase": "testing", "agent_id": "claude-pm"} +``` + +**Valid values**: `planning`, `development`, `testing`, `review`, `deployment` + +--- + +### `PUT /v1/sdlc/mode` + +Set the project quality mode. Returns the updated config. + +**Request** +```json +{"mode": "enterprise", "agent_id": "claude-pm"} +``` + +**Valid values**: `quick`, `standard`, `enterprise` + +--- + +### `POST /v1/decision` + +Log a completed agent action. Feeds the co-occurrence learning loop. +Call this after every significant change (file edit, test run, fix). + +**Request** +```json +{ + "agent_id": "claude-backend-1", + "phase": "development", + "entity_name": "AuthService", + "action": "edit", + "related_entities": ["TokenValidator", "UserRepository"], + "outcome": "success", + "notes": "Added refresh token support" +} +``` + +**Valid actions**: `edit`, `test`, `review`, `fix_violation` + +**Valid outcomes**: `success`, `violation`, `reverted` + +**Response** +```json +{"status": "recorded"} +``` + +Learning effect: After this call, `context_patterns` gains (or strengthens) two +bidirectional pairs: `AuthService ↔ TokenValidator` and `AuthService ↔ UserRepository`. +Future Context Packets for any of these entities will include the others in `pattern_hints`. + +--- + +### `GET /v1/patterns` + +Get learned co-occurrence patterns. Useful for debugging and auditing. + +```sh +curl "http://localhost:11435/v1/patterns?trigger=AuthService&limit=5" +``` + +**Query parameters** + +| Param | Default | Description | +|---|---|---| +| `trigger` | (all) | Filter to patterns for a specific entity name. | +| `limit` | `20` | Maximum number of patterns to return. | + +**Response** +```json +{ + "count": 2, + "patterns": [ + {"trigger": "AuthService", "co_change": "TokenValidator", "reason": "co-edited in same session", "confidence": 0.87}, + {"trigger": "AuthService", "co_change": "UserRepository", "reason": "", "confidence": 0.72} + ] +} +``` + +--- + +## Go API (library mode) + +Import `pkg/brain` to embed the Brain directly without running an HTTP server. + +```go +import ( + "context" + "github.com/synapses/synapses-intelligence/config" + "github.com/synapses/synapses-intelligence/pkg/brain" +) + +// Build the Brain from config. +cfg := config.DefaultConfig() +cfg.Enabled = true +cfg.Model = "qwen3:1.7b" +b := brain.New(cfg) + +// Ingest a code entity. +resp, err := b.Ingest(ctx, brain.IngestRequest{ + NodeID: "auth:AuthService", + NodeName: "AuthService", + NodeType: "struct", + Package: "auth", + Code: `type AuthService struct { jwtKey []byte }`, +}) +// resp.Summary = "Central auth coordinator..." +// resp.Tags = ["auth", "jwt"] + +// Build a Context Packet (fast path — no LLM call). +pkt, err := b.BuildContextPacket(ctx, brain.ContextPacketRequest{ + AgentID: "my-agent", + Snapshot: brain.SynapsesSnapshotInput{ + RootNodeID: "auth:AuthService", + RootName: "AuthService", + RootType: "struct", + CalleeNames: []string{"TokenValidator"}, + }, + EnableLLM: false, +}) +// pkt is nil when Brain is unavailable — fall back to raw context. +if pkt != nil { + fmt.Println(pkt.RootSummary) + fmt.Printf("Quality: %.1f\n", pkt.PacketQuality) +} + +// Log a decision to feed learning. +_ = b.LogDecision(ctx, brain.DecisionRequest{ + AgentID: "my-agent", + Phase: "development", + EntityName: "AuthService", + Action: "edit", + RelatedEntities: []string{"TokenValidator"}, + Outcome: "success", +}) +``` + +### `Brain` interface + +```go +type Brain interface { + // Semantic summaries + Ingest(ctx, IngestRequest) (IngestResponse, error) + Enrich(ctx, EnrichRequest) (EnrichResponse, error) + Summary(nodeID string) string + + // Architectural analysis + ExplainViolation(ctx, ViolationRequest) (ViolationResponse, error) + Coordinate(ctx, CoordinateRequest) (CoordinateResponse, error) + + // Context Packet + BuildContextPacket(ctx, ContextPacketRequest) (*ContextPacket, error) + + // Learning loop + LogDecision(ctx, DecisionRequest) error + GetPatterns(trigger string, limit int) []PatternHint + + // SDLC + SetSDLCPhase(phase SDLCPhase, agentID string) error + SetQualityMode(mode QualityMode, agentID string) error + GetSDLCConfig() SDLCConfig + + // Diagnostics + Available() bool + ModelName() string +} +``` + +Use `brain.New(cfg)` for production and `&brain.NullBrain{}` for tests or when the +Brain is disabled. `NullBrain` satisfies the interface with all zero-value returns — +no panics, no errors. + +--- + +## Integration with Synapses + +synapses-intelligence is designed to run as a sidecar next to a Synapses MCP server. +See [INTELLIGENCE.md](INTELLIGENCE.md) for the complete integration guide including: + +- The three Synapses injection points (file indexer, get_context, get_violations) +- The `BrainClient` interface Synapses must implement +- Proposed `synapses.json` config additions +- Known integration challenges and how to solve them +- v0.4.0 improvement roadmap + +**Three-second summary of integration**: +1. Synapses starts the brain sidecar (or connects to it at `localhost:11435`). +2. On every file index event, Synapses calls `POST /v1/ingest` for each changed entity + (fire-and-forget goroutine, non-blocking). +3. On `get_context`, Synapses calls `POST /v1/context-packet` with the graph snapshot. + If the packet is non-nil, it is prepended to the response. If nil (Brain unavailable), + the raw context is returned unchanged. +4. On `get_violations`, Synapses calls `POST /v1/explain-violation` for each violation + and attaches the explanation + fix hint. + +--- + +## SDLC workflow + +``` +planning → development → testing → review → deployment → planning +``` + +Set the phase at the start of each stage: + +```sh +brain sdlc phase development # start writing code +brain sdlc phase testing # switch to test mode — constraints enabled, no LLM insight +brain sdlc phase review # full review mode — all sections, enterprise checklist +brain sdlc phase deployment # freeze code — only team status and guidance shown +``` + +The phase is shared across all agents working on the project (stored in brain.sqlite). +Agents receive `phase_guidance` in every Context Packet telling them what to do. + +--- + +## Quality modes + +| Mode | Tests required | Docs required | PR checklist | Use case | +|---|---|---|---|---| +| `quick` | No | No | No | Prototypes, hotfixes | +| `standard` (default) | Unit | No | No | Normal development | +| `enterprise` | Unit + integration | Full GoDoc | Yes + CHANGELOG | Production, open source | + +```sh +brain sdlc mode enterprise # enable full quality gate +``` + +--- + +## Data stored in brain.sqlite + +| Table | Contents | TTL / Pruning | +|---|---|---| +| `semantic_summaries` | 1-sentence summary + tags per node | Never pruned (re-ingest overwrites) | +| `violation_cache` | Rule violation explanations per (rule_id, file) | 7 days | +| `insight_cache` | LLM-generated insights per (node_id, phase) | 6 hours; also invalidated on re-ingest | +| `sdlc_config` | Current project phase + quality mode (single row) | Never pruned | +| `context_patterns` | Co-occurrence pairs (trigger ↔ co_change) | 14 days if co_count < 2 | +| `decision_log` | Agent decision history | 30 days | + +Reset everything with `brain reset` or `DELETE` all rows via the `/v1/reset` endpoint. + +--- + +## Development + +```sh +make build # build ./bin/brain +make test # run all tests with -v +make test-short # run tests in short mode (skips slow paths) +make lint # go vet ./... +make tidy # go mod tidy +make bench # run benchmarks in internal packages +``` + +### Adding a new feature + +1. New internal logic goes in `internal//`. +2. Public types go in `pkg/brain/types.go`. +3. Wire the new method in `pkg/brain/brain.go` and add a no-op to `pkg/brain/null.go`. +4. Add the HTTP handler in `server/server.go`. +5. Add a CLI command in `cmd/brain/main.go`. +6. **Import rule**: `internal/*` packages must NEVER import `pkg/brain`. Use local types + and let `pkg/brain/brain.go` do the conversion. + +--- + +## License + +See [LICENSE](LICENSE) in the repository root.