Integrate hybrid retrieval into the agent SDK and MCP service

[voidrot]

Summary

Integrate the chosen hybrid retrieval path into the existing retrieval SDK and MCP boundary without breaking the markdown source-of-truth model or the deterministic fallback path.

Why this work is needed

The backend choice and index pipeline are not enough by themselves; the retrieval API surface needs to consume those derived indexes and still preserve scope enforcement and fallback behavior.

Scope

• Add hybrid candidate-provider or fusion behavior to the retrieval SDK.
• Integrate the resulting retrieval path into the MCP service.
• Preserve deterministic fallback to the current lexical path.
• Ensure visibility enforcement happens before results leave the boundary.
• Add tests for fallback and scoped retrieval behavior.

Out of scope

• Frontend search UX.
• Replacing markdown as the source of truth.

Acceptance criteria

• The SDK can use the new hybrid retrieval path.
• MCP can serve hybrid retrieval safely.
• Fallback and security behavior are explicit and test-covered.

Dependencies

• Depends on Epic: Hybrid retrieval implementation over reconstructable indexes #60.
• Depends on Add embedding pipeline and rebuildable secondary indexes #70.
• Depends on Replace MCP static bearer auth with Better Auth-backed validation #71.