Add natural language query decomposition for search and graph traversal

[ravisuhag]

Context

AI agents and end users increasingly interact with metadata through natural language. Compass should accept free-form questions and route them to the right combination of semantic search and graph traversal.

Scope

• Accept natural language queries through the search API
• Classify query intent: discovery ("find tables about X"), traversal ("what depends on X"), lookup ("who owns X"), or diagnostic ("what changed in X recently")
• Decompose into structured operations: semantic search, graph traversal, entity lookup, or a combination
• Return unified results regardless of which operations executed

Examples

• "What tables does the revenue dashboard depend on?" → lineage traversal
• "Find something related to customer lifetime value" → semantic search
• "Who owns the payments pipeline?" → ownership lookup
• "What changed upstream of this table this week?" → traversal + temporal filter

Design Considerations

• Can start with heuristic/pattern-based decomposition before adding LLM-based understanding
• Should work through both REST/gRPC and MCP interfaces

Dependencies

• Hybrid search
• Multi-hop graph traversal API

References

• Compass Roadmap — Natural Language Query