What would you like to be added:
Extend and persist the index mechanism in Admin to support prefix matching queries and multi-replica deployment scenarios.
The following improvements are proposed (see full design in #1381):
-
Extend Index Query Logic
Refactor the Indexer definition to support more logical operators beyond the current Equals-only matching (e.g., prefix matching). Update upper-layer components (Manager, Console) accordingly to align with the new Indexer interface.
-
Memory Store: Trie-based Index
For users who choose in-memory storage, introduce a Trie (prefix tree) structure to efficiently support prefix matching queries. Leader-Follower architecture does not apply in this mode, so the index remains in-memory.
-
DB Store: Persistent Index
For users who choose database storage, where Discovery and Engine run under a Leader-Follower architecture (Proposal: support Master-Slave architecture among components #1380), persist indexes into the database by introducing new index tables. Index queries should support combined conditions including prefix matching, ensuring that frontend queries remain effective across all replicas in a multi-replica deployment.
Why is this needed:
Currently, Admin maintains all indexes in memory with a simple map[idx_value][resource_key] structure, and only supports Equals matching. This has two limitations:
- Limited query capability: Prefix matching and other query patterns are not supported, restricting the expressiveness of search and query operations from the frontend.
- Multi-replica incompatibility: When Admin is deployed with multiple replicas, only one replica maintains the in-memory index (due to Leader-Follower in Proposal: support Master-Slave architecture among components #1380). This means frontend queries routed to follower replicas cannot access the index, causing inconsistent query results across replicas.
Persisting indexes to the database and extending the query logic addresses both issues, making Admin's index mechanism more robust, expressive, and production-ready for distributed deployments.
Proposal: #1381
What would you like to be added:
Extend and persist the index mechanism in Admin to support prefix matching queries and multi-replica deployment scenarios.
The following improvements are proposed (see full design in #1381):
Extend Index Query Logic
Refactor the
Indexerdefinition to support more logical operators beyond the currentEquals-only matching (e.g., prefix matching). Update upper-layer components (Manager,Console) accordingly to align with the newIndexerinterface.Memory Store: Trie-based Index
For users who choose in-memory storage, introduce a Trie (prefix tree) structure to efficiently support prefix matching queries. Leader-Follower architecture does not apply in this mode, so the index remains in-memory.
DB Store: Persistent Index
For users who choose database storage, where
DiscoveryandEnginerun under a Leader-Follower architecture (Proposal: support Master-Slave architecture among components #1380), persist indexes into the database by introducing new index tables. Index queries should support combined conditions including prefix matching, ensuring that frontend queries remain effective across all replicas in a multi-replica deployment.Why is this needed:
Currently, Admin maintains all indexes in memory with a simple
map[idx_value][resource_key]structure, and only supportsEqualsmatching. This has two limitations:Persisting indexes to the database and extending the query logic addresses both issues, making Admin's index mechanism more robust, expressive, and production-ready for distributed deployments.
Proposal: #1381