Problem
Currently, deploying Arranger requires one service instance per data table or index. Managing multiple indices, a common scenario, means duplicated configuration, compounded infrastructure overhead, and unnecessary resource consumption, as each Arranger instance reserves CPU and RAM regardless of active query load.
A search API service should be capable of handling multiple indices, reducing operational burden and resource costs while maintaining the same functionality.
Proposed Solution
A search API service should be created to support multi-index configuration within a single service instance. Rather than deploying one container per index, a single search API service would register and route requests across multiple indices, each with their own configuration.
Docker Compose Example
Instead of four separate services each with a mounted config directory, a single service would accept an array of index configurations, either via a single config file listing all indices, or via indexed environment variables (similar to how the setup service already handles ES_INDEX_0_*, ES_INDEX_1_*, etc.):
search-api:
image: ghcr.io/overture-stack/search-api:x.x.x
container_name: search-api
ports:
- "5050:5050"
volumes:
- ./apps/setup/configs/arrangerConfigs:/app/modules/server/configs
environment:
ES_HOST: http://elasticsearch:9200
ES_USER: elastic
ES_PASS: myelasticpassword
We will do this with an express server that maps configs within the target directory 1:1 to arranger services. Once deployed the search api service would expose a single endpoint and route each query to the appropriate index based on the index name included in the request.
Problem
Currently, deploying Arranger requires one service instance per data table or index. Managing multiple indices, a common scenario, means duplicated configuration, compounded infrastructure overhead, and unnecessary resource consumption, as each Arranger instance reserves CPU and RAM regardless of active query load.
A search API service should be capable of handling multiple indices, reducing operational burden and resource costs while maintaining the same functionality.
Proposed Solution
A search API service should be created to support multi-index configuration within a single service instance. Rather than deploying one container per index, a single search API service would register and route requests across multiple indices, each with their own configuration.
Docker Compose Example
Instead of four separate services each with a mounted config directory, a single service would accept an array of index configurations, either via a single config file listing all indices, or via indexed environment variables (similar to how the setup service already handles
ES_INDEX_0_*,ES_INDEX_1_*, etc.):We will do this with an express server that maps configs within the target directory 1:1 to arranger services. Once deployed the search api service would expose a single endpoint and route each query to the appropriate index based on the index name included in the request.