feat: Add SearchableToolset for dynamic tool discovery from large tool catalogs

[vblagoje]

Why

Large tool catalogs overwhelm LLM context windows. Agents need a way to discover tools on-demand rather than receiving all tool definitions upfront.

• fixes Add Tool Search Tool #10323

What

• SearchableToolset: Toolset subclass with BM25-based tool discovery
• Uses Haystack's built-in InMemoryDocumentStore.bm25_retrieval() internally for BM25L search — no custom search engine, no extra dependencies
• search_tools(query, k) bootstrap tool for LLM-driven discovery
• Passthrough mode for catalogs below search_threshold (default: 8)
• clear() method for resetting discovered tools between agent runs
• Full serialization support (to_dict/from_dict)

How can it be used

from haystack.tools import SearchableToolset                                                                                                                                                                 
from haystack.components.agents import Agent                                                                                                                                                                 
                                                                                                                                                                                                             
# Large catalog - LLM discovers via search_tools 
tools = <any type of tool or toolset we support as long as it is ToolsType>                                                                                                                                                           
toolset = SearchableToolset(catalog=tools)                                                                                                                                      
agent = Agent(chat_generator=generator, tools=toolset)                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                        

How did you test it

• Unit tests for SearchableToolset (passthrough, discovery, iteration, serialization)
• Integration with ComponentTool warm-up verification

Notes for the reviewer

• BM25L search is delegated to InMemoryDocumentStore.bm25_retrieval(), reusing well-tested Haystack infrastructure instead of a hand-rolled engine
• Embedding mode deferred to future PR
• Tools warmed up lazily on discovery, not at index time

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Feb 25, 2026 10:14am

[coveralls]

Pull Request Test Coverage Report for Build 22308342425

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.02%) to 92.67%

---

Files with Coverage Reduction	New Missed Lines	%
core/pipeline/async_pipeline.py	2	66.67%

Totals
Change from base Build 22302678937:	-0.02%
Covered Lines:	15437
Relevant Lines:	16658

---

💛 - Coveralls

[vblagoje]

@sjrl @julian-risch - give me a 1-2 days to test it thoroughly with large mcp toolsets and if all good I'll open this PR. This is the general direction @mpangrazzi and I talked about. LMK if you agree.

[anakin87]

Cool implementation!

I left some comments and there's still some work to be done, but it seems a good direction.

[sjrl]

Looking very cool! One additional question I had is that this wouldn't work with MCPToolset as it is right? So after this PR we'd need to add support for an SearchableMCPToolset as well in our MCP integration?

[vblagoje]

Looking very cool! One additional question I had is that this wouldn't work with MCPToolset as it is right? So after this PR we'd need to add support for an SearchableMCPToolset as well in our MCP integration?

@sjrl @anakin87 I thought it would. It warms up tools so perhaps it should work. Still didn't get around to test but I will today or tomorrow. Will report back!

[vblagoje]

@sjrl @anakin87 I tried it with itinerary agent and it doesn't work that well. Investigating...

[vblagoje]

These were good improvements, one last thing I want to do is to make sure is that SearchableToolset is robust and battle tested - i.e. that it works even if tools are loaded lazily i.e in MCPToolset and potentially others. It works great now in RL demo with itinerary agent when MCPToolset is eager but not lazy. A few more days and I'll open this PR @sjrl @anakin87

[sjrl]

We say this in the docstrings which sounds nice, but I don't think our Agent is set up to call this method. If we think it should do that perhaps we should open up a follow-up issue to update Agent to utilize this method? (Not entirely sure what that would look like tbh).

[anakin87]

#10671

[sjrl]

Still feel a little odd highlighting this since I don't know how I'd use clear in practice, nor is it highlighted in the code example

[anakin87]

I can remove the mention from the release note.

Just to share an example

from haystack.tools import create_tool_from_function, SearchableToolset
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

def get_weather(city: str) -> str:
    """Get current weather for a city."""
    return f"Weather in {city}: 22°C, sunny"


def add_numbers(a: int, b: int) -> int:
    """Add two numbers together."""
    return a + b


def multiply_numbers(a: int, b: int) -> int:
    """Multiply two numbers."""
    return a * b


def get_stock_price(symbol: str) -> str:
    """Get stock price by ticker symbol."""
    return f"{symbol}: $150.00"


def search_database(query: str) -> str:
    """Search the database for records."""
    return f"Found 5 records matching '{query}'"


def send_email(to: str, subject: str, body: str) -> str:
    """Send an email to a recipient."""
    return f"Email sent to {to}"


def calculate_tax(amount: float, rate: float) -> float:
    """Calculate tax on an amount."""
    return amount * rate


def convert_currency(amount: float, from_currency: str, to_currency: str) -> float:
    """Convert currency from one to another."""
    return amount * 1.1  # Simplified conversion


# Test fixtures
weather_tool =  create_tool_from_function(get_weather)
add_tool = create_tool_from_function(add_numbers)
multiply_tool = create_tool_from_function(multiply_numbers)
stock_tool = create_tool_from_function(get_stock_price)
search_tool = create_tool_from_function(search_database)
send_email_tool = create_tool_from_function(send_email)
calculate_tax_tool = create_tool_from_function(calculate_tax)
convert_currency_tool = create_tool_from_function(convert_currency)
large_catalog = [weather_tool, add_tool, multiply_tool, stock_tool, search_tool, send_email_tool, calculate_tax_tool, convert_currency_tool]


searchable_toolset = SearchableToolset(catalog=large_catalog)
agent = Agent(tools=searchable_toolset, chat_generator=OpenAIChatGenerator(model="gpt-4.1-mini"))

result= agent.run(messages=[ChatMessage.from_user("What's the weather in Milan?")])
print(result["messages"])
# ...

print(len(searchable_toolset))
# 2
print(searchable_toolset._discovered_tools)
# {'get_weather': Tool(name='get_weather', description='Get current weather for a city.', parameters={'properties': {'city': {'type': 'string'}}, 'required': ['city'], 'type': 'object'}, function=<function get_weather at 0x1017a6700>, outputs_to_string=None, inputs_from_state=None, outputs_to_state=None)}

searchable_toolset.clear()

print(len(searchable_toolset))
# 1
print(searchable_toolset._discovered_tools)
# {}


result= agent.run(messages=[ChatMessage.from_user("How many records in the database for query: 'apple'. Use the appropriate tool to search the database.")])
print(result["messages"])
# ...

print(len(searchable_toolset))
# 2
print(searchable_toolset._discovered_tools)
# {'search_database': Tool(name='search_database', description='Search the database for records.', parameters={'properties': {'query': {'type': 'string'}}, 'required': ['query'], 'type': 'object'}, function=<function search_database at 0x1189276a0>, outputs_to_string=None, inputs_from_state=None, outputs_to_state=None)}

[sjrl]

Thanks for the example and opening the issue! The issue should help us figure out how this can be used in a production app rather than just in a script.

[sjrl]

Looks good! Just two minor comments about how a user is practically meant to use the clear method. But probably something we can do in a follow up PR