Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions guides/learn/function-calling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,7 @@ llm.register_function(
"get_current_weather",
fetch_weather_from_api,
cancel_on_interruption=True, # Cancel if user interrupts (default: True)
timeout_secs=30.0, # Optional: Override global timeout for this function
)
```

Expand All @@ -311,6 +312,7 @@ async def get_current_weather(params: FunctionCallParams, location: str, format:
llm.register_direct_function(
get_current_weather,
cancel_on_interruption=False, # Don't cancel on interruption
timeout_secs=60.0, # Optional: Override global timeout for this function
)
```

Expand All @@ -320,9 +322,12 @@ llm.register_direct_function(

- **`cancel_on_interruption=True`** (default): Function call is cancelled if user interrupts
- **`cancel_on_interruption=False`**: Function call continues even if user interrupts
- **`timeout_secs=None`** (default): Optional per-tool timeout in seconds. Overrides the global `function_call_timeout_secs` for this specific function

Use `cancel_on_interruption=False` for critical operations that should complete even if the user starts speaking. Function calls are async, so you can continue the conversation while the function executes. Once the result returns, the LLM will automatically incorporate it into the conversation context. LLMs vary in terms of how well they incorporate changes to _previous_ messages, so you may need to experiment with your LLM provider to see how it handles this.

Use `timeout_secs` to set a specific timeout for a function that differs from the global default. For example, you might want a longer timeout for database queries or shorter timeouts for quick lookups.

### 3. Create the Pipeline

Include your LLM service in your pipeline with the registered functions:
Expand Down
4 changes: 4 additions & 0 deletions server/utilities/service-switchers/llm-switcher.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ llm_switcher.register_function(
Whether to cancel this function call when a user interruption occurs.
</ParamField>

<ParamField path="timeout_secs" type="Optional[float]" default="None">
Optional per-tool timeout in seconds. Overrides the global `function_call_timeout_secs` for this specific function.
</ParamField>

## Usage Examples

### Basic LLM Switching
Expand Down