docs(simulator): updated tool_simulator docs

[poshinchen]

Description

Added toolSimulator related docs

Related Issues

N/A

Type of Change

• New content
• Content update/revision

Checklist

• I have read the CONTRIBUTING document
• My changes follow the project's documentation style
• I have tested the documentation locally using npm run dev
• Links in the documentation are valid and working

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Documentation Preview Ready

Your documentation preview has been successfully deployed!

Preview URL: https://d3ehv1nix5p99z.cloudfront.net/pr-cms-752/docs/user-guide/quickstart/overview/

Updated at: 2026-04-13T17:02:43.528Z

[github-actions]

Assessment: Comment

Good addition to the evals SDK documentation. The ToolSimulator docs are well-structured, follow the established patterns from the user_simulation.mdx page, and provide clear code examples with progressive complexity (basic → shared state → experiment integration → troubleshooting).

Review Details

• Accuracy: The word "deterministic" appears in both files when describing LLM-powered simulation — since responses are LLM-generated, this is misleading and should be softened to "controllable" (see inline comments).
• Code Examples: The "Integration with Experiments" example is missing from pydantic import BaseModel, Field imports (see inline comment on tool_simulation.mdx L187). Other code examples are complete and well-structured.
• Structure & Consistency: The page follows the same section pattern as user_simulation.mdx (Overview → Key Features → Basic Usage → Integration → Best Practices → Troubleshooting → Related Docs). The comparison table in index.mdx is a nice touch.
• Links: All internal links (index.md, user_simulation.md, quickstart.md, goal_success_rate_evaluator.md) reference files that exist in the repo.

Nice comprehensive documentation with the shared state and troubleshooting sections — those will save users a lot of debugging time.

[ybdarrenwang]

There are a couple of useful tips for tool simulator that we're about to share in an upcoming post:

1. Running independent simulator instances

You can create multiple ToolSimulator instances side by side. Each instance maintains its own tool registry and state, so you can run parallel experiment configurations in the same codebase:

simulator_a = ToolSimulator()
simulator_b = ToolSimulator()
# Each instance has an independent tool registry and state --
# ideal for comparing agent behavior across different tool setups.

2. Seeding state from real data

Because initial_state_description accepts natural language, you can get creative with how you seed context. For tools that interact with tabular data, use a DataFrame.describe() call to generate statistical summaries and pass those statistics directly as the state description. ToolSimulator will generate responses that reflect realistic data distributions, without ever accessing the actual data.

Might worth adding them here as well?