Skip to content

test(langchain): Add tool execution test#5739

Open
alexander-alderman-webb wants to merge 11 commits intowebb/langchain/add-basic-testfrom
webb/langchain/tool-test
Open

test(langchain): Add tool execution test#5739
alexander-alderman-webb wants to merge 11 commits intowebb/langchain/add-basic-testfrom
webb/langchain/tool-test

Conversation

@alexander-alderman-webb
Copy link
Contributor

@alexander-alderman-webb alexander-alderman-webb commented Mar 23, 2026

Description

Add a test that uses langchain functionality introduced in v1.0 of the library.

Re-use Responses API response with a tool call request from openai-agents test.

Issues

Reminders

@github-actions
Copy link
Contributor

github-actions bot commented Mar 23, 2026

Semver Impact of This PR

🟢 Patch (bug fixes)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

  • (asgi) Add option to disable suppressing chained exceptions by alexander-alderman-webb in #5714
  • (logging) Separate ignore lists for events/breadcrumbs and sentry logs by sl0thentr0py in #5698

Bug Fixes 🐛

Anthropic

  • Set exception info on streaming span when applicable by alexander-alderman-webb in #5683
  • Patch AsyncStream.close() and AsyncMessageStream.close() to finish spans by alexander-alderman-webb in #5675
  • Patch Stream.close() and MessageStream.close() to finish spans by alexander-alderman-webb in #5674

Other

  • (starlette) Catch Jinja2Templates ImportError by alexander-alderman-webb in #5741

Documentation 📚

  • Add note on AI PRs to CONTRIBUTING.md by sentrivana in #5696

Internal Changes 🔧

Langchain

  • Add tool execution test by alexander-alderman-webb in #5739
  • Add basic agent test with Responses call by alexander-alderman-webb in #5726
  • Replace mocks with httpx types by alexander-alderman-webb in #5724
  • Consolidate available tools assertion by alexander-alderman-webb in #5721
  • Consolidate span origin assertion by alexander-alderman-webb in #5723

Other

  • Add -latest alias for each integration test suite by sentrivana in #5706
  • Use date-based branch names for toxgen PRs by sentrivana in #5704
  • 🤖 Update test matrix with new releases (03/19) by github-actions in #5703
  • Add client report tests for span streaming by sentrivana in #5677

Other

  • Update CHANGELOG.md by sentrivana in #5685

🤖 This preview updates automatically when you update the PR.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 23, 2026

Codecov Results 📊

13 passed | Total: 13 | Pass Rate: 100% | Execution Time: 7.97s

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 14381 uncovered lines.


Generated by Codecov Action

@alexander-alderman-webb alexander-alderman-webb marked this pull request as ready for review March 23, 2026 10:00
@alexander-alderman-webb alexander-alderman-webb requested a review from a team as a code owner March 23, 2026 10:00
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Redundant assertion contradicts comment and prior assertion
    • I removed the contradictory comment and the redundant weaker assertion because the existing exact-count assertion already covers this condition.

Create PR

Or push these changes by commenting:

@cursor push f520920c95
Preview (f520920c95)
diff --git a/tests/integrations/langchain/test_langchain.py b/tests/integrations/langchain/test_langchain.py
--- a/tests/integrations/langchain/test_langchain.py
+++ b/tests/integrations/langchain/test_langchain.py
@@ -336,9 +336,6 @@
     assert chat_spans[1]["origin"] == "auto.ai.langchain"
     assert tool_exec_span["origin"] == "auto.ai.langchain"
 
-    # We can't guarantee anything about the "shape" of the langchain execution graph
-    assert len(list(x for x in tx["spans"] if x["op"] == "gen_ai.chat")) > 0
-
     # Token usage is only available in newer versions of langchain (v0.2+)
     # where usage_metadata is supported on AIMessageChunk
     if "gen_ai.usage.input_tokens" in chat_spans[0]["data"]:

This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

Copy link
Member

@ericapisani ericapisani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking questions, LGTM otherwise

"Tool calls should be recorded when send_default_pii=True and include_prompts=True"
)
tool_calls_data = chat_spans[0]["data"][SPANDATA.GEN_AI_RESPONSE_TOOL_CALLS]
assert isinstance(tool_calls_data, (list, str)) # Could be serialized
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a loose assertion like this (where we're checking if it's a list or a str) makes me a bit nervous in the context of a test where we have control over everything, because we should be certain about what value we can expect to get back in the response.

Is the serialization that's mentioned here controlled by the value of send_default_pii or include_prompts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this was copy-pasted from another test.
but 100% agree we shouldn't be spreading a bad pattern. The value is always a string and the test asserts the type now: 09673b7

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants