fix: use async LLM calls in converter for async workflows (issue #5230)#5231
Open
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
Open
fix: use async LLM calls in converter for async workflows (issue #5230)#5231devin-ai-integration[bot] wants to merge 3 commits intomainfrom
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
Conversation
When using akickoff() (async workflow), the _export_output method and _ainvoke_guardrail_function were calling synchronous llm.call() through the converter module, which blocks the event loop. Changes: - Add ato_pydantic() and ato_json() async methods to Converter class that use llm.acall() instead of llm.call() - Add async utility functions: aconvert_to_model(), ahandle_partial_json(), aconvert_with_instructions() - Add _aexport_output() async method to Task class - Wire _aexport_output into _aexecute_core and _ainvoke_guardrail_function replacing the sync _export_output calls in async paths - Add comprehensive async tests covering all new async converter paths Fixes #5230 Co-Authored-By: João <joao@crewai.com>
Contributor
Author
|
Prompt hidden (unlisted session) |
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Co-Authored-By: João <joao@crewai.com>
Addresses Cursor Bugbot feedback: the function-calling path in ato_json was still calling _create_instructor().to_json() synchronously, blocking the event loop. Now wrapped in asyncio.to_thread() to offload the blocking call to a separate thread. Co-Authored-By: João <joao@crewai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
When using
akickoff()(async workflow), the converter module was called synchronously viaself._export_output()inside_aexecute_coreand_ainvoke_guardrail_function, which callsllm.call()and blocks the event loop.This PR adds async counterparts throughout the converter pipeline:
Converter.ato_pydantic()/Converter.ato_json()— usellm.acall()instead ofllm.call()aconvert_to_model(),ahandle_partial_json(),aconvert_with_instructions()— async versions of the module-level utility functionsTask._aexport_output()— async version of_export_outputwired into_aexecute_core(line 605) and both branches of_ainvoke_guardrail_function(lines 1286, 1331)Sync methods are untouched; backward compatibility is preserved.
Review & Testing Checklist for Human
ato_jsonstill calls sync_create_instructor().to_json()whensupports_function_calling()is True — this means the function-calling JSON path still blocks the event loop. The instructor library doesn't expose an async interface here. Verify this is acceptable or needs arun_in_executorwrapper._aexport_outputcall sites intask.pyare correct (lines ~605, ~1286, ~1331). Confirm no other async paths still call the sync_export_output.ato_pydanticnon-function-calling fallback calls synchandle_partial_jsononValidationError— this is safe (no LLM call, just regex/JSON parsing) but worth confirming.output_pydanticoroutput_jsonset on a task, usingcrew.akickoff(), and confirm the event loop is no longer blocked during output conversion.Notes
ato_pydantic/ato_jsonhappy paths, retry logic, max-attempt errors, all three async utility functions, partial JSON handling, and explicit assertions thatllm.call()is never invoked from the async path.Link to Devin session: https://app.devin.ai/sessions/fec9a6f7345348bd99ecbc42d20e7876
Note
Medium Risk
Updates async task execution to use new async conversion helpers, which changes how structured outputs are parsed and may affect task results/guardrail retries in async workflows. Risk is contained to async paths; sync behavior is preserved.
Overview
Fixes async workflows (
Task._aexecute_core/_ainvoke_guardrail_function) so structured output export no longer calls the synchronous converter and blocks the event loop.Adds an async converter pipeline in
utilities/converter.py(Converter.ato_pydantic/ato_jsonplusaconvert_to_model/ahandle_partial_json/aconvert_with_instructions) and wires it via a newTask._aexport_output, with expanded async test coverage ensuringllm.call()is never used on the async path.Written by Cursor Bugbot for commit 415b894. This will update automatically on new commits. Configure here.