fix: low code conversational agents to emit start tool call on approval#99
fix: low code conversational agents to emit start tool call on approval#99JoshParkSJ wants to merge 1 commit intomainfrom
Conversation
75d1dfb to
6722760
Compare
| resume_data = await self.chat_bridge.wait_for_resume() | ||
|
|
||
| await ( | ||
| self._emit_start_tool_call_on_confirmation_approval( |
There was a problem hiding this comment.
I don't think this is the right place to do this .. tool call confirmation is just one of multiple interruptable use-cases
There was a problem hiding this comment.
Where do you recommend this go? I have detailed justification on why I couldn't get langchain repo to work in the PR description, but runtime is (from my understanding) only place that can emit start tool call between AImessage.toolCall and toolMessage. We don't want socket logic in tool_node.py either
There was a problem hiding this comment.
@JoshParkSJ
If I'm reading this correctly, the current order of operations is AIMessage => startToolCall => startInterrupt (which is the expected, correct sequence), and what you're trying to achieve is AIMessage => startInterrupt => startToolCall.
If that's right, I'd push back on solving this in the runtime. The confusing UX you're describing, users seeing both tool call UI and interrupt UI simultaneously, is a client-side state machine problem. The client should be able to handle receiving startToolCall before startInterrupt without rendering both at the same time. Reordering backend events to paper over a client rendering issue is the wrong layer to fix this, and it pulls protocol sequencing logic into the runtime in a way that'll be hard to unpick later.
There was a problem hiding this comment.
It may be worth exploring how these guys https://github.com/ag-ui-protocol/ag-ui handle the same scenario
There was a problem hiding this comment.
That's a good point - I'll move this handling to CAS. Appreciate the feedback!
We need this logic in the runtime because the alternatives are
uipath-pythonbut we should try to keep transport logic only here. Not CAS protocol logic.uipath-langchainbut given langchain's lifecycle of AIMessage with toolcall + ToolMessage, there's no way to squeeze in an interrupt + start tool call event.Currently we
startToolCallafter Langchain's AIMessage with tool call attached to it. We have two options to squeeze in an interrupt here:startToolCallandstartInterruptafter Langchain's AIMessage with tool call attached to it, which can confuse users since they'll see both the tool call UI as well as the interrupt UI.startInterruptafter Langchain's AIMessage with tool call attached to it, defer thestartToolCalluntil we see ToolMessage being streamed, which can confuse users since they'll seestartToolCallAFTER the tool has already finished.These are our only options for the langchain repo, because there's nothing in between AiMessage with tool call and ToolMessage. What's between them is tool node, but we don't want to pollute CAS specific protocol logic in a langgraph tool node.
The "best" solution here, is to emit
startInterruptafter Langchain's AIMessage with tool call attached to it, emitstartToolCallfrom runtime layer (only layer that sits between interrupt resolving and is about to execute the langgraph tool node), and let tool node finish normally. We also emitendToolCallnormally by emitting it when we see ToolMessage. This fits user's expectations the best, and we already have logic for bridging CAS protocol events in the runtime layer.That's my thought process, let me know if you disagree / want to put this logic elsewhere, but I think this is the most optimal product UX and respects code boundry the most