Inside EncodableCallable we have a validation process wherein we check that the synthesized function has the right type signature. This check is done syntactically on the ast of the annotation.
In practice this is causing LLM calls to fail unnecessarily:
def foo() -> Optional[dict]:
pass
Will raise an exception of the LLM generates a program dict | None however mypy would not complain. We should either drop this syntactic checking, or switch to a proper mypy based validation.
@matthew9671 can produce a reproducible example
Inside
EncodableCallablewe have a validation process wherein we check that the synthesized function has the right type signature. This check is done syntactically on the ast of the annotation.In practice this is causing LLM calls to fail unnecessarily:
Will raise an exception of the LLM generates a program dict | None however mypy would not complain. We should either drop this syntactic checking, or switch to a proper mypy based validation.
@matthew9671 can produce a reproducible example