feat: partially implements optimize_from_config#119
feat: partially implements optimize_from_config#119andrewklatzke wants to merge 2 commits intoaklatzke/AIC-2071/optimize-method-move-promptsfrom
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
| "The optimization config has no acceptance statements, judges, or ground truth " | ||
| "responses, and no on_turn callback was provided. At least one is required to " | ||
| "evaluate optimization results." | ||
| ) |
There was a problem hiding this comment.
Ground truth validation bypasses check but fails downstream
Medium Severity
The early validation in _build_options_from_config treats groundTruthResponses as a valid evaluation criterion (has_ground_truth bypasses the "no criteria" check), but ground truth is never passed to OptimizationOptions and that dataclass's __post_init__ only checks for judges or on_turn. When a config has ground truth but no judges and no on_turn, the early check passes, then OptimizationOptions(judges=judges or None, ...) converts the empty dict to None, and __post_init__ raises with the less helpful message "Either judges or on_turn must be provided" instead of the descriptive one mentioning ground truth.
Additional Locations (1)
There was a problem hiding this comment.
This is not yet implemented


Requirements
Describe the solution you've provided
Implements an LDApiClient that can talk to the Launchdarkly API to fetch configs and post results back to the optimization process. Runs is not yet implemented, so the post back will silently fail for now. Properly pulls the config from the LD and maps it to the correct options to run the optimization from the provided config key (with some options for handling the call, etc.).
This also adds validation on the response/input for the POST call
Additional context
There are two paths for running an optimization:
This implements the second path. None of the "core" logic for the optimization has changed.
Call to this method looks like: